Skip to content

feat(jupyter): rewrite kernel in JS, drop zeromq/runtimelib deps#34083

Open
lunadogbot wants to merge 1 commit into
mainfrom
claude/deno-jupiter-js-rewrite-UBH0U
Open

feat(jupyter): rewrite kernel in JS, drop zeromq/runtimelib deps#34083
lunadogbot wants to merge 1 commit into
mainfrom
claude/deno-jupiter-js-rewrite-UBH0U

Conversation

@lunadogbot
Copy link
Copy Markdown
Contributor

Summary

Replaces the Rust-based ZMQ kernel implementation (using zeromq, runtimelib/jupyter_runtime, and jupyter-protocol crates) with a pure-JS ZMTP 3.1 implementation running on top of Deno.listen().

Architecture

  • Main OS thread — new cli/js/jupyter_kernel.js (~800 lines) owns all ZMQ/Jupyter protocol work: ZMTP 3.1 framing, NULL handshake, HMAC-SHA256 message signing, and all five channel loops (shell, control, iopub, heartbeat, stdin). Uses only non-blocking Deno.listen() TCP, so the event loop never blocks.
  • Background OS thread — runs the REPL session (user code evaluation, CDP, TypeScript). Created via a cloned CliMainWorkerFactory.
  • The two threads talk over tokio::mpsc channels. The main thread can interrupt user code at any time by calling IsolateHandle::terminate_execution() on the handle sent back from the background thread via a oneshot channel.

Changes

  • cli/js/jupyter_kernel.js — new JS file: ZMTP framing helpers, NULL handshake, Jupyter wire protocol (HMAC-SHA256 via crypto.subtle), RouterSocket/PubSocket server classes, startJupyterKernel() entry point
  • cli/ops/jupyter.rs — rewritten: deno_jupyter_kernel extension (get_connection_info, repl_evaluate, recv_iopub, interrupt, etc.) and deno_jupyter_repl extension (broadcast, print, create_png, get_buffer)
  • cli/tools/jupyter/mod.rs — new two-thread orchestration; REPL background thread sends isolate handle via oneshot so the kernel thread can interrupt it
  • cli/tools/jupyter/server.rs — deleted (all Rust ZMQ code gone)
  • cli/tools/jupyter/install.rs — replace jupyter_runtime::dirs with platform-specific path logic (XDG on Linux, ~/Library on macOS, %APPDATA% on Windows)
  • cli/lib/worker.rs, cli/worker.rs#[derive(Clone)] on worker factories so one factory can create workers on two threads
  • cli/cdp.rs — add missing Serialize/Deserialize derives needed by new ops
  • cli/js/40_jupyter.js — update enableJupyter() for new op signatures
  • runtime/js/99_main.js — update NOT_IMPORTED_OPS for new kernel ops
  • tests/integration/jupyter_client.rs — new pure-Rust ZMTP 3.1 test client (ReqSocket, DealerSocket, SubSocket, RouterSocket) replacing the zeromq crate dependency
  • tests/integration/jupyter_tests.rs — rewritten to use the new in-tree client
  • Cargo.toml / cli/Cargo.toml / tests/integration/Cargo.toml — remove zeromq, runtimelib, jupyter-protocol dependencies

Test plan

  • cargo check -p deno passes with no errors
  • cargo check --manifest-path tests/integration/Cargo.toml passes
  • cargo test jupyter — integration tests for heartbeat, kernel_info, execute_request, interrupt, shutdown, HTTP server
  • deno jupyter --install registers the kernel correctly on each platform
  • Manual: jupyter console --kernel deno connects and evaluates code

Generated by Claude Code

Replace the Rust ZMQ + zeromq/runtimelib/jupyter-protocol crate
implementation with a pure-JS ZMTP 3.1 implementation using
Deno.listen() for TCP.

Architecture:
- Main OS thread runs the new jupyter_kernel.js (ZMQ/Jupyter protocol,
  always non-blocking TCP I/O via Deno.listen())
- Background OS thread runs the REPL session (user code evaluation,
  CDP, TypeScript)
- Threads communicate via tokio mpsc channels; main thread can interrupt
  the background thread at any time via IsolateHandle::terminate_execution()

Changes:
- cli/js/jupyter_kernel.js: new ~800-line JS file implementing ZMTP 3.1
  framing, NULL handshake, Jupyter wire protocol (HMAC-SHA256), and
  all kernel channels (shell, control, iopub, heartbeat, stdin)
- cli/ops/jupyter.rs: rewritten ops for two-worker architecture
- cli/tools/jupyter/mod.rs: new two-thread orchestration with cloned
  CliMainWorkerFactory; background REPL thread sends isolate handle via
  oneshot channel so main thread can interrupt it
- cli/tools/jupyter/server.rs: deleted (all Rust ZMQ code removed)
- cli/tools/jupyter/install.rs: replace jupyter_runtime::dirs with
  platform-specific implementation
- cli/lib/worker.rs, cli/worker.rs: derive Clone on worker factories
- cli/cdp.rs: add Serialize/Deserialize derives needed by new ops
- cli/js/40_jupyter.js: update enableJupyter() for new op signatures
- runtime/js/99_main.js: update NOT_IMPORTED_OPS for new kernel ops
- tests/integration/jupyter_client.rs: new pure-Rust ZMTP 3.1 test client
  replacing zeromq
- tests/integration/jupyter_tests.rs: rewrite to use new test client
- Cargo.toml: remove zeromq, runtimelib, jupyter-protocol workspace deps
@lunadogbot lunadogbot force-pushed the claude/deno-jupiter-js-rewrite-UBH0U branch from 8dd7735 to 5b96996 Compare May 15, 2026 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant