feat(jupyter): rewrite kernel in JS, drop zeromq/runtimelib deps#34083
Open
lunadogbot wants to merge 1 commit into
Open
feat(jupyter): rewrite kernel in JS, drop zeromq/runtimelib deps#34083lunadogbot wants to merge 1 commit into
lunadogbot wants to merge 1 commit into
Conversation
Replace the Rust ZMQ + zeromq/runtimelib/jupyter-protocol crate implementation with a pure-JS ZMTP 3.1 implementation using Deno.listen() for TCP. Architecture: - Main OS thread runs the new jupyter_kernel.js (ZMQ/Jupyter protocol, always non-blocking TCP I/O via Deno.listen()) - Background OS thread runs the REPL session (user code evaluation, CDP, TypeScript) - Threads communicate via tokio mpsc channels; main thread can interrupt the background thread at any time via IsolateHandle::terminate_execution() Changes: - cli/js/jupyter_kernel.js: new ~800-line JS file implementing ZMTP 3.1 framing, NULL handshake, Jupyter wire protocol (HMAC-SHA256), and all kernel channels (shell, control, iopub, heartbeat, stdin) - cli/ops/jupyter.rs: rewritten ops for two-worker architecture - cli/tools/jupyter/mod.rs: new two-thread orchestration with cloned CliMainWorkerFactory; background REPL thread sends isolate handle via oneshot channel so main thread can interrupt it - cli/tools/jupyter/server.rs: deleted (all Rust ZMQ code removed) - cli/tools/jupyter/install.rs: replace jupyter_runtime::dirs with platform-specific implementation - cli/lib/worker.rs, cli/worker.rs: derive Clone on worker factories - cli/cdp.rs: add Serialize/Deserialize derives needed by new ops - cli/js/40_jupyter.js: update enableJupyter() for new op signatures - runtime/js/99_main.js: update NOT_IMPORTED_OPS for new kernel ops - tests/integration/jupyter_client.rs: new pure-Rust ZMTP 3.1 test client replacing zeromq - tests/integration/jupyter_tests.rs: rewrite to use new test client - Cargo.toml: remove zeromq, runtimelib, jupyter-protocol workspace deps
8dd7735 to
5b96996
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the Rust-based ZMQ kernel implementation (using
zeromq,runtimelib/jupyter_runtime, andjupyter-protocolcrates) with a pure-JS ZMTP 3.1 implementation running on top ofDeno.listen().Architecture
cli/js/jupyter_kernel.js(~800 lines) owns all ZMQ/Jupyter protocol work: ZMTP 3.1 framing, NULL handshake, HMAC-SHA256 message signing, and all five channel loops (shell, control, iopub, heartbeat, stdin). Uses only non-blockingDeno.listen()TCP, so the event loop never blocks.CliMainWorkerFactory.tokio::mpscchannels. The main thread can interrupt user code at any time by callingIsolateHandle::terminate_execution()on the handle sent back from the background thread via aoneshotchannel.Changes
cli/js/jupyter_kernel.js— new JS file: ZMTP framing helpers, NULL handshake, Jupyter wire protocol (HMAC-SHA256 viacrypto.subtle),RouterSocket/PubSocketserver classes,startJupyterKernel()entry pointcli/ops/jupyter.rs— rewritten:deno_jupyter_kernelextension (get_connection_info, repl_evaluate, recv_iopub, interrupt, etc.) anddeno_jupyter_replextension (broadcast, print, create_png, get_buffer)cli/tools/jupyter/mod.rs— new two-thread orchestration; REPL background thread sends isolate handle via oneshot so the kernel thread can interrupt itcli/tools/jupyter/server.rs— deleted (all Rust ZMQ code gone)cli/tools/jupyter/install.rs— replacejupyter_runtime::dirswith platform-specific path logic (XDG on Linux,~/Libraryon macOS,%APPDATA%on Windows)cli/lib/worker.rs,cli/worker.rs—#[derive(Clone)]on worker factories so one factory can create workers on two threadscli/cdp.rs— add missingSerialize/Deserializederives needed by new opscli/js/40_jupyter.js— updateenableJupyter()for new op signaturesruntime/js/99_main.js— updateNOT_IMPORTED_OPSfor new kernel opstests/integration/jupyter_client.rs— new pure-Rust ZMTP 3.1 test client (ReqSocket,DealerSocket,SubSocket,RouterSocket) replacing thezeromqcrate dependencytests/integration/jupyter_tests.rs— rewritten to use the new in-tree clientCargo.toml/cli/Cargo.toml/tests/integration/Cargo.toml— removezeromq,runtimelib,jupyter-protocoldependenciesTest plan
cargo check -p denopasses with no errorscargo check --manifest-path tests/integration/Cargo.tomlpassescargo test jupyter— integration tests for heartbeat, kernel_info, execute_request, interrupt, shutdown, HTTP serverdeno jupyter --installregisters the kernel correctly on each platformjupyter console --kernel denoconnects and evaluates codeGenerated by Claude Code