Add `proxy` and `onion` support for `ckb-network` #4733

eval-exec · 2024-12-02T05:15:52Z

What problem does this PR solve?

This PR want to add socks5 proxy and onion support for ckb.

Add network.proxy and network.onion config:

# [network.proxy]
### The proxy_url is user-specified SOCKS5 address, like: socks5://127.0.0.1:7891
# proxy_url = "socks5://127.0.0.1:9050"

### Toggle random authentication for the proxy server.
### Default setting: true
###
### When enabled (true), CKB will generate a random username and password for each connection
### to the proxy server.
### For a Tor Server, this increases privacy by ensuring the proxy cannot correlate
### multiple connections.
### Refer to the 'IsolateSOCKSAuth' section in the following documentation:
### https://manpages.debian.org/testing/tor/torrc.5.en.html
###
### Important Note:
### If you are using a non-Tor SOCKS5 proxy that supports, but does not require authentication,
### enabling this may lead to connection rejections. In such cases, set proxy_random_auth to false.
# proxy_random_auth = true

# [network.onion]
### If CKB needs to run behind the Onion network,
### the user should start a Tor server and set proxy_url to the Tor server's address.
### then MUST set [network.proxy.proxy_url] to tor server's address.
### The default Tor SOCKS5 proxy URL is socks5://127.0.0.1:9050.
### For more information on configuring Tor, visit: https://linux.die.net/man/1/tor

### Enable or disable listening on the Onion network.
### When set to true, CKB will accept incoming connections over Tor.
### You can get the onion listening address in log, or by local_node_info RPC
# listen_on_onion = true

### The onion service will proxy incoming traffic to `p2p_listen_address`.
### If the CKB's peer-to-peer listen address is not set to the default 127.0.0.1
### with the port specified in `[network].listen_addresses` for IPv4, you should configure this field.
# p2p_listen_address = "127.0.0.1:8115"

### The address and port of the Tor SOCKS5 proxy server that CKB will use to establish
### outbound connections to other Onion services. If not set, it will use network.proxy.proxy_url
# onion_server = "127.0.0.1:9050"

### The address and port of the Tor ControlPort, which allows CKB to communicate with
### the Tor process for tasks like creating Onion services.
### Default: "127.0.0.1:9051"
# tor_controller = "127.0.0.1:9051"

### The password for authenticating with the Tor ControlPort.
### This should match the 'HashedControlPassword' set in your Tor configuration.
### Leave empty if no authentication is required.
# tor_password = ""

Start a tor server: tor --SocksPort 9050 --ControlPort 9051, finaly, start the ckb node, if network.onion.listen_on_onion is true, it will auto connect to the tor server, and use tor control port to create onion service.

Related changes

Add ProxyConfig and OnionConfig to network config
Support socks5 authentication like: socks5://<username>:<password>@server_host:port
Support connect to onion ckb node, and listen on the onion hidden network
[WIP] Integration test for ckb-onion
- node1(onion) can succesfully connect node2(node2) in the onion network
- noode1(onion) can succesfully connect to node2(non-onion)
- node1(onion) connect node0(onion), node2(onion) connect to node0, expect node1 and node2 can reach each other
- node1(non-onion) can use [network.proxy] via a normal socks5 proxy
ckb-onion related configuration's documentation in ckb.toml

Note:
Need to setup promethues for tor process: https://gitlab.torproject.org/tpo/onion-services/onionprobe

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code ci-runs-only: [ quick_checks,linters ]

Side effects

None

Release note

Title Only: Include only the PR title in the release note.

eval-exec · 2024-12-18T08:39:30Z

nervosnetwork/tentacle#392

eval-exec · 2025-11-13T06:42:32Z

Just rebased onto develop branch.

Add Tor package installation to macOS and Windows CI workflows for Tor integration testing.

Add Go toolchain installation to macOS and Ubuntu CI workflows for building obfs4 proxy.

Add obfs4 submodule for obfuscation proxy support and Makefile targets for building obfs4proxy.

Add ckb-onion crate and torut dependency for Tor integration, and update tentacle dependencies.

Add NetworkAddresses struct to manage onion addresses alongside regular network addresses.

Add proxy module with URL validation for Tor proxy configuration and error handling.

Add onion service configuration options to network configuration for Tor integration.

Update network service builder to support proxy and onion service configuration.

Add ckb-onion crate for managing onion service integration with Tor network.

Update peer store to properly handle and store onion addresses for Tor network peers.

Update identify protocol to properly advertise and discover onion addresses for Tor network peers.

Update network state to manage public addresses including onion addresses for Tor integration.

Add onion service configuration options to network configuration and update ckb.toml template.

Signed-off-by: Eval EXEC <[email protected]>

eval-exec · 2025-11-14T14:24:49Z

I removed the struct NetworkAddresses because it was unnecessary. Then I rebased onto develop branch.

Invite @driftluo @quake @zhangsoledad @doitian @chenyukang review again.

eval-exec · 2025-11-17T06:56:15Z

In latest commit: 48013a4

I move "tor"/"onion" related instructions from [network.proxy] to [network.onion] @quake

quake · 2025-11-17T09:37:37Z

util/onion/src/onion_service.rs

+        ))
+    })?;
+    file.write_all(
+        &base64::engine::general_purpose::STANDARD


can we write the key as binary directly to the file, why need base64 encoding/decoding here?

Yes we can.
I use base64 encoding/decoding, because bitcoin use base64 encoding/decoding, I followed bitcoin's style:

❯ cat ~/.bitcoin/onion_v3_private_key ED25519-V3:aCSgwUGBquFBdiQ5S3DN3ZZ64O7tMqrtkwji7huY702sFmZNiVqsyAc4f53xF9ecDX7QrcM2PxuJePn72jFFnw==%

quake · 2025-11-17T10:08:16Z

util/onion/src/onion_service.rs

+            loop {
+                tokio::select! {
+                    _ = ticker.tick() => {
+                        let uptime = tor_controller.get_uptime().await;


it looks like you are using get_uptime as a heartbeat, since the Tor control connection is just a TCP stream, you can rely on TCP keepalive instead. Once keepalive is enabled, the OS will automatically detect when the Tor control port goes down, whether Tor crashes, restarts, or the connection drops.

Keepalive related code may be put in TorController::new.

it looks like you are using get_uptime as a heartbeat

Yes.

you can rely on TCP keepalive instead

If we rely solely on TCP keep-alive, we’d still need to keep reading or writing the socket to detect failure — and doing so may interfere with the Tor control-port protocol by consuming unexpected data. Hence I prefer an application-level heartbeat via tor_contoller.get_uptime().

eval-exec · 2025-11-17T15:22:48Z

@quake In the latest commit: 62e959b , I added onion_external_port: u16 to [network.onion] to support configuring onion service's external port.

chenyukang · 2025-11-18T02:12:11Z

network/src/protocols/identify/protocol.rs

-            listen_addrs.push(Multiaddr::try_from(addr.bytes().raw_data().to_vec()).ok()?)
+            match Multiaddr::try_from(addr.bytes().raw_data().to_vec()) {
+                Ok(multi_addr) => {
+                    listen_addrs.push(multi_addr);


any specific reason for ignore this error?

chenyukang · 2025-11-18T02:26:51Z

resource/ckb.toml

+### the user should start a Tor server and set proxy_url to the Tor server's address.
+### then MUST set [network.proxy.proxy_url] to tor server's address.


this two line have duplicated meanning?

resource/ckb.toml

chenyukang · 2025-11-18T02:32:02Z

resource/ckb.toml


+# [network.proxy]
+### The proxy_url is user-specified SOCKS5 address, like: socks5://127.0.0.1:7891
+# proxy_url = "socks5://127.0.0.1:9050"


this option will be ignored if user already set onion_server?

chenyukang · 2025-11-18T02:44:05Z

test/src/lib.rs

+        "{} --bin {} --obfs4proxy-bin {} --port {} {}",
        canonicalize_path(env::args().next().unwrap_or_else(|| "ckb-test".to_string())).display(),
        canonicalize_path(global::binary()).display(),
+        canonicalize_path(global::obfs4proxy_binary()).display(),


have you checked out https://github.com/jmwample/ptrs ?
I'm not sure, it's just a crate and may need more investigation on it.

eval-exec force-pushed the exec/proxy branch 5 times, most recently from fe5d648 to e55f631 Compare December 5, 2024 03:37

eval-exec force-pushed the exec/proxy branch 10 times, most recently from de239fd to f9f0733 Compare December 24, 2024 06:15

eval-exec mentioned this pull request Dec 30, 2024

IdentifyMessage::decode allow failure on Unsupported MultiAddr #4759

Closed

eval-exec force-pushed the exec/proxy branch 3 times, most recently from 587e905 to 865759d Compare December 31, 2024 08:49

zhangsoledad force-pushed the develop branch from 1de1447 to 7480a2a Compare January 7, 2025 09:12

eval-exec force-pushed the exec/proxy branch 3 times, most recently from 260c2fe to c0f57bb Compare January 21, 2025 10:45

eval-exec added the m:network label Jan 21, 2025

eval-exec force-pushed the exec/proxy branch 5 times, most recently from 6169766 to 12e27f6 Compare January 24, 2025 06:29

eval-exec added 17 commits November 14, 2025 22:19

ci: add Tor installation to GitHub Actions workflows

24cab08

Add Tor package installation to macOS and Windows CI workflows for Tor integration testing.

ci: add Go toolchain setup to GitHub Actions workflows

1775125

Add Go toolchain installation to macOS and Ubuntu CI workflows for building obfs4 proxy.

build: add obfs4 submodule and build targets

5fcaaa3

Add obfs4 submodule for obfuscation proxy support and Makefile targets for building obfs4proxy.

deps: add onion/tor dependencies

186c703

Add ckb-onion crate and torut dependency for Tor integration, and update tentacle dependencies.

network: add NetworkAddresses struct for onion addresses

547ffbd

Add NetworkAddresses struct to manage onion addresses alongside regular network addresses.

network: add proxy URL validation and configuration

7c67295

Add proxy module with URL validation for Tor proxy configuration and error handling.

config: add onion service configuration handling

a6a9417

Add onion service configuration options to network configuration for Tor integration.

network: update service builder for proxy support

4d458ed

Update network service builder to support proxy and onion service configuration.

feat: add onion crate for Tor integration

a29f0ca

Add ckb-onion crate for managing onion service integration with Tor network.

network: update peer store to handle onion addresses

6292520

Update peer store to properly handle and store onion addresses for Tor network peers.

network: update identify protocol for onion address sharing

d3772c5

Update identify protocol to properly advertise and discover onion addresses for Tor network peers.

network: update network state for public address management

a81b99e

Update network state to manage public addresses including onion addresses for Tor integration.

config: add onion service configuration options

e11c0b6

Add onion service configuration options to network configuration and update ckb.toml template.

log: extract modify_logger_filter to mute fast-socks5 log

fa04100

Apply code readbility suggestion

83976d3

Signed-off-by: Eval EXEC <[email protected]>

Change MultiAddr to Multiaddr

e9af5b4

Signed-off-by: Eval EXEC <[email protected]>

ckb-network: Remove struct NetworkAddresses

149d2ce

eval-exec force-pushed the exec/proxy branch from 738892a to 149d2ce Compare November 14, 2025 14:19

config: move [network.proxy] onion insturction to [network.onion]

48013a4

quake reviewed Nov 17, 2025

View reviewed changes

onion: support config onion service port by onion_external_port

62e959b

chenyukang reviewed Nov 18, 2025

View reviewed changes

resource/ckb.toml Show resolved Hide resolved

chenyukang reviewed Nov 18, 2025

View reviewed changes

		### the user should start a Tor server and set proxy_url to the Tor server's address.
		### then MUST set [network.proxy.proxy_url] to tor server's address.

Add proxy and onion support for ckb-network #4733

Are you sure you want to change the base?

Add proxy and onion support for ckb-network #4733

Conversation

eval-exec commented Dec 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Related changes

Check List

Release note

Uh oh!

eval-exec commented Dec 18, 2024

Uh oh!

eval-exec commented Nov 13, 2025

Uh oh!

eval-exec commented Nov 14, 2025

Uh oh!

eval-exec commented Nov 17, 2025

Uh oh!

quake Nov 17, 2025

Choose a reason for hiding this comment

Uh oh!

eval-exec Nov 17, 2025

Choose a reason for hiding this comment

Uh oh!

quake Nov 17, 2025

Choose a reason for hiding this comment

Uh oh!

eval-exec Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eval-exec commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chenyukang Nov 18, 2025

Choose a reason for hiding this comment

Uh oh!

chenyukang Nov 18, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

chenyukang Nov 18, 2025

Choose a reason for hiding this comment

Uh oh!

chenyukang Nov 18, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Add `proxy` and `onion` support for `ckb-network` #4733

Add `proxy` and `onion` support for `ckb-network` #4733

eval-exec commented Dec 2, 2024 •

edited

Loading

eval-exec Nov 17, 2025 •

edited

Loading

eval-exec commented Nov 17, 2025 •

edited

Loading