clickhouse-rust: `Network("channel closed")` on bulk insert (3000+ rows, 155 columns)

[snailwei]

clickhouse-rust: Network("channel closed") on bulk insert (3000+ rows, 155 columns)

Crate: clickhouse v0.15.0
Repository: https://github.com/ClickHouse/rust-clickhouse
ClickHouse Server: 24.x (direct connection, no proxy)

Problem

Using the Inserter API to insert ~3000 rows with 155 columns reliably fails with Network("channel closed"). The same data inserts successfully via plain HTTP POST (reqwest, 0.1s).

Reproduction

use std::collections::HashMap;

let mut inserter = client
    .inserter::<Row>("cn_stock_order")
    .await?
    .with_setting("send_timeout", "600")
    .with_setting("max_insert_block_size", "800")
    .end()
    .await?;

for (&sid, features) in &features_map {
    inserter.write(&row).await?;
}
inserter.end().await?; // fails here: Network("channel closed")

Row struct: 157 fields (date Date, symbol String, 155 Float64 columns).

Server settings observed:

• http_send_timeout = 30s (socket-level, not overridable by client)
• http_receive_timeout = 30s
• send_timeout = 300s

Root Cause Analysis

The crate's Inserter uses chunked transfer encoding with an internal spawned tokio task that streams rows to the server. Between chunks, idle gaps form on the connection. When a gap exceeds the server's http_send_timeout (30s), the server closes the socket.

Key observations:

1. with_setting("send_timeout", "600") has no effect — http_send_timeout is enforced at the socket layer and cannot be overridden by client-side settings.
2. with_setting("max_insert_block_size", "800") with periodic end() helps smaller batches but still fails on ~3000 rows.
3. SSE dataset (~3000+ rows) times out consistently. SZE dataset (~3077 rows) sometimes succeeds, suggesting it's a timing-dependent race.
4. Plain HTTP POST with the same data (full body, single request) completes in 0.1s — the server can accept the data instantly when sent without chunking.

Expected Behavior

Inserting 3000 rows with 155 columns (~3 MB uncompressed TSV) should succeed without socket timeouts, as the ClickHouse server accepts the same data instantly via direct HTTP POST.

Workaround

Use reqwest with plain HTTP POST:

let query = "INSERT INTO cn_stock_order FORMAT TabSeparated";
let url = format!("http://host:8123/?user=default&query={}", urlencoding(query));
reqwest::Client::new()
    .post(&url)
    .body(tsv_body)
    .send()
    .await?;

Environment

• clickhouse crate: 0.15.0 (features: chrono, inserter, zstd)
• ClickHouse server: 24.x on 172.16.210.220:8123
• Rust: edition 2024, tokio 1.x (full features)
• No nginx/reverse proxy (direct ClickHouse connection confirmed)

[abonander]

See also #421

[pavel-rosputko]

We occasionally experience "channel closed" too. Do I understand correctly, that data isn't inserted in this case and subsequent successful commit will do nothing?

[snailwei]

It seems not fixed yet after two months