Lotus Chain Export Times Out

[jamerduhgamer]

Checklist

• This is not a security-related bug/issue. If it is, please follow please follow the security policy.
• I have searched on the issue tracker and the lotus forum, and there is no existing related issue or discussion.
• I am running the Latest release, the most recent RC(release canadiate) for the upcoming release or the dev branch(master), or have an issue updating to any of these.
• I did not make any code changes to lotus.

Lotus component

• lotus daemon - chain sync
• lotus fvm/fevm - Lotus FVM and FEVM interactions
• lotus miner/worker - sealing
• lotus miner - proving(WindowPoSt/WinningPoSt)
• lotus JSON-RPC API
• lotus message management (mpool)
• Other

Lotus Version

lotus version
Daemon:  1.30.0+mainnet+git.9c1964e8a+api1.5.0
Local: lotus version 1.30.0+mainnet+git.9c1964e8a

Repro Steps

1. Run 'lotus chain export --recent-stateroots=900 --skip-old-msgs snapshot_mainnet_2025-02-06'
2. See error:

2025-02-07T09:08:58.155Z WARN rpc [email protected]/websocket.go:871 websocket long time no response {"lastAction": "pong", "time": 14.236651866}
2025-02-07T10:42:29.823Z ERROR rpc [email protected]/websocket.go:850 Connection timeout {"remote": "127.0.0.1:1234", "lastAction": ""}
2025-02-07T10:42:29.824Z WARN rpc [email protected]/websocket.go:861 failed to write close message: write tcp 127.0.0.1:51338->127.0.0.1:1234: use of closed network connection
2025-02-07T10:42:29.824Z WARN rpc [email protected]/websocket.go:864 websocket close error {"error": "close tcp 127.0.0.1:51338->127.0.0.1:1234: use of closed network connection"}
ERROR: incomplete export (remote connection lost?) '

Describe the Bug

Trying to create a snapshot of the chain using the following command

lotus chain export --recent-stateroots=900 --skip-old-msgs snapshot_mainnet_2025-02-06

and it times out after a few days with the incomplete export error.

Logging Information

lotus log level was set to debug for the course of the chain export attempt. the file is >9 GB so let me know what log lines to look for!

[rvagg]

I had a quick look at this and added some logging per-chunk and got nothing before my node borked and I gave up (disk problems, possibly unrelated). But I was surprised that I didn't even get any initial chunks through to the client, I thought it should have at least sent a CAR header. Perhaps it's buffering more than I imagined and then needed to do an initial walk, but my expectation from glancing through the code is that this is a standard traversal. The channel from the server side is delimited in 1MiB chunks so it should be spitting these across the websocket to the client.

Needs deeper investigation, it really should be able to write the file as it traverses, not buffer up a whole bunch and then do it in one go.