decentraland / deployments-to-sqs / 24614481170

90%

main: 90%

Pull #523

github

fix: stage storage streams to disk to survive S3 mid-download aborts

During catalyst re-sync the download queue saturates and snapshot
iteration pauses on onSizeLessThan(1000). The S3 GET socket backing
the snapshot stream sits idle and AWS closes it, emitting
`error: aborted` on the IncomingMessage. readline re-emits it on its
Interface which has no listener, and the process dies under
--abort-on-uncaught-exception.

Wrap the storage component so retrieve().asStream() first pipelines
the body to a unique file in os.tmpdir() and returns a
fs.createReadStream of it. The drain runs at full speed so the S3
socket can't idle out, and a mid-download abort rejects asStream()
for snapshot-fetcher's retry layer to handle instead of escaping as
an uncaught exception.

Also bump @types/node and typescript to match the Node 24 runtime.

28 of 35 branches covered (80.0%)

Branch coverage included in aggregate %.

29 of 29 new or added lines in 2 files covered. (100.0%)

198 of 211 relevant lines covered (93.84%)

2.47 hits per line