decentraland / deployments-to-sqs / 24615061203

90%

push

github

fix: stage storage streams to disk to survive S3 mid-download aborts (#523)

* fix: stage storage streams to disk to survive S3 mid-download aborts

During catalyst re-sync the download queue saturates and snapshot
iteration pauses on onSizeLessThan(1000). The S3 GET socket backing
the snapshot stream sits idle and AWS closes it, emitting
`error: aborted` on the IncomingMessage. readline re-emits it on its
Interface which has no listener, and the process dies under
--abort-on-uncaught-exception.

Wrap the storage component so retrieve().asStream() first pipelines
the body to a unique file in os.tmpdir() and returns a
fs.createReadStream of it. The drain runs at full speed so the S3
socket can't idle out, and a mid-download abort rejects asStream()
for snapshot-fetcher's retry layer to handle instead of escaping as
an uncaught exception.

Also bump @types/node and typescript to match the Node 24 runtime.

* fix: address review feedback on storage wrapper

- Cap stageToDisk with AbortSignal.timeout to prevent hangs on stalled
  (alive-but-not-producing) S3 streams.
- Use `catch (error: unknown)` with instanceof narrowing for type safety.
- Comment the asRawStream bypass so future maintainers know to wrap it
  if a consumer starts using it.
- Note in the forwarded-methods comment why we prefer explicit `bind`
  over `{ ...inner }` spread.
- Switch afterEach to resetAllMocks so mockResolvedValueOnce queues do
  not leak between tests; re-seed logsMock.getLogger in beforeEach.
- Add a write-failure test for the disk-full path.

* fix: log and guard temp-file cleanup in asStream

- Log unlink failures in the readStream close handler so repeated
  deletion errors surface instead of silently accumulating temp files.
- Also listen on the readStream 'error' event as a belt-and-suspenders
  against future Node changes in event ordering, guarded by a single
  `cleaned` flag to keep the unlink idempotent and avoid spurious
  ENOENT noise.
- Document the remaining ... (continued)

28 of 38 branches covered (73.68%)

Branch coverage included in aggregate %.

36 of 39 new or added lines in 2 files covered. (92.31%)

205 of 221 relevant lines covered (92.76%)

2.48 hits per line