decentraland / snapshots-fetcher / 27568386375

push

github

fix: security, correctness and performance hardening + regression tests (#199)

* fix: security, correctness and performance hardening + regression tests

Hardening pass over the download, sync and parsing paths, with regression
tests for each behavioral change.

Security
- Validate content hashes (alphanumeric CIDs) before building filesystem
  paths or storage keys, preventing path traversal from untrusted hashes.
- Cap decompressed download size to bound gzip bombs / runaway responses.
- Add a socket-inactivity timeout to content downloads so stalled
  connections reject instead of hanging forever.
- Resolve relative redirects against the redirecting URL and reject
  non-http(s) redirect targets.
- Cap buffered JSON response size in fetchJson.
- Bound per-entity content-file download concurrency.
- Cap per-file invalid-line error logging to avoid log flooding.
- Validate the /snapshots response shape, dropping malformed entries.

Bug fixes
- pointerChangesStartingTimestamp: treat a last timestamp of 0 (genesis)
  as valid (=== undefined instead of falsy).
- Guard Math.max over an empty snapshot list (fall back to genesis).
- downloadContentFile: dedup on the content hash, not the file path.
- Guard JSON.parse of missing entity content and malformed snapshot lines.
- Serialize queued sync jobs correctly so the queue drains past the 2nd
  job under overlapping syncWithServers calls.
- exponential-fallof-retry: reset the started flag once the loop exits so
  isStopped() is accurate and the component can restart; make the retry
  sleep abortable by stop().
- Warm up the deployer at genesis (minStartingPoint !== undefined).
- Tolerate a deployer that over-reports markAsDeployed (>= with a
  save-once guard) so a snapshot can't be re-processed forever.

Performance
- Fetch each server's snapshots concurrently instead of serially.
- Batch the processed-snapshot lookup into a single storage call and run
  the per-snapshot decisions with bounded concurrency... (continued)

166 of 219 branches covered (75.8%)

Branch coverage included in aggregate %.

179 of 195 new or added lines in 11 files covered. (91.79%)

685 of 759 relevant lines covered (90.25%)

16.84 hits per line