decentraland / catalyst-storage / 30227514701

push

github

feat!: fix silent-corruption bugs, drop unused modes, shrink the public surface (#119)

* feat!: fix silent-corruption bugs, drop unused modes, shrink the public surface

Findings from a full review of the library, plus the removals that a usage audit
across snapshots-fetcher, worlds-content-server, catalyst, deployments-to-sqs and
asset-bundle-registry showed to be safe.

Bugs fixed (each with a regression test):

* A store handed an already-consumed stream committed a 0-BYTE object and
  resolved. In a content-addressed store that is permanent: exist(id) answers
  true, so nothing re-fetches. The in-memory backend already rejected these;
  folder-based and S3 stored nothing and reported success. All three now reject
  with ERR_STREAM_PREMATURE_CLOSE.
* The .gzip id reservation was case-sensitive, so on a case-folding filesystem
  (APFS/NTFS/SMB) storing `<id>.GZIP` overwrote another id's compressed
  representation: that id then failed to inflate, reported a fabricated
  contentSize, and was enumerated while the id that clobbered it was not. The
  reserved staging directory had the same hole.
* allFileIds() could omit an id that held a complete representation for the whole
  enumeration, because the raw/gzip dedup was decided from a directory read and
  applied to a later one. A small directory is now decided from a single read; a
  large one confirms the skip against the filesystem.
* The S3 startup probe inferred "this principal cannot s3:ListBucket" from a 403
  AccessDenied — which is also what a scoped policy returns for a MISSPELLED
  bucket, turning a typo into a node that reported every id absent for its whole
  lifetime. Lenient reads are now the explicit report403AsAbsent option.
* Legacy no-rename range reads returned ZERO bytes to a paused-mode consumer.
* ENAMETOOLONG aborted a whole delete batch; S3 enumeration stopped after one
  page on endpoints that omit IsTruncated; the in-memory backend handed out its
  live stored buffer and val... (continued)

619 of 695 branches covered (89.06%)

Branch coverage included in aggregate %.

209 of 212 new or added lines in 13 files covered. (98.58%)

1147 of 1172 relevant lines covered (97.87%)

1218.99 hits per line