decentraland / catalyst-storage / 30103367624

push

github

fix: store content atomically to avoid partial files on interrupted writes (#108)

* fix: store content atomically to avoid partial files on interrupted writes

storeStream wrote directly to the final content-addressed path, so a process
killed mid-write (oom, eviction, power loss) left a truncated or zero-byte file
there. its cleanup only ran on a stream error, not on a kill. because exist()
only checks that the path is present, that partial file was then treated as a
valid cached copy and never re-fetched.

write to a temp file in the same directory and rename it into place, which is
atomic within a filesystem, so a reader sees either the previous file or the
fully-written new one. remove the temp file on any failure, and skip temp files
in allFileIds so a crash-orphaned one is never enumerated as content. adds
rename to the fs component.

* fix: sweep orphaned temp files at startup and match them precisely

addresses review feedback on the atomic-store change:

- sweep temp files orphaned by an interrupted write once at startup, in the
  background and awaited by stop(), so a rare crash mid-write no longer leaks a
  partial temp file.
- identify temp files with a precise `<id>.<32-hex>.tmp` pattern (in both
  allFileIds and the sweep) so a real id that legitimately ends in `.tmp` is
  never skipped or removed.
- scope the storeStream comment to process death and note that power-loss
  durability would require an fsync, which is unnecessary for re-downloadable
  content-addressed files.

* fix: keep fs rename optional with a direct-write fallback

addresses review feedback on #108: requiring `rename` on the exported
IFileSystemComponent would be a breaking contract change for any custom
filesystem adapter. make `rename` optional and fall back to the original direct
write when it is absent, so existing implementers keep compiling and working;
the bundled createFsComponent still provides rename and takes the atomic path.

* fix: stage atomic writes in... (continued)

224 of 259 branches covered (86.49%)

Branch coverage included in aggregate %.

293 of 305 new or added lines in 3 files covered. (96.07%)

626 of 645 relevant lines covered (97.05%)

88.85 hits per line