nbari / backup / 27150222772

push

github

data plane: content-addressed blob storage, run upload, and verify/repair

Builds the local write path on top of the metadata engine: a backup `run` now
compresses, encrypts, and writes content-addressed blobs to one or more
configured filesystem destinations, and a new `verify`/`--repair` command keeps
those destinations consistent with the catalog. Version 0.1.0. Unix only.

Storage & config
- content-addressed LocalStore: sharded layout (ab/cd/<id>), atomic temp+rename
  (unique per-writer temp names), overwrite semantics (replaces orphan blobs from
  interrupted runs), hex-only key validation (blocks path traversal), remove()
- per-backup destinations via -t/--to on `new`/`edit` (repeatable; filesystem now,
  s3:// recognized and skipped with a notice)

Run upload (Phase 1)
- seal each new file's content: zstd compress (raw fallback) + ChaCha20-Poly1305,
  content id + codec bound as AAD, per-content key wrapped to the backup public key
- dedup by keyed content id (one set query, not a blocking lookup per file); write
  to every destination; metadata-only when none
- bounded parallel seal/encrypt/store worker pool with progress
- interrupted-run safety: a version is completed only after its blobs are stored,
  so `view` shows the last complete snapshot and a re-run re-stores cleanly

Verify / repair
- existence check across destinations (no secrets); --repair restores missing
  blobs by copying from a healthy destination (key preserved) or re-sealing from
  source when gone everywhere (catalog key updated before blobs; tries duplicate
  sources; errors on a missing Files row; unrecoverable when source gone/changed)
- content ids checked concurrently with a bounded pool; results folded race-free,
  unrecoverable ids returned sorted

Pre-merge hardening
- bind content_id (content/file keys) and a fixed label (naming key) as AAD when
  wrapping keys, so a catalog wrapped-key row can't be reused under another id
- write the public key + sealed naming ... (continued)

1293 of 1479 new or added lines in 27 files covered. (87.42%)

3334 of 4152 relevant lines covered (80.3%)

15.08 hits per line