bedrock-kv / bedrock / b11e27ed0ab2dd8e9b4b6bc0494aad10c1cc13cf

82%

Build Type

push

github

Committed by web-flow

Commit Message

bedrock-s1zr: prune a shard's snapshots past a keep-last limit (#278)

Every snapshot an Olivine materializer wrote stayed in object storage
forever: the deletion primitive existed but nothing in `lib/` called it.
This change adds a keep-last-K retention policy, set by one manifest
param, and prunes a shard's older snapshots right after each successful
upload.

Ticket: bedrock-s1zr
Stacked on: #269 (bedrock-zi44/snapshot-upload-policy)

## Why

A materializer uploads a snapshot synchronously before an idle spin-down
and in a fire-and-forget task after a compaction cutover. Both write a
put-if-not-exists object under `s/{tag}/` keyed by inverted version;
neither removed anything. #269 throttled how often a snapshot is
written, but the objects still accumulated without bound, one per
spin-down in practice (bedrock-947).

Deleting a snapshot is safe because it is derived state: anything it
held can be rebuilt by replaying chunks forward from an older snapshot.
Chunks stay untouched (bedrock-wxf.6.11). Retention owes one thing: a
shard is never left with zero snapshots, since a cold start with no
baseline replays from version zero.

## What changed

- New pure policy `SnapshotRetention` with one field, `keep_last`, read
from the manifest param `snapshot_keep_last`. Only an explicit positive
integer sets it.
- The knob rides the same path as `idle_timeout`: manifest params,
worker startup opts, a new `State` field.
- Both upload sites prune immediately after the write returns `:ok`. No
scheduled prune exists.

A worker without the param deletes nothing and issues no listing. With
it set, an upload may log `Pruned snapshots past the retention limit` or
warn `Snapshot retention pruning failed`. Neither changes the upload's
result or the spin-down decision.

## How it works

The prune lists the shard's snapshots, newest-first because keys are
inverted versions, and the policy returns the Kth newest existing
version as a floor, or `:keep_all` when nothing is o... (continued)

Coverage Stats

25 of 25 new or added lines in 3 files covered. (100.0%)

33 existing lines in 2 files now uncovered.

7081 of 8774 relevant lines covered (80.7%)

1161.02 hits per line