iotaledger / iota / 29217649699

69%

Build Type

push

github

Committed by web-flow

Commit Message

feat(core): replace `full_checkpoint_content` table with a size-bounded in-memory cache (#12146)

## Description

Every checkpoint synced via p2p state sync had its full contents
(transactions + effects + signatures) written to disk twice: once into
the perpetual store (transactions/effects) and once as a serialized blob
into the `full_checkpoint_content` RocksDB table — only for the blob to
be deleted again ~5,000 checkpoints later by the checkpoint executor.
This is pure write amplification plus tombstone/compaction churn for
data that lives minutes to hours.

The table was purely a read optimization: every consumer already has a
complete fallback that reconstructs `FullCheckpointContents` from the
permanent `checkpoint_content` table plus the transaction and effects
stores. Validators never populated the table at all (they don't sync
contents via p2p), so the reconstruction path has always been the
production serving path on validators.

This PR replaces the table with a size-bounded in-memory cache:

- **New `FullCheckpointContentsCache`**
(`iota-core/src/checkpoints/full_checkpoint_contents_cache.rs`): keyed
by sequence number with a contents-digest index, byte-accounted via
`bcs::serialized_size`, evicting lowest sequence numbers first when over
budget (keeps the newest window — what tip-following peers request). It
serves the checkpoint executor's bulk transaction loads, peer requests
via state sync, and the state-sync re-check.
- **New config knob** `full-checkpoint-contents-cache-size-mb` in
`NodeConfig` (default **1024 MiB**, `0` disables the cache; consumers
then always use the reconstruction fallback).
- **Dropped column families**: `full_checkpoint_content` and
`checkpoint_sequence_by_contents_digest` are marked
`#[deprecated_db_map]` and auto-dropped on the first open after upgrade.
No migration needed — the data was a short-lived cache and is
reconstructable.
- **Removed** the hardcoded `NUM_SAVED_FULL_CHECKPOINT_CONTENTS = 5_000`
pruni... (continued)

Coverage Stats

481 of 485 new or added lines in 7 files covered. (99.18%)

635 existing lines in 13 files now uncovered.

163489 of 239273 relevant lines covered (68.33%)

456225.45 hits per line