iotaledger / iota / 26485628984

67%

Build Type

push

github

Committed by web-flow

Commit Message

upstream(iota-core): cache signed_effects_digests in memory (#11612)

# Description of change

Port of MystenLabs/sui#26431 ("Cache `signed_effects_digests` in memory
to avoid disk reads").

`get_signed_effects_digest(&tx_digest)` is called on every transaction
this validator processes and reads from the RocksDB
`signed_effects_digests` column family. The cardinality of that table is
bounded by the set of transactions for which the validator has signed
effects but which haven't yet been finalized in a checkpoint — small
(hundreds to low thousands), but the call is high-frequency and the vast
majority of lookups return `None`.

Adds `signed_effects_digests_cache: DashMap<TransactionDigest,
TransactionEffectsDigest>`:

- Populated from disk at epoch startup (one-time scan of the column
family).
- Updated on every successful write to disk.
- Pruned at the same time as the on-disk batch delete in the
finalized-checkpoint path.
- `get_signed_effects_digest` reads only from the in-memory map.

Disk writes are unchanged — crash recovery still re-populates the cache
from disk on next startup.

A test-mode-gated consistency check compares cache vs disk on every
read, with a one-second retry to handle the write-before-cache-insert
race (the disk batch is flushed first, then the cache is inserted; a
concurrent read in the millisecond window between would otherwise
spuriously panic). On divergence, the assertion panics with the
offending tx digest.

The patch mirrors upstream almost line-for-line. IOTA's
`AuthorityPerEpochStore::new` returns `IotaResult<Arc<Self>>` (not bare
`Arc<Self>`), and the cleanup function is
`notify_commit_of_finalized_checkpoint` (not
`handle_finalized_checkpoint`) — code mapped accordingly.

## Links to any relevant issues

- Upstream: MystenLabs/sui#26431

## How the change has been tested

- [ ] Basic tests (linting, compilation, formatting, unit/integration
tests)
- [ ] Patch-specific tests (correctness, functionality coverage)
- [ ]... (continued)

Coverage Stats

14 of 21 new or added lines in 1 file covered. (66.67%)

5309 existing lines in 139 files now uncovered.

152188 of 227510 relevant lines covered (66.89%)

466477.49 hits per line