iotaledger / iota / 31919687752

70%

Build Type

push

github

Committed by web-flow

Commit Message

feat(core+network): state sync improvements (#12455)

# Description of change

Speeds up catch-up from the checkpoint archive, bounds the disk it can
take up while doing so, and makes the checkpoint execution pipeline's
bottleneck visible. Six independent changes:

**Archive sync: verify in parallel, insert in batches** (`iota-network`,
`iota-storage`, `iota-types`, `iota-core`)

- The archive sync worker pool ran a single worker that verified
authority signatures, verified content digests, and inserted each
checkpoint sequentially — capping sync at roughly one core. Workers now
run in parallel (new `verify-concurrency` config, default: CPU cores)
and do only the order-independent verification; a new reducer receives
the results in sequence order, chain-checks each against its
predecessor, and inserts them.
- Checkpoints at the head of an epoch can reach a worker before the
previous epoch's last checkpoint is committed, so their committee isn't
in the store yet. Signature verification for those is deferred to the
reducer, where the committee is guaranteed to be available, and runs as
one batched signature verification per epoch (new
`CertifiedCheckpointSummary::batch_verify_authority_signatures`, which
falls back to verifying one by one so a failure still names the
checkpoint).
- A worker's signature and digest verification runs in `spawn_blocking`:
with one worker per core it would otherwise hold up checkpoint execution
on the same runtime, and execution progress gates how far sync may run
ahead.
- The reducer commits up to 500 checkpoints per store insertion via a
new `WriteStore::try_insert_synced_checkpoints`. The `RocksDbStore`
override writes all summaries in one batch, all contents rows in one
batch, and the synced watermark once per batch — still notifying waiters
of every checkpoint, which the checkpoint executor relies on. The
default trait implementation keeps the one-at-a-time behaviour for other
stores.
- Batches close at epoch boundaries... (continued)

Coverage Stats

321 of 488 new or added lines in 11 files covered. (65.78%)

538 existing lines in 25 files now uncovered.

169447 of 243111 relevant lines covered (69.7%)

467070.16 hits per line