iotaledger / iota / 18299309643

push

github

feat(consensus): add shard reconstructor component to Starfish (#8751)

# Description of change

This PR introduces a new component to Starfish consensus, called
**ShardReconstructor**.
Its handle becomes part of an authority node.

Upon receiving a new block bundle via streaming and verifying its
components, the authority service component sends **transaction
messages** to the ShardReconstructor:
- `ShardMessage` – a single shard of encoded transaction data
corresponding to some block reference and transaction commitment
- `FullTransactionMessage` – a full transaction, that indicates that
these transaction data was verified against the block header.

The ShardReconstructor:
- Collects shards per block (`ShardAccumulator`). 
- Filters shards for blocks that are already processed (e.g. with
`FullTransactionMessage`)
- Detects when enough shards (`info_length`) are available to
reconstruct
- Offloads reconstruction jobs to a specified number async workers
- Verifies commitments against reconstructed transactions
- Forwards verified transactions to the core
(`CoreThreadDispatcher::add_transactions`) if the respected headers are
already part of the dag state
- Periodically evicts old state based on the DAG’s GC round

Unit and integration tests cover:
- Reconstruction only after `info_length` shards
- Stopping shard collection when a full transaction message is received
- Multi-round recovery where one authority is blocked, verifying
reconstruction from other authorities

## Links to any relevant issues

Fixes #8673 

## How the change has been tested

- [x] Basic tests (linting, compilation, formatting, unit/integration
tests)
- [x] Patch-specific tests (correctness, functionality coverage)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] I have checked that new and existing unit tests pass locally with
my changes

### Release Notes

- [x] Protocol: Starfish consensus allows for decoding of the
transaction data from ... (continued)

914 of 1003 new or added lines in 15 files covered. (91.13%)

231398 of 351459 relevant lines covered (65.84%)

2386905.0 hits per line