iotaledger / iota / 18299309643
64%

DEFAULT BRANCH: develop
Ran
Jobs 1
Files 1432
Run time 3min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 65.839% (+0.09%) from 65.754%
18299309643

push

github

web-flow 
feat(consensus): add shard reconstructor component to Starfish (#8751)

# Description of change

This PR introduces a new component to Starfish consensus, called
**ShardReconstructor**.
Its handle becomes part of an authority node.

Upon receiving a new block bundle via streaming and verifying its
components, the authority service component sends **transaction
messages** to the ShardReconstructor:
- `ShardMessage` – a single shard of encoded transaction data
corresponding to some block reference and transaction commitment
- `FullTransactionMessage` – a full transaction, that indicates that
these transaction data was verified against the block header.

The ShardReconstructor:
- Collects shards per block (`ShardAccumulator`). 
- Filters shards for blocks that are already processed (e.g. with
`FullTransactionMessage`)
- Detects when enough shards (`info_length`) are available to
reconstruct
- Offloads reconstruction jobs to a specified number async workers
- Verifies commitments against reconstructed transactions
- Forwards verified transactions to the core
(`CoreThreadDispatcher::add_transactions`) if the respected headers are
already part of the dag state
- Periodically evicts old state based on the DAG’s GC round

Unit and integration tests cover:
- Reconstruction only after `info_length` shards
- Stopping shard collection when a full transaction message is received
- Multi-round recovery where one authority is blocked, verifying
reconstruction from other authorities

## Links to any relevant issues

Fixes #8673 

## How the change has been tested

- [x] Basic tests (linting, compilation, formatting, unit/integration
tests)
- [x] Patch-specific tests (correctness, functionality coverage)
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] I have checked that new and existing unit tests pass locally with
my changes

### Release Notes

- [x] Protocol: Starfish consensus allows for decoding of the
transaction data from ... (continued)

914 of 1003 new or added lines in 15 files covered. (91.13%)

59 existing lines in 13 files now uncovered.

231398 of 351459 relevant lines covered (65.84%)

2386905.0 hits per line

New Missed Lines in Diff

Lines Coverage ∆ File
1
92.54
 -0.19% crates/starfish/core/src/authority_service.rs
1
98.9
 1.31% crates/starfish/core/src/encoder.rs
2
83.13
 0.21% crates/starfish/core/src/core_thread.rs
2
96.71
 0.0% crates/starfish/core/src/data_manager.rs
5
92.33
 -0.72% crates/starfish/core/src/authority_node.rs
16
89.31
 89.31% crates/starfish/core/src/decoder.rs
62
90.34
 crates/starfish/core/src/shard_reconstructor.rs

Uncovered Existing Lines

Lines Coverage ∆ File
1
91.18
 0.0% crates/iota-cluster-test/src/test_case/native_transfer_test.rs
1
61.36
 1.03% crates/iota-core/src/authority/authority_store_pruner.rs
1
70.14
 -0.08% crates/iota-core/src/checkpoints/mod.rs
1
95.64
 -0.27% crates/iota-core/src/signature_verifier.rs
1
89.31
 89.31% crates/starfish/core/src/decoder.rs
1
81.56
 -0.06% crates/starfish/core/src/synchronizer.rs
2
93.22
 -3.39% consensus/core/src/commit_consumer.rs
2
85.21
 -0.1% crates/iota-core/src/authority/authority_per_epoch_store.rs
3
79.2
 -0.42% crates/starfish/core/src/network/tonic_network.rs
4
92.88
 -0.3% crates/starfish/core/src/transactions_synchronizer.rs
6
92.54
 -0.19% crates/starfish/core/src/authority_service.rs
13
91.03
 -0.8% consensus/core/src/synchronizer.rs
23
74.11
 -3.42% consensus/core/src/authority_service.rs
Jobs
ID Job ID Ran Files Coverage
1 nextest+simtest - 18299309643.1 1434
65.83
 GitHub Action Run
Source Files on build 18299309643