iotaledger / iota / 35043580069

70%

Build Type

push

github

Committed by web-flow

Commit Message

feat(indexer): prune `optimistic_transactions` by optimistic sequence number (#12852)

# Description of change

This PR adds the following changes as the first stage of #12747:

- Adds a `first_optimistic_sequence_number` column to epochs (`NOT
NULL`), holding the first `optimistic_sequence_number` value belonging
to each epoch. Every optimistic sequence number assigned before the
epoch started is below the stored value, so it translates an epoch
retention boundary into a delete range for `optimistic_transactions`.
-New epoch rows are filled by a column default: the latest value
generated by the sequence behind
`tx_global_order.optimistic_sequence_number` + 1, or 0 if the sequence
was never used."
- Existing epochs are backfilled by the migration. Each epoch gets, in
order of preference:
- the lowest `optimistic_sequence_number` among rows indexed at or after
the epoch started
    (matched via their `global_sequence_number` stamps),
- the next value the sequence will assign, when no such row exists
(epochs newer than the
    last optimistic transaction),
  - 0, when no optimistic transaction was ever indexed.
- The migration also resets the `optimistic_transactions` watermark
bounds (`lowest_unpruned_key`, `min_available_tx`) to
`MIN(optimistic_sequence_number)`.
- Switches the `optimistic_transactions` pruning strategy to delete by
`optimistic_sequence_number` (its primary key), bounded by the new
column, instead of by `global_sequence_number`. Since the key is unique,
a range of width N holds at most N rows, so the previous DELETE ...
LIMIT loop is replaced by a plain ranged delete per chunk.

## Links to any relevant issues

fixes #12851 

## How the change has been tested

Ran indexer tests
```
cargo test -p iota-indexer --profile simulator --features shared_test_runtime
```
Manual testing:

- Localnet and indexer from develop without pruning. Once synced, I ran
5 optimistic txs and another 5 in a later epoch, stopped the indexer and
restarted it ... (continued)

Coverage Stats

0 of 55 new or added lines in 4 files covered. (0.0%)

2165 existing lines in 45 files now uncovered.

181196 of 258926 relevant lines covered (69.98%)

434629.74 hits per line