prisma-risk / tsoracle / 26372668875

95%

push

github

test(driver-paxos): deterministic step-driver for the paxos harness (+ convert the steppable tests) (#312)

* test(driver-paxos): drive restart_replay via a deterministic step-driver

restart_replay (RocksDB-backed 3-node restart/replay) flaked in CI: its
drive_until poll loops have fixed real-time budgets (2–3s) that a slow,
contended runner occasionally overran while real omnipaxos election +
replication + catch-up made progress. The async harness model (per-node tokio
tick + pump tasks on a multi_thread runtime) added scheduling nondeterminism on
top. madsim can't help — the test's whole purpose is real RocksDB, which has no
simulator.

Add a synchronous stepping path, additive to the existing async path:

- PaxosRunner::tick_once() -> Vec<Message> (tick + observe leadership + return
  outbound for the caller to route) and handle_incoming(msg).
- StandaloneHost::step()/apply_once()/deliver() — sync siblings of the runner
  tick + apply task, with a host-held decided cursor seeded past recovery.
- MemNetwork::deliver_now(msg) — sync sibling of the async deliver (delivery was
  already non-blocking try_send).
- Harness Cluster::step()/step_until() — tick every running node, route through
  MemNetwork, deliver inbound, apply; no wall-clock sleep. A node is "running"
  iff it holds a host + handle (so stop_node..rebuild is naturally skipped).

Convert restart_replay to step_until: it now converges in deterministic
simulated steps (~0.04s, 3/3 reproducible) with real RocksDB + replay still
under test. The async path (start_all/drive_until) and every other harness test
are untouched; remaining harness tests can convert in follow-ups.

* test(driver-paxos): convert partition_churn, three_node, snapshot_* to step_until

Build on the deterministic step-driver: convert the remaining cleanly-steppable
paxos harness tests off the real-time drive_until path, eliminating the budget
overruns that flaked CI (snapshot_restart flaked #312's own coverage run).

- par... (continued)

41 of 41 new or added lines in 3 files covered. (100.0%)

11294 of 11897 relevant lines covered (94.93%)

522184.9 hits per line