prisma-risk / tsoracle / 26245990416

95%

Build Type

push

github

Committed by web-flow

Commit Message

feat(stress): in-process raft topology for the stress harness (#37)

* build(stress): add openraft deps for raft topology

* feat(stress): RaftTopology spawn boots in-process openraft cluster

Replaces the stub in `benchmarks/stress/src/topology/raft.rs` with a real
N-node implementation backed by `openraft_toolkit::test_fakes::MemNetwork`.
Each node owns a `RocksdbLogStore` in a tempdir, a `HighWaterStateMachine`,
and a dedicated `tsoracle::Server` bound to a loopback port via
`StandaloneHost` + `OpenraftDriver`. Membership is initialized on node 1
once every node is registered with the shared network, and `spawn` waits
up to 2 seconds for an initial leader before returning.

`RaftController` exposes `endpoints()`, `current_leader()` (reads
`Raft::metrics().current_leader`), and `shutdown()` (drains each node's
oneshot shutdown sender). Chaos primitives (`kill_leader`, `pause_leader`,
arm/disarm failpoints) return `ChaosOutcome::Skipped` for now and will be
replaced in follow-up PRs with `MemNetwork` partition-based chaos.

Adds `rocksdb` and `tempfile` to `[dependencies]` (the toolkit's log store
requires both, and they are not test-only here since `spawn` is library
code). One test boots a 3-node cluster, asserts three endpoints and a
leader, and shuts down — runs in ~80 ms.

* feat(stress): raft topology kill_leader via network partition

Replaces the Skipped placeholder on RaftController::kill_leader with a real
chaos op: read the current openraft leader from any node's metrics, isolate
that NodeId on the shared MemNetwork PartitionController for ~1.5s so the
remaining quorum elects a fresh leader, then heal so the next chaos op still
has a quorum to work with.

The 750ms isolation window from the original sketch flaked locally — the
followers in openraft 0.10 need a few heartbeat-interval ticks after the
election-timeout window before they actually escalate to Candidate, so the
sleep is bumped to 1500ms (still well inside a chaos cycle, but stab... (continued)

Coverage Stats

372 of 395 new or added lines in 2 files covered. (94.18%)

1 existing line in 1 file now uncovered.

5302 of 5565 relevant lines covered (95.27%)

97164.2 hits per line