prisma-risk / tsoracle / 26382945449

95%

push

github

test(stress): run paxos shutdown test under virtual time (#398)

shutdown_completes_server_tasks guarded server-task completion with a
wall-clock tokio::time::timeout. On a 2-vCPU CI runner the libtest
harness runs this file's many multi_thread topology tests concurrently,
each with its own 4-worker runtime plus rocksdb threads, so the spawned
drain task could be scheduling-starved past the bound with nothing
actually wrong — a flake. Tuning the bound (5s, then 30s) only moves the
cliff; the timeout is wall-clock and the contention is unbounded.

Run the test under tokio virtual time (start_paused) instead, matching
the standalone_shutdown regression test. The topology's MemNetwork,
runner ticks, election, wait_for_leader poll, and the completion guard
all advance in simulated time; the tonic servers serve over loopback but
have no peer connections (peers use MemNetwork), so a healthy drain
finishes the instant its shutdown oneshot fires. The timeout now only
elapses when the runtime goes idle with a handle unfinished — a genuine
drain hang — so it can never trip on CPU starvation, and the bound costs
no wall clock.

Verified: healthy run completes in ~0.03s and stays green 20/20 under a
12-core CPU load that previously drove the group to the old 5s ceiling;
a negative check (shutdown never fired) still fails fast via the
sim-time guard rather than hanging.

1 of 1 new or added line in 1 file covered. (100.0%)

12844 of 13538 relevant lines covered (94.87%)

429652.92 hits per line