prisma-risk / tsoracle / 26214097441

95%

push

github

fix(server): poison serving state when leader-watch panics in into_router (#29)

The spawned leader-watch task in `Server::into_router` only called
`step_down_due_to_consensus_rejection` on the `Err` branch of
`run_leader_watch`. If the task panicked instead, the step-down was
skipped and `ServingState::Serving` stayed published while the watch
was dead — the inverse of the "fail-safe even when the handle is not
observed" guarantee documented on `into_router`. The gap affected
embedders mounting the tsoracle service alongside their own services
via `into_router` who never observe the `JoinHandle`. `serve` /
`serve_with_shutdown` / `serve_with_listener` were already safe because
their `join_to_server_result` translator surfaces the panic.

Fix: wrap the awaited future in `std::panic::AssertUnwindSafe(...)
.catch_unwind()`. On panic, call `step_down_due_to_consensus_rejection`
to publish `NotServing`, then `std::panic::resume_unwind` to re-raise
so handle observers continue to receive `ServerError::WatchPanic`
unchanged. Byte-identical behavior for existing callers; the new
property is the poisoning step for handle droppers.

Test scaffolding:
- New failpoint site `server::fence::after_serving_published` in
  `fence.rs`, immediately after the fence publishes
  `ServingState::Serving` and drops the drain guard. This is the only
  location from which a panic can fire *from a Serving state*, which is
  what the regression test needs to distinguish fixed vs. unfixed
  behavior — existing failpoints all fire before Serving is published.
- New test
  `panic_after_serving_published_poisons_state_when_handle_dropped`:
  mounts `into_router` directly, drops the `JoinHandle`, panics the
  watch via the new failpoint, then polls a real gRPC `GetTs` until it
  returns `FAILED_PRECONDITION` (2s deadline). Verified to fail on
  unfixed code with the expected assertion message.
- Updated the comment on
  `fence_panic_after_persist_advances_durable_but_not_serving` to
... (continued)

16 of 16 new or added lines in 2 files covered. (100.0%)

2439 of 2608 relevant lines covered (93.52%)

211842.46 hits per line