databendlabs / openraft / 29088517269
87%

DEFAULT BRANCH: main
Ran
Jobs 1
Files 265
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 87.096% (-0.07%) from 87.165%
29088517269

push

github

drmingdrmer 
fix: bound state-machine-worker calls against worker death

# Summary

Calls whose responder lives in the state-machine worker -- `get_snapshot`,
`begin_receiving_snapshot`, `external_state_machine_request`, and
`with_state_machine` -- resolved a send or upgrade failure by joining
`RaftCore` unboundedly. But the worker owns that channel's receiving end
and can die on its own -- e.g. a panic in a state-machine method -- while
`RaftCore` keeps running, so the join never completed and the call hung
forever. They now resolve the real stop cause through a bounded wait.

# Details

- Add `RaftInner::get_core_stop_error_bounded()`, mirroring `recv_msg`'s
  dropped-responder handling: wait up to `RECV_CORE_STOP_TIMEOUT` for
  `RaftCore` to actually stop, then fall back to `Fatal::Stopped`.
- `send_sm_command`, `external_state_machine_request`, and
  `with_state_machine` route every send/upgrade failure through it.
  `external_state_machine_request` previously masked the real cause behind
  a bare `Fatal::Stopped`, contradicting its documented outcomes.
- Add `get_snapshot_returns_stopped_when_sm_worker_dies` and
  `with_state_machine_returns_stopped_when_func_panics`, backed by a
  memstore `panic_on_apply` fault that kills the worker while `RaftCore`
  lives; both assert `Fatal::Stopped` within the bounded wait instead of
  hanging.

15 of 17 new or added lines in 4 files covered. (88.24%)

352 existing lines in 30 files now uncovered.

17474 of 20063 relevant lines covered (87.1%)

156900.44 hits per line

Uncovered Changes

Lines Coverage ∆ File
1
95.38
 -0.19% openraft/src/raft/api/protocol.rs
1
93.92
 0.56% openraft/src/raft/mod.rs

Coverage Regressions

Lines Coverage ∆ File
66
92.44
 0.04% openraft/src/core/raft_core.rs
34
79.9
 0.0% openraft/src/engine/command.rs
28
6.67
 -93.33% openraft/src/storage/v2/raft_state_machine.rs
22
0.0
 0.0% legacy/src/network_v1/sender.rs
22
93.92
 0.56% openraft/src/raft/mod.rs
20
72.45
 -5.1% openraft/src/core/sm/command.rs
16
0.0
 0.0% legacy/src/network_v1/adapter.rs
16
31.03
 0.0% openraft/src/raft/leader.rs
14
85.86
 0.6% openraft/src/replication/snapshot_transmitter.rs
13
73.08
 0.0% legacy/src/network_v1/snapshot_receiver.rs
13
52.78
 0.15% openraft/src/core/raft_msg/raft_msg_name.rs
12
97.19
 -0.02% openraft/src/engine/engine_impl.rs
12
72.88
 0.0% openraft/src/engine/engine_output.rs
10
72.92
 3.87% legacy/src/network_v1/streaming.rs
6
91.43
 0.0% openraft/src/engine/handler/leader_handler/mod.rs
6
89.83
 0.0% openraft/src/network/v2/network.rs
5
0.0
 0.0% legacy/src/network_v1/network.rs
5
89.55
 0.84% openraft/src/core/sm/worker.rs
5
95.38
 -0.19% openraft/src/raft/api/protocol.rs
5
95.45
 0.0% stores/memstore-custom-node-id/src/lib.rs
4
27.27
 0.0% openraft/src/engine/testing.rs
4
96.72
 0.07% stores/memstore/src/lib.rs
3
98.03
 0.0% openraft/src/engine/command_scheduler.rs
3
98.83
 0.0% openraft/src/engine/handler/replication_handler/mod.rs
3
92.5
 0.83% openraft/src/raft/linearizable_read/linearizer.rs
1
99.25
 0.0% openraft/src/core/io_flush_tracking/watch_progress.rs
1
93.51
 0.0% openraft/src/core/tick.rs
1
99.38
 0.0% openraft/src/engine/handler/following_handler/mod.rs
1
97.44
 0.0% openraft/src/engine/handler/log_handler/mod.rs
1
99.21
 0.0% openraft/src/engine/handler/vote_handler/mod.rs
Jobs
ID Job ID Ran Files Coverage
1 29088517269.1 265
87.1
 GitHub Action Run
Source Files on build 29088517269