Logflare / logflare / 35007352916-1

83%

main: 83%

Build Type

Pull #3991

github

Committed by Baishan

Commit Message

Replace dead Workers and re-scan for stranded sealed files

Process.monitor(pid) on each Worker was never paired with a :DOWN
handler — RotatingWal has no message loop of its own, since its
callbacks run synchronously inside durable_buffer's own Partition
process. A crashed Worker (started unlinked by design, so a bad upload
can't take down the local WAL commit path) stayed in state.workers
forever, silently swallowing every future dispatch Enum.random/1 chose
it for, and whatever segment it was holding when it died sat unrecovered
until a full backend restart.

commit/4 now checks each worker's liveness first and respawns any dead
one with the same sub_partition_index, then re-runs
recover_sealed_segments/1 to pick back up whatever was stranded. This
trades a narrow chance of double-dispatching some other, still
in-flight segment for not waiting on a full restart to recover.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Pull Request Pull Request #3991: Replace spool Partition/Committer stack with durable_buffer

Coverage Stats

281 of 306 new or added lines in 16 files covered. (91.83%)

1 existing line in 1 file now uncovered.

15118 of 18226 relevant lines covered (82.95%)

1973.02 hits per line