Logflare / logflare / 37351260852-1

84%

Build Type

push

github

Committed by web-flow

Commit Message

fix: optionally ack stale SpoolAck rows on sweep (#4077)

* Add optional ack-on-sweep for stale SpoolAck rows

A small, continuous trickle of events never reach a real ack (e.g. a
claim whose owner crashed before completing) and today just sit until
SpoolAck.sweep_stale/1 discards the ledger row without ever acking the
underlying queue message. A handful of these can cause a much larger
retry storm once the message redelivers.

sweep_stale/1 now atomically claims each stale row (ets.take, so it
can't race a legitimate in-flight ack) and, behind a new config flag
(ack_stale, default false), performs the real queue ack before
discarding it. Stopgap until the source of the missing acks is fixed --
meant to be disabled again once it is.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* feat: add SPOOL_ACK_STALE emergency lever, fix flaky SpoolAckTest

SPOOL_ACK_STALE env var (default false) toggles ack_stale at runtime via
the existing Env.get_boolean helper, without needing a redeploy — an
emergency lever for when everything starts retrying.

Also fixed the CI failure on this PR: SpoolAckTest's "nil handle ack/2
is a no-op" test was intermittently receiving an unexpected :ack_called
message. Root cause: consumer_pipeline_test.exs deliberately leaves a
failed_handle row in the shared, globally-named :spool_ack ETS table
(verifying a failed dispatch never acks). The new ack_stale: true sweep
test's sweep_stale(0) call sweeps the entire table regardless of
threshold, catches that leftover row too, and fires an extra unawaited
async queue-ack cast for it since ack_stale is now on. That cast can
land during any later, unrelated test, inheriting whatever Mimic stub
is active then. Fixed by clearing :spool_ack before and after every
test in this file, so it's immune to leaks from any other file
regardless of run order. Verified against the real leak across 25
random seeds.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix: parse SPOOL_AC... (continued)

Coverage Stats

6 of 7 new or added lines in 1 file covered. (85.71%)

3 existing lines in 3 files now uncovered.

16031 of 19190 relevant lines covered (83.54%)

2251.51 hits per line