Logflare / logflare / f229ab208bfcae0ce060e829f0a96ef075f59317-PR-3626

82%

main: 82%

Build Type

Pull #3626

github

Committed by djwhitt

Commit Message

tune(queue janitor): 30s staleness age, 10k per-pass limit

Lower the stale :processing age from 120s to 30s and raise the per-pass limit
from 1,000 to 10,000.

A live :processing row is bounded to ~10s (batch fill plus a BigQuery insert
hard-capped by the Finch pool/receive timeouts, with no in-place retry), so 30s
keeps a comfortable margin against resetting a slow-but-live batch while
recovering orphaned rows sooner and limiting how long they squat on the queue's
size budget. The 10k limit is sized to clear a crashed pipeline's in-flight
cohort (processors + batchers) in a single pass rather than draining slowly
across many passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Pull Request Pull Request #3626: refactor(ingest queue): timestamp-based stale :processing recovery

Coverage Stats

28 of 33 new or added lines in 3 files covered. (84.85%)

3 existing lines in 3 files now uncovered.

13334 of 16370 relevant lines covered (81.45%)

5140.88 hits per line