Logflare / logflare / a083884c9293387d2d027fe9480108f448261fe9-PR-3668

81%

main: 81%

Build Type

Pull #3668

github

Committed by djwhitt

Commit Message

fix: skip empty ClickHouse insert when a routed batch has no surviving rows

The metadata fast path routes messages in handle_message/3 without re-checking
ETS, so if every row in a batch is deleted (e.g. by the QueueJanitor) before
encode_message/7 runs, stream_compress/4 returns good == [] and a gzip payload
with zero RowBinary rows. finalize_insert/6 previously POSTed that empty payload
to ClickHouse; insert_log_events_compressed/4 has no empty guard (unlike the
non-compressed Ingester.insert/5). Add a good == [] clause that fails the missing
messages without calling ClickHouse, and a handle_batch/4 test asserting the
insert is not attempted for an all-missing routed batch.

Reported in ClickHouse's review of PR #3668.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Pull Request Pull Request #3668: fix: eliminate ClickHouse pipeline CPU regression

Coverage Stats

23 of 23 new or added lines in 4 files covered. (100.0%)

7 existing lines in 3 files now uncovered.

14037 of 17364 relevant lines covered (80.84%)

24739.06 hits per line