Logflare / logflare / e8798a9194990b9f1f075c4fe253e36efe3923d7-PR-3616

81%

main: 82%

Build Type

Pull #3616

github

Committed by djwhitt

Commit Message

perf(bq pipeline): skip per-event ETS lookup in ack when source has no labels

ack/3 looked up the full LogEvent from ETS for every successful event solely
to resolve telemetry labels, copying the entire event out of the cache table
per event. The label mapping was also re-parsed from source.labels on every
event (get_labels_from_event called get_labels_mapping per call).

Resolve the label mapping once per ack. When it is empty — the common case
for sources without configured labels — the event body is never read, so skip
the ETS lookup entirely and emit telemetry directly. When labels are present,
the lookup is still required and unchanged.

Adds Sources.extract_labels/2 (mapping-aware extraction) with get_labels_from_event/2
delegating to it. Behavior note: a successful event missing from ETS at ack
time now still emits ingest telemetry in the no-labels case (previously the
lookup miss suppressed it) — correct, since the event was confirmed inserted.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Pull Request Pull Request #3616: perf(ingest): update_counter claim + BigQuery pipeline allocation cleanups

Coverage Stats

22 of 25 new or added lines in 4 files covered. (88.0%)

16 existing lines in 2 files now uncovered.

13226 of 16292 relevant lines covered (81.18%)

5138.32 hits per line