Logflare / logflare
81%
main: 82%

LAST BUILD BRANCH: refactor/source-schema-from-flatmap
DEFAULT BRANCH: main
Repo Added
Files 487
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH perf/ingest-queue-take-pending-ids
branch: perf/ingest-queue-take-pending-ids
CHANGE BRANCH
x

coverage: 81.181% (-0.01%) from 81.193%
e68633a73a549f4797ff8c21d67e6c0aa4ffbe5d-PR-3616

Pull #3616

github

djwhitt 
perf(bq pipeline): emit ingest telemetry in handle_batch, drop ack ETS lookup

ack/3 re-fetched the full LogEvent from ETS per successful event solely to
resolve telemetry labels — a second copy of an event handle_batch/4 had
already pulled out of ETS. Emit the per-event ingest telemetry in
handle_batch instead, where the LogEvent is already in hand via `triples`,
resolving the label mapping and backend metadata once per batch.

ack now only needs each event's id to finalize queue state and never touches
the event body, removing the per-event lookup entirely (both the labelled and
unlabelled paths). Replaces the maybe_emit_event_telemetry machinery added in
the previous commit.

Behavior note: telemetry now fires at batch-insert time rather than at ack.
The set is identical (handle_batch's succeeded == ack's successful) except on
the exceptional path where handle_batch raises after emitting — consistent
with the pipeline already emitting regardless of BigQuery insert errors.
Events missing from ETS are not in `triples`, so they emit no ingest
telemetry (matching the original pre-skip behavior).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pull Request #3616: perf(ingest): update_counter claim + BigQuery pipeline allocation cleanups

24 of 26 new or added lines in 4 files covered. (92.31%)

2 existing lines in 1 file now uncovered.

13226 of 16292 relevant lines covered (81.18%)

5083.48 hits per line

Relevant lines Covered
16292 RELEVANT LINES 13226 COVERED LINES
5083.48 HITS PER LINE
Source Files on perf/ingest-queue-take-pending-ids

Recent builds

Builds Branch Commit Type Ran Committer Via Coverage
e68633a7... perf/ingest-queue-take-pending-ids perf(bq pipeline): emit ingest telemetry in handle_batch, drop ack ETS lookup ack/3 re-fetched the full LogEvent from ETS per successful event solely to resolve telemetry labels — a second copy of an event handle_batch/4 had already pulled out of... Pull #3616 djwhitt github
81.18
e8798a91... perf/ingest-queue-take-pending-ids perf(bq pipeline): skip per-event ETS lookup in ack when source has no labels ack/3 looked up the full LogEvent from ETS for every successful event solely to resolve telemetry labels, copying the entire event out of the cache table per event. The... Pull #3616 djwhitt github
81.18
c015f9fc... perf/ingest-queue-take-pending-ids test(ingest): cover claim-reset reclaim cycle and process_data source threading Adds behavioral coverage for the two changes that previously only had static or indirect assertions: - take_pending_ids/2: an event re-added after being claimed is c... Pull #3616 djwhitt github
81.13
8681670e... perf/ingest-queue-take-pending-ids perf(bq pipeline): collect batch events and metrics in a single pass handle_batch/4 traversed the batch three times: Enum.map to extract log events, length for the count, and Enum.sum_by for the byte total. Fold them into one Enum.reduce. Log eve... Pull #3616 djwhitt github
81.14
9ae237b6... perf/ingest-queue-take-pending-ids test(ingest queue): take_pending_ids tests and claim-strategy benchmark Add a take_pending_ids/2 describe block covering claim correctness, the remainder-left-pending case, no re-claim across sequential calls, and a concurrent-claim race test ass... Pull #3616 djwhitt github
81.19
See All Builds (3992)