Logflare / logflare / 0b5f9eaa50fd49d3ba2b639d9b493806b52fc1af

82%

Build Type

push

github

Committed by web-flow

Commit Message

perf: fuse to_bigquery_column_spec into a single binary walk (#3490)

* test: characterize :to_bigquery_column_spec fused pipeline

Lock in the observable behavior of the `:to_bigquery_column_spec`
key transformation before fusing the five sub-passes into a single
binary walk. Covers each rule in isolation, cumulative leading-`_`
prepending across rules, suppression of leading-digit prepend when
a prefix or dash already prepended, pass-through cases (atom/int/
empty-string keys, already-valid keys), PCRE byte-mode Latin-1 word
classification on multibyte input, and the 128-byte field-length
boundary including prefix-induced length growth.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* perf: fuse to_bigquery_column_spec into single binary walk

The legacy implementation piped each map key through five sequential
sub-passes (strip_bq_prefix → dashes_to_underscores →
alter_leading_number → alphanumeric_only → enforce_field_length).
Most keys hit none of the rewrite branches but still paid all five
scans, with `:re.run/3` and `:binary.match/2` showing up at ~13%
combined wall time in the edge log fixture.

Walk the binary once, tracking the dash/non-alnum flags and detecting
the bq-reserved-prefix and leading-digit cases upfront. Each rule
still contributes at most one leading underscore; the digit rule is
suppressed when a prefix or dash has already prepended, preserving the
order of the original pipeline. The byte classifier mirrors PCRE's
byte-mode \w table (incl. Latin-1 letter ranges 192..214, 216..246,
248..255 and the lone-letter bytes 170/181/186) so non-ASCII keys
produce identical column names to the prior `~r/\W/` replacement.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* bench: record o11y-1828 fused column-spec walk deltas

Captures the LogEvent.make benchmark on top of the o11y-1829 baseline
(640151ae9). Single-pass binary walk over each ingest key replaces the
five sub-pass pipeline, cutt... (continued)

Coverage Stats

26 of 26 new or added lines in 1 file covered. (100.0%)

1 existing line in 1 file now uncovered.

13183 of 16237 relevant lines covered (81.19%)

5115.51 hits per line