moonbitlang / core / 6760

89%

push

github

perf(regex): two-level symbol table for non-ASCII input (#4268)

`Table::map` served ASCII from a direct table and resolved every code unit
>= 128 with a binary search over `(lo, hi, symbol)` triples, so patterns
with many classes paid ~10 ns per non-ASCII character. The table is now
two-level: one `index` entry per 256-code-unit block gives the offset of
that block's 256 symbols in `blocks`, and blocks whose 256 units all map to
one symbol share a physical block per symbol, so the table stays small.
Block 0 is always emitted first, so code units below `min(256, ub + 1)`
are a single load; above that, `map` checks `c <= ub` (panicking as the
old search did) and does two dependent loads with no search.

Native, 200k-char CJK scans: 40-class alternation 1.97 ms -> 0.24 ms,
`[[:digit:]]+` 446 us -> 222 us; js 3.69 -> 1.07 ms and 1.15 -> 1.07 ms.
ASCII scans unchanged. No interface change. Part of #4263.

Adds unit tests for run numbering, block boundaries, materialized blocks,
byte-sized and unaligned upper bounds, and the bound checks.

Codex CLI review (two rounds; the first caught that `ub` inside a block was
not enforced, now fixed): "The P2 is resolved: fast_limit and the
upper-bound guard prevent access beyond ub, including padded entries. The
added tests cover all three regression cases. No additional correctness
regressions found."

Signed-off-by: Codex CLI <codex@openai.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>

24 of 24 new or added lines in 2 files covered. (100.0%)

16474 of 18369 relevant lines covered (89.68%)

333752.12 hits per line