moonbitlang / core / 6854

89%

Build Type

push

github

Committed by bobzhang

Commit Message

perf(builtin): count characters eight code units at a time in char_length

`String::char_length`, `char_length_eq` and `char_length_ge` (and the
`StringView` variants, which delegate to them) walked every code unit.
On native and wasm they now count eight UTF-16 code units per step.

A walk that does not abort counts `units - pairs`, where `pairs` is the
number of leading surrogates that are not the last unit of the range.
Each step loads eight units, finds the leading-surrogate lanes, and, if
there are any, checks them against the non-trailing lanes of the eight
units loaded one unit later (aborting with "invalid surrogate pair"
exactly as before), then subtracts the popcount of the leading lanes. A
pair straddling two blocks is counted by its leading lane (as zero) and
its trailing unit (as one). A block with a leading surrogate and no unit
after it inside the range is left to the scalar walk, which counts that
surrogate as one character, as before.

`char_length_eq`/`char_length_ge` stop once the count reaches `len`; the
vector steps only run while `len - count >= 8`, so they examine no unit
the scalar walk would not and end in the same state. Since these two
methods do not validate their offsets, only ranges inside the string take
the vector path; other ranges keep the scalar walk. Loop guards are
written as differences so they cannot overflow.

On js and wasm-gc the public functions are unchanged. On native and wasm
they keep the scalar walk inline for ranges shorter than 16 code units
(and, for eq/ge, `len < 16`) and call the outlined vector loop only for
longer ones, so short inputs pay no extra call. The public interface is
unchanged.

Signed-off-by: Codex CLI <codex@openai.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Coverage Stats

75 of 85 new or added lines in 2 files covered. (88.24%)

21 existing lines in 1 file now uncovered.

16604 of 18576 relevant lines covered (89.38%)

332852.23 hits per line