freeeve / roaringrange / 28279688357

84%

Build Type

push

github

Committed by freeeve

Commit Message

feat(index)!: optional case-sensitive indexing (default case-fold)

Add a case_sensitive index-creation setting across every text surface. The
default (false) case-folds, so all existing builds stay byte-identical -- every
pre-existing golden test is unchanged.

Per-format encoding of the case-sensitive flag (all unset => byte-identical):
- RRTI term index: header flags bit2 (FLAG_CASE_SENSITIVE = 4).
- RRSI trigram: append-only v4 (v3 16B header + trailing 2B flags at offset 16);
  reader accepts v3 and v4, rrs_boot_len is version-aware.
- RRSF facet: reserved u16 bit0; facet_key hashes raw bytes when set.
- RRSS manifest: flags bit4; split-set reader threads it to ngram/facet pruning,
  delta SplitSetWriter::resume inherits it from the base.

Build folding and query folding stay symmetric on every surface (the build ==
query correctness invariant). Readers derive case_fold = !flag.

API:
- Rust and Go config structs expose case_sensitive/CaseSensitive (default false):
  TermIndexConfig, SplitBuildConfig, TermSplitBuildConfig, WriterConfig. Builders
  translate to the internal case_normalization (true = fold) via !case_sensitive.
- Low-level/operation fns gain back-compat ...With(.., case_fold) siblings
  (tokenize_with, ngram_keys_with, write_index_with, write_facets_with,
  WriteTermIndexWith) so the existing public fns are unbroken.
- Python TermBuilder/SplitSetBuilder/TermSplitSetBuilder take case_sensitive=False.
- wasm reader reads v4/flags via the Rust reader (reader-only).

BREAKING CHANGE: Rust config structs gain a required case_sensitive field, so
struct-literal construction must set it explicitly.

Cross-language conformance goldens (Rust generates, Rust and Go assert
byte-for-byte): testdata/rrs_monolith_cs_build_golden.txt (v4 RRSI) and
testdata/rrti_term_split_cs_golden.txt (RRTI + RRSF + manifest). Go FuzzNgramKeys
extended with the case-sensitive property.

Coverage Stats

100 of 108 new or added lines in 8 files covered. (92.59%)

2 existing lines in 1 file now uncovered.

1484 of 1760 relevant lines covered (84.32%)

33.45 hits per line