joaoh82 / rust_sqlite / 25284563739

70%

push

github

feat(fts): 8b — SQL surface for full-text search (#79)

Phase 8 sub-phase 8b per docs/phase-8-plan.md. Wires the standalone
FTS algorithms (8a) into the SQL executor end-to-end. Mirrors the
Phase 7d.2 HNSW integration shape across every touchpoint.

User-visible surface:

  CREATE INDEX ix ON docs USING fts (body);

  -- predicate: row contains any query term?
  SELECT id FROM docs WHERE fts_match(body, 'rust embedded');

  -- BM25 ranking, top-k:
  SELECT id FROM docs
   WHERE fts_match(body, 'rust embedded')
   ORDER BY bm25_score(body, 'rust embedded') DESC
   LIMIT 10;

Engine plumbing:

- src/sql/db/table.rs:
  - new FtsIndexEntry { name, column_name, index: PostingList,
    needs_rebuild }, mirroring HnswIndexEntry field-for-field
  - Table::fts_indexes alongside hnsw_indexes
  - maintain_fts_on_insert hook called after secondary-index +
    HNSW maintenance in insert_row
  - deep_clone propagates fts_indexes for transaction snapshots
- src/sql/executor.rs:
  - IndexMethod::Fts arm + "fts" string-match in CREATE INDEX
    USING dispatch
  - create_fts_index validates TEXT (rejects VECTOR/JSON/INTEGER
    with a clear error), seeds PostingList from existing rows,
    pushes FtsIndexEntry
  - fts_match / bm25_score scalar fns in eval_function — bare
    column ident in arg 0, TEXT expression in arg 1; both error if
    no FTS index covers the column
  - try_fts_probe optimizer hook recognizes
    `ORDER BY bm25_score(col, 'q') DESC LIMIT k` and serves it
    from PostingList::query (top-k lookup). ASC falls through.
    Mirrors try_hnsw_probe's WHERE-drop posture per Q6
  - DELETE / UPDATE flag fts_indexes[i].needs_rebuild = true,
    same shape as HNSW
  - name uniqueness check spans btree + hnsw + fts namespaces
- src/sql/pager/mod.rs:
  - rebuild_dirty_fts_indexes runs at save_database start, walks
    current rows of dirty FTS indexes, replaces the PostingList
  - rebuild_fts_index replays CREATE INDEX SQL on open via
    execute_create_index ... (continued)

247 of 284 new or added lines in 4 files covered. (86.97%)

6053 of 10311 relevant lines covered (58.7%)

1.16 hits per line