joaoh82 / rust_sqlite / 25079608728

69%

Build Type

push

github

Committed by web-flow

Commit Message

Phase 7d.3: HNSW persistence + DELETE/UPDATE rebuild trigger (#52)

Closes out Phase 7d. Three intertwined pieces:

  1. **HNSW graph persistence** via a new KIND_HNSW (0x05) cell tag.
     New src/sql/pager/hnsw_cell.rs holds the HnswNodeCell encoder
     (varint node_id + per-layer neighbor lists). Each HNSW index
     gets its own page tree parallel to secondary indexes; the tree
     reuses TableLeaf pages because Cell::peek_rowid works
     uniformly across all cell kinds (it just reads the first
     varint after the kind tag, which is node_id here). Slot-
     directory binary search by node_id therefore works without
     HNSW-specific page-level code.

     Save: stage_hnsw_btree → stage_hnsw_leaves emit one cell per
     node, sibling-chained leaves + interior layers as the index
     grows past one page. The rootpage goes into sqlrite_master.

     Open: load_hnsw_nodes walks the leaves, decodes cells,
     hands the (id, layers) tuples to the new
     HnswIndex::from_persisted_nodes constructor — no algorithm
     runs on the load path; the loaded graph is bit-identical to
     what was saved. Compatibility path retained for files saved
     before persistence (rootpage == 0): execute_create_index
     re-walks current rows, same as 7d.2.

  2. **DELETE / UPDATE on HNSW-indexed tables now works.** The 7d.2
     refusals are removed. New `needs_rebuild` flag on
     HnswIndexEntry marks an index as stale when DELETE removes a
     row or UPDATE touches its column. save_database calls
     rebuild_dirty_hnsw_indexes at the top, before any immutable
     borrows: walks the table's current Vec<f32> column, builds a
     fresh HnswIndex, replaces the entry's index, clears the flag.
     Then the staging loop serializes the rebuilt graph.

     Cost: O(N · ef_construction · log N) per dirty index per save.
     Fine for ≤ 10k-row tables, expensive for ≥ 100k-row bulk
     deletes. Matches the trade-off SQLite makes for FTS5
     (rebuild-on-muta... (continued)

Coverage Stats

251 of 293 new or added lines in 5 files covered. (85.67%)

5 existing lines in 2 files now uncovered.

5211 of 7347 relevant lines covered (70.93%)

1.46 hits per line