joaoh82 / rust_sqlite / 24986180889

69%

Build Type

push

github

Committed by web-flow

Commit Message

Phase 7b: vector distance functions + ORDER BY expressions (#44)

Three SQL functions land in the executor's expression evaluator:

  - vec_distance_l2(a, b)      Euclidean √Σ(aᵢ−bᵢ)²
  - vec_distance_cosine(a, b)  1 − a·b/(‖a‖·‖b‖); errors on zero-mag
  - vec_distance_dot(a, b)     −(a·b); negated so smaller-is-closer
                                like the others (pgvector convention)

All three return Value::Real(f64). Internal math is f32 (matches
the Vec<f32> input type from VECTOR(N) columns); widening to f64
at the return boundary so distances slot cleanly into the
executor's existing arithmetic/comparison paths.

The KNN query shape works end-to-end:

  SELECT id FROM docs
  ORDER BY vec_distance_l2(embedding, [0.1, 0.2, ..., 0.0])
  LIMIT 10;

That requires a parser change beyond just adding the functions:
ORDER BY previously only accepted bare column refs. Phase 7b
widens it to accept arbitrary expressions, with `eval_expr`
called per-row in `sort_rowids`. Strict superset — `ORDER BY col`
still works because Expr::Identifier takes the same path. Sort-key
results pre-computed up front so the comparator runs O(N log N)
against pre-evaluated Values rather than re-evaluating the
expression O(N log N) times (will matter once 7d's HNSW is the
hot path).

**Scope correction recorded in docs/phase-7-plan.md:** Q6 anticipated
pgvector-style operators (<-> / <=> / <#>) as a "tiny parser
change". Reality: sqlparser fails outright on `<->` and `<#>`
("Expected: an expression, found: ->"); only `<=>` parses, and
that's MySQL Spaceship null-safe-equality. Supporting all three
needs either a sqlparser fork or a SQL-string preprocessor —
neither tiny.

Decision: ship 7b with **functions only**. Operators move to a
follow-up sub-phase 7b.1. Note added to phase-7-plan.md and to
supported-sql.md so users reading the docs see the rationale.
KNN queries still work — just verbose; 7b.1 swaps to operator
form without other behavior change.

**Other parser change ... (continued)

Coverage Stats

148 of 164 new or added lines in 3 files covered. (90.24%)

1 existing line in 1 file now uncovered.

4390 of 6435 relevant lines covered (68.22%)

1.25 hits per line