jzombie / rust-triplets / 38472798552

96%

Build Type

push

github

Committed by web-flow

Commit Message

feat: Add precomputed embedding channel and composite record keys  (#168)

### Added

- **Precomputed-embedding channel through the sampler.** `RecordSection.embedding`
  and `RecordChunk.embedding` (`Option<Arc<[f32]>>`, `#[serde(skip)]` — transient,
  re-ingested on resume, never persisted or compared). The sampler stamps every
  materialized chunk from its parent section in `materialize_chunks`, so chunking
  backends stay agnostic and vectors ride the pipeline as Arc pointer copies.
- **`SrdSource` populates all section embeddings** (anchor + candidate/positive/
  negative) from its store vectors, so sampled pairs carry exact store vectors
  on every chunk with no side lookups.
- **Single `SrdSource::open` with explicit mode choice.** `mode: None` detects
  from entry 0 in the same open (no probe handle); `mode: Some(m)` pins the
  mode for empty stores and is verified against entry 0 otherwise — a mismatch
  returns loud `SrdError::ModeMismatch` (empty stores give `EmptyStore`).
  One open per dataset, ever.
- **Native weighted source selection (deficit round-robin).** `TripletSampler`
  apportions batch picks across sources by the per-call weight map: each draw
  credits every eligible source its weight quantum and serves the highest
  deficit, deducting the round total. Deterministic (no RNG — same state, same
  choice), exact over each weight cycle (0.75/0.25 yields exactly 6/2 anchors
  per 8-row batch). Uniform/empty maps take the legacy uniform-cycling path
  bit-identically, so existing deterministic sequences are unchanged. One
  shared sampler with N registered sources is now the mixing topology — no
  outer wrappers, cross-source negatives and global dedup intact.
- **`RecordChunk.source` is required** (no `#[serde(default)]`): a chunk without
  a source resolves split lookups against the wrong identity, so deserialization
  without it fails loudly instead of defaulting to `""`.

### Changed

- **BREAKING: `FileSplitStore` storage versi... (continued)

Coverage Stats

531 of 545 new or added lines in 20 files covered. (97.43%)

5 existing lines in 4 files now uncovered.

22225 of 23140 relevant lines covered (96.05%)

4663.04 hits per line