jzombie / rust-triplets / 29205262095

96%

push

github

Add offline teacher-embedding precompute pipeline (#125)

* Add offline teacher-embedding precompute pipeline and simd-r-drive
source crate

Introduces two new crates for knowledge distillation data preparation:

- `triplets-srd-source`: Low-level read/write for pair and triplet
  embeddings stored in simd-r-drive format, with entry encoding,
  batch I/O, and `SrdSource` for resumable reads.

- `triplets-offline-embedder`: Offline data materialization engine that
  fetches batches from a sampler, embeds via a pluggable teacher
  callback,
  and writes results to per-split stores. Features interleaved split
  scheduling, background prefetching, a 5% error-rate circuit breaker,
  zero-copy validation, and Ctrl+C-safe flushing.

Also extends the `Sampler` trait with a `save_sampler_state` hook and
adds documentation for the offline embedding pipeline to the README.

* cargo udeps and sort

* Update deprecated method

* Fix Rustdoc errors

3834 of 3990 new or added lines in 11 files covered. (96.09%)

24649 of 25723 relevant lines covered (95.82%)

4498.6 hits per line