jzombie / rust-triplets
96%

DEFAULT BRANCH: main
Repo Added
Files 50
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH main
branch: main
CHANGE BRANCH
x

coverage: 95.82% (-0.005%) from 95.825%
29603096773

push

github

web-flow 
v0.25.0-alpha (#132)

* Bump parquet from 58.3.0 to 59.1.0 (#126)

Bumps [parquet](https://github.com/apache/arrow-rs) from 58.3.0 to 59.1.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/apache/arrow-rs/compare/58.3.0...59.1.0)

---
updated-dependencies:
- dependency-name: parquet
  dependency-version: 59.1.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add `Sampler` mixed strategy tests (#127)

* Remove `hf-hub` dependency and legacy `datasets-server` fallback paths  (#130)

Why: Full control over download and cache handling.

Also, fixes new 503 errors which were starting to originate from deprecated
`datasets-server` fallbacks.

Replace the datasets-server /parquet, /size, and /info endpoints with
the Hub API tree endpoint (/api/datasets/{dataset}/tree/main). Remove
the hf-hub crate dependency entirely, eliminating the three legacy
fallback paths: sibling-based candidate resolution, ClassLabel /info
resolution, and global row count /size queries.
Key changes:
- remote_url_for_candidate handles bare relative paths, full HTTP URLs,
  and url::-prefixed candidates
- candidate_target_path resolves CDN /resolve/ URLs and bare relative
  paths
- Tree endpoint scoped to config subdirectory; falls back to root for
  "default"/empty config
- Path traversal guard rejects ParentDir/Prefix components
- HEAD fallback in download_and_materialize_shard_with_runtime uses
  fetch_remote_size_with_runtime
- target_matches_expected_size rejects 0-byte files when expected_bytes
  is None
- Mock servers return full CDN URLs; HfMockServer uses org/dataset
  dataset name
- Removed: label_maps, size_endpoint, info_endpoint, known_total_rows(),
  collect_candidates_from_sibling... (continued)

5137 of 5662 new or added lines in 35 files covered. (90.73%)

11 existing lines in 4 files now uncovered.

21068 of 21987 relevant lines covered (95.82%)

5318.83 hits per line

Relevant lines Covered
21987 RELEVANT LINES 21068 COVERED LINES
5318.83 HITS PER LINE
Source Files on main

Recent builds

Builds Branch Commit Type Ran Committer Via Coverage
29603096773 main v0.25.0-alpha (#132) * Bump parquet from 58.3.0 to 59.1.0 (#126) Bumps [parquet](https://github.com/apache/arrow-rs) from 58.3.0 to 59.1.0. - [Release notes](https://github.com/apache/arrow-rs/releases) - [Changelog](https://github.com/apache/ar... push web-flow github
95.82
29205262095 main Add offline teacher-embedding precompute pipeline (#125) * Add offline teacher-embedding precompute pipeline and simd-r-drive source crate Introduces two new crates for knowledge distillation data preparation: - `triplets-srd-source`: Low-level... push web-flow github
95.82
29175230798 main Add HuggingFace dict dataset support, weighted scheduling, and explicit dot-path columns (#124) * add HuggingFace dict dataset support, weighted scheduling, and explicit dot-path columns - Add `negative=` column mapping for HF dict datasets (e.g... push web-flow github
95.76
25981838565 main Fix hasher portability issues (#114) This is a breaking change from previous persistent store splits. Also: - Fixes incorrect step tracking (https://github.com/jzombie/rust-triplets/pull/111) - Fixes regressions for duplicate texts in text-only... push web-flow github
95.8
25880329315 main Persist step counter with epochs (#107) * Rework duplicate text detection * Ingestion step counter influences HF shards * Save current epoch step with persisted state * Add support for reserved source IDs * Prepare for 0.22.0-alpha push web-flow github
95.75
25864812165 main Fix duplicate texts across ingestion window (#105) push web-flow github
95.66
25847873070 main Prevent duplicate text samples in batch (#103) (#104) push web-flow github
95.65
25847174982 main Prevvent duplicate text samples in batch (#103) * Prevent duplicate text samples in batches * Add/rename helper methods for clarity * Prepare for 0.21.3-alpha push web-flow github
95.65
25844009165 main Add manual HF source-entry override (#102) * Add documentation link * Add ability to manually specify URL * Use single entry format * Prepare for 0.21.2-alpha push web-flow github
95.65
25841435460 main Fix incorrect prose usage of "Hugging Face" (#101) * Fix incorrect prose usage of "Hugging Face" * Add disclaimers for non-affiliation * Debug flaky macOS build issue * Prepare for 0.21.1-alpha push web-flow github
95.65
See All Builds (443)