moeyensj / difi / 24811935441

90%

Build Type

push

github

Committed by moeyensj

Commit Message

Record cross-partition unique counts in scenario manifest

Per-scenario findable_count / found_count in the manifest are SUMS across
partitions, so an object findable in K sliding windows contributes K. For
survey-level "how many asteroids did the linker actually recover?" you want
distinct-object counts, which today required grouping over all_objects.parquet
post-hoc.

Adds three fields to ScenarioManifest, alongside the existing sums:
- unique_findable_count: distinct objects with findable=true in ≥ 1 partition
- unique_found_count: distinct objects with ≥ 1 found-pure / found-contaminated
  linkage across all partitions (None for CIFI-only runs)
- unique_completeness: unique_found_count / unique_findable_count × 100
  (None for CIFI-only runs or when no objects are findable)

Computed by a single pass over all_objects (already in memory when the
manifest is written) via common::compute_unique_counts. Single-partition
runs have unique_* == sum_* by construction — no behavior change there.
On a 76-partition sliding neomod run (per the empirical validation), the
sum findable_count is ~200k (object, partition) pairs while
unique_findable_count is ~15k distinct objects — a ~13× difference that
`unique_completeness` surfaces directly.

README updates the Pipeline / Completeness section with a per-partition
vs. cross-partition table explaining when to quote which number.

Tests: `scenario_manifest_reports_unique_findable_found_and_completeness`
asserts unique ≤ sum and completeness ∈ [0, 100] on a sliding run;
`cifi_only_manifest_has_unique_findable_but_no_unique_found_or_completeness`
asserts CIFI-only runs omit the DIFI-dependent fields.

Verified:
- cargo fmt -- --check
- cargo clippy --all-targets --all-features -- -D warnings
- cargo test --features cli  # 60 passed (+2 new)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Coverage Stats

23 of 24 new or added lines in 3 files covered. (95.83%)

82 existing lines in 6 files now uncovered.

2868 of 3174 relevant lines covered (90.36%)

551.06 hits per line