yzhao062 / pyod / 25584117718

94%

master: 94%

Build Type

push

github

Committed by web-flow

Commit Message

fix: ADEngine stability metric is now informative (closes #667) (#668)

* docs: add ADEngine refactor design spec for issue #667

Plan to fix the constant stability metric (#667) and pay down
ad_engine.py tech debt over 3 PRs:

- PR 1: stability bug fix (closes #667). Replace Jaccard of nested
  top-k with standardized cutoff gap. 11 regression tests.
- PR 2: structural decomposition into 4 private helper modules.
- PR 3: type hints, validation, declarative NL parser, polish.

Plan-reviewed by Codex over 2 rounds; 11 findings resolved.
Implementation plan to follow.

* docs: add ADEngine refactor implementation plan

17-task plan executing the spec (9db1b59) across 3 PRs.
Identifies parallelization opportunities for subagent dispatch.

* fix: ADEngine stability metric is now informative (closes #667)

Replaces the Jaccard-of-nested-top-k formula (mathematically constant
~0.817 for k>=5) with a standardized cutoff gap. The new formula is
local, deterministic, scale-invariant, and pairs cleanly with the
global `separation` and cross-detector `agreement` slots.

Cascade: `quality['overall']` and `quality['verdict']` may shift on
the same data. Trigger threshold in od_expert/SKILL.md:91 may need
empirical recalibration; deferred to a follow-up.

Tests: 11-test TestStabilityMetric class covering responds-to-data,
determinism, k=0/n/1/n-1, ties at boundary, scale invariance,
constant scores, non-finite scores, unit-interval invariant.

Plan-reviewed and implement-reviewed by Codex.

Spec: docs/superpowers/specs/2026-05-07-adengine-refactor-design.md
Plan: docs/superpowers/plans/2026-05-07-adengine-refactor-plan.md

Closes #667

Coverage Stats

96 of 97 new or added lines in 2 files covered. (98.97%)

1 existing line in 1 file now uncovered.

16137 of 17270 relevant lines covered (93.44%)

10.28 hits per line