wwu-mmll / confound_corrected_cpm / 35742708320
88%
main: 88%

LAST BUILD BRANCH: develop
DEFAULT BRANCH: main
Ran
Jobs 1
Files 37
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 87.278%. Remained the same
35742708320

push

github

NilsWinter 
Audit the test suite: the count and the runtime are different problems

RELEASE_PLAN.md's test-suite section was written from a skim and got two things
wrong, both now corrected against the code: TestTaskTypeDetection is not covered
by test_validation.py (that file only tests check_data and get_variable_names),
and the four subsumed sklearn comparisons in test_scoring.py are subsumed for a
better reason than "same assertions" -- the comprehensive test uses signal plus
noise rather than pure noise, so a scale error actually shows there.

Measured rather than assumed. 237 of the 359 tests cost about five seconds
combined; the 336s of runtime is 122 tests in nine files. Instrumenting
CPMAnalysis.run across a full suite run gives 70 in-process pipeline runs, and
two files account for 25 of them while needing 9 distinct configurations
between them -- test_report_states_its_configuration runs 17 analyses over 5
configurations, test_confound_api runs 8 over 4.

So cutting the count and cutting the runtime are almost disjoint jobs. The count
is inflated by parametrizing over things that are not test cases: 76 of the 359
are two AST checks spread over the package's own source files. The runtime is
redundant pipeline runs, which module-scoped fixtures fix without deleting a
single test.

TEST_AUDIT.md goes group by group and says what would still fail if the code
regressed: 15 tests are genuinely redundant, 27 more are one behaviour spread
over many test functions and should merge rather than go, and the three most
expensive files should stay expensive -- test_integration.py is the only guard
against example rot, and test_feature_interactions.py's 2^4 grid exists because
two features were silently inert in combination while working alone, which is
the one thing a full factorial catches and per-feature tests do not.

Projected 359 -> 241 tests and 5m40s -> ~4m45s with nothing uniquely-covering
removed. No test has been changed; this is for Nils to decide on.

Al... (continued)

2504 of 2869 relevant lines covered (87.28%)

0.87 hits per line

Jobs
ID Job ID Ran Files Coverage
1 35742708320.1 37
87.28
GitHub Action Run
Source Files on build 35742708320