adaptive-machine-learning / CapyMOA / 38802101661

85%

Build Type

push

github

Committed by web-flow

Commit Message

feat(metrics): replace metric object with typed dicts

Results are now typed dictionaries instead of the PrequentialResults object. Each research domain has its own result type and evaluate function:

- capymoa.classifier.evaluate_classifier -> ClassifierResults
- capymoa.regressor.evaluate_regressor -> RegressorResults
- capymoa.uncertainty.evaluate_prediction_interval -> PredictionIntervalResults
- capymoa.anomaly.evaluate_anomaly -> AnomalyResults
- capymoa.ssl.evaluate_ssl -> SSLResults
- capymoa.ocl.evaluate_ocl -> OCLResults

One shared test-then-train loop serves every domain, with the Java fast path for MOA learners. Passing a mapping of names to learners evaluates several on one pass. Results are built once at the end of the run and hold plain values, so they pickle and go into pandas directly. Whole-stream metrics are under results["cumulative"], per-window metrics under results["windowed"], and drift positions are stored in the result.

Metrics MOA computes are all kept, and some are new to the results: macro precision, recall and F1, per-class metrics per window, rrmse, and the anomaly metrics accuracy, kappa, kappa_m, recall, g_mean and pos_neg_ratio.

Changes that need code updates (see docs/release_notes.rst for the full migration table):

- prequential_evaluation returns the new result types and takes only stream and learner by position.
- prequential_evaluation_multiple_learners, prequential_ssl_evaluation, prequential_evaluation_anomaly and ocl_train_eval_loop are removed in favour of the evaluate functions.
- Evaluator classes moved to each domain's evaluate module, and plots to each domain's plot module.
- Prediction-interval MAE, RMSE, RMAE and RRMSE over the whole stream change from 0.15. MOA's interval evaluator counted each instance twice and scored MAE on the lower bound; this is fixed in the new MOA jar (Waikato/moa#337).

Refs: adaptive-machine-learning/backlog#142

Assisted-by: claude-code:claude-opus-5-5
Assisted-by: claud... (continued)

Coverage Stats

1470 of 2042 new or added lines in 49 files covered. (71.99%)

1 existing line in 1 file now uncovered.

11801 of 13827 relevant lines covered (85.35%)

0.85 hits per line