MITLibraries / transmogrifier-ab-diff / 11729396796

85%

main: 86%

Build Type

Pull #48

github

Committed by ghukill

Commit Message

Create final records dataset from diffs and metrics

Why these changes are being introduced:

With the full pipeline mostly formed it was observed that we could
simplify the final artifacts produced by a run by combining the 'diffs'
and 'metrics' datasets into a final 'records' dataset.

This dovetails with the new optional env var 'PRESERVE_ARTIFACTS' where
every other artifact except this one would be removed after successful creation.

How this addresses that need:
* Adds new core function 'create_final_records' that writes a final
dataset containing all records and diff information needed for statistical
and individual record analysis.

Side effects of this change:
* Fewer final disparate datasets for a run

Relevant ticket(s):
* https://mitlibraries.atlassian.net/browse/TIMX-370

Pull Request Pull Request #48: TIMX 383 - pipeline tweaks for large runs

Run Details

80 of 109 new or added lines in 8 files covered. (73.39%)

8 existing lines in 2 files now uncovered.

774 of 904 relevant lines covered (85.62%)

0.86 hits per line