MITLibraries / transmogrifier-ab-diff / 11408587656

96%

main: 86%

push

github

Parse metrics even with zero diffs

Why these changes are being introduced:

A bug surfaced where attempting to calculate metrics from
the diffs dataset would throw an error if zero records
had any diffs.  This was the result of dynamically preparing
SQL queries for DuckDB that looked for any field with a value
of 1. With no diffs, there were no fields, and the 'OR' clause
was malformed.

In the process of addressing this bug, it was determined
this somewhat brittle SQL query generation could be avoided
by carrying over the 'has_diff' column from the diffs dataset
and use that to identify rows in the sparse matrix that had
some kind of diff.

How this addresses that need:
* add 'has_diff' column to sparse matrix, carried over from
diffs dataset
* utilize 'has_diff' column to identify sparse matrix
rows with diffs
* remove dynamic field based 'OR' queries that were problematic
when no fields to check for

Side effects of this change:
* None

Relevant ticket(s):
* https://mitlibraries.atlassian.net/browse/TIMX-372

5 of 6 new or added lines in 2 files covered. (83.33%)

570 of 592 relevant lines covered (96.28%)

0.96 hits per line