MITLibraries / transmogrifier / 20178296089

99%

main: 99%

push

github

Error loading exclusion lists exits early without write

Why these changes are being introduced:

If there are any errors loading an exclusion list for a source, e.g.
the file is not accessible or not parseable, we want the transformation process
to fail completely without writes to the TIMDEX dataset.

As-is, we found that all records from the run will write to the dataset but with
action="error".  This is good when *some* records fails to understand what they
were and why, but an exclusion list error will fail for *all* records making
this not helpful.

How this addresses that need:

A new custom exception `CriticalError` is introduced that when raised
will break the Transformer.__next__() iterator.  In turn, this will
bubble up to the CLI level and terminate the run with a non-zero
process.

This provides a mechanism from within a Transformer's transformation
logic to raise an exception that will break the Transformer.__next__
iteration.

Side effects of this change:
* Failure to load an exclusion list will terminate the run early
and explicitly.
* In the future, catastrophic errors that suggest all records in
the run will fail should also raise this new, custom exception.

Relevant ticket(s):
* https://mitlibraries.atlassian.net/browse/USE-278

9 of 12 new or added lines in 2 files covered. (75.0%)

1804 of 1824 relevant lines covered (98.9%)

0.99 hits per line