MITLibraries / transmogrifier / 10406601165

99%

push

github

Deduplicate list fields in TIMDEX record (#206)

Why these changes are being introduced:
* Improve data quality of TIMDEX records
by reducing duplication of data in list fields.

How this addresses that need:
* Create an attrs converter function to dedupe list of items
* Create hash method for TIMDEX objects
* Set hash methods in custom classes
* Set 'converter=dedupe' for every list field in TimdexRecord
* Add unit tests verifying functionality of hash and dedupe methods

Side effects of this change:
* Deduplication is highly likely to result in diffs
when comparing transformed records before and after this
change. However (and more importantly), reducing duplicates
improves the data quality of TIMDEX records.

Relevant ticket(s):
* https://mitlibraries.atlassian.net/browse/TIMX-332

31 of 31 new or added lines in 1 file covered. (100.0%)

1673 of 1682 relevant lines covered (99.46%)

0.99 hits per line