MITLibraries / transmogrifier / 11974545807

96%

main: 99%

push

github

Establish feature flag branching for parquet work

Why these changes are being introduced:

We will be using a feature flag approach while modifying Transmogrifier
to output transformed files to a parquet dataset instead of JSON and TXT
files.

We are doing so with the goal of full backwards compatability until
that refactor is complete.

How this addresses that need:
* Utilizes a new optioanl env var 'ETL_VERSION' to provide an ETL version
for code to branch from.
* The Transformer.__next__() method calls dedicated sub-methods depending
on the version.
* Functional stubs have been added for v2 behavior, but are mostly designed
for envisioning the feature flag branching, and will have their own pass
for completeness and testing.
* CLI command is updated to call new Transformer methods based on the ETL
version.
* All feature flag logic branching is noted by comments, suitable for
removal when the development work is complete.

Side effects of this change:
* Transmogrifier remains fully backwards compatible, either via the
absence of env var 'ETL_VERSION' or if the value equals '1'.

Relevant ticket(s):
* https://mitlibraries.atlassian.net/browse/TIMX-404

42 of 111 new or added lines in 3 files covered. (37.84%)

1703 of 1780 relevant lines covered (95.67%)

0.96 hits per line