MITLibraries / transmogrifier-ab-diff / 11728992114

86%

push

github

Add option to download input files using a local MinIO server (#49)

Why these changes are being introduced:
* Downloading extract files improves the performance of the app by
reducing requests sent to AWS S3 and avoiding repeated downloads of
extract files used across multiple container runs. Having extract files
available on local disk also minimizes the occurence of network issues
or AWS credentials timing out during a transform. These changes introduces
a locally hosted MinIO server to act as a "local S3 bucket" as part of
the A/B diff workflow.

How this addresses that need:
* Add a Docker Compose YAML file to run local MinIO server
* Add Makefile commands for starting and stopping local MinIO server
* Add option '--download-files' to run-diff CLI command
* Add 'download_input_files' function to extras
* Update 'run_ab_transforms' to support use of local MinIO server

Side effects of this change:
* None

Relevant ticket(s):
* https://mitlibraries.atlassian.net/browse/TIMX-353

27 of 61 new or added lines in 5 files covered. (44.26%)

717 of 818 relevant lines covered (87.65%)

0.88 hits per line