MITLibraries / dspace-submission-composer / 16576813332

Pull #188

github

Add CLI command to sync data between two locations

Why these changes are being introduced:
DSC supports submissions to both test and prod instances of MITL's
DSpace repositories. Deposits to test are performed in the Dev and
Stage environments, while deposits to prod are performed in the
Prod environment. The required data must be present in, or uploaded to,
the appropriate S3 buckets for each environment. This method
provides an easy way to sync the data between two S3 buckets
using the AWS CLI 'aws s3 sync' command.

How this addresses that need:
* Add 'sync' CLI command
* Add optional env var 'S3_BUCKET_SYNC_SOURCE'
* Update Dockerfile to include install of AWS CLI (and its dependencies)
* Add 'sync' unit test with moto_server

Side effects of this change:
* In the case of DSC workflows, the command will be used to sync data
from an S3 bucket to (an S3 bucket) in another AWS account. While the
implemented 'sync' DSC CLI command is quite simple, this relies on
configuring bucket policies and IAM Roles and policies to
enable cross-account data transfer.

Relevant ticket(s):
* https://mitlibraries.atlassian.net/browse/IN-1352

36 of 42 new or added lines in 2 files covered. (85.71%)

972 of 1004 relevant lines covered (96.81%)

0.97 hits per line