MITLibraries / s3-bagit-validator / 14620961261

89%

main: 90%

Build Type

Pull #69

github

Committed by ghukill

Commit Message

Bulk validate resume via previous results

Why these changes are being introduced:

The bulk-validate CLI command would heavily benefit from the ability to
resume a run, or retry failed AIPs from a run.

How this addresses that need:

As the bulk validation runs, results are incrementally written to
the output CSV.  This ensures that if there is some kind of error,
or the process quits unexpectedly, there is an artifact of what
has happened so far.

Additionally, when a bulk validation runs starts if a file already
exists for the given output CSV, the results are considered for
the run: any valid AIPs are skipped and any failed AIPs are
retried if --retry-failed flag is set, otherwise they are skipped
as well.

The net effect is that bulk validation is more fault tolerant by
allowing a user to "resume" a run as long as they provide the
same output CSV filepath.

Side effects of this change:
* None

Relevant ticket(s):
* https://mitlibraries.atlassian.net/browse/IR-228

Pull Request Pull Request #69: IR 228 - resume/retry bulk validation

Coverage Stats

40 of 57 new or added lines in 1 file covered. (70.18%)

21 existing lines in 3 files now uncovered.

609 of 685 relevant lines covered (88.91%)

0.89 hits per line