MITLibraries / hrqb-client / 9893464975

99%

main: 96%

push

github

Improve Quickbase upsert reporting

Why these changes are being introduced:

Formerly, each QuickbaseUpsertTask would individually log how metrics
about modified records.  This made it difficult to quickly get a holistic
picture of the run.  The methods were also logging oriented vs data oriented,
which made working with that data tricky.

How this addresses that need:
* Utilize luigi Events to pinpoint tasks only from the current run
* Shift parsing up upsert results to QBClient
* Add method HRQBPipelineTask.aggregate_upsert_results() to aggregate
all upsert results, across all Load tasks that fired during the run

By utilizing luigi Events, this also paves the way for better task
data cleanup, if we want to only pinpoint tasks that ran during the
last run (though not yet implemented).

Side effects of this change:
* All upsert results are written to logs in an aggregate form

Relevant ticket(s):
* https://mitlibraries.atlassian.net/browse/HRQB-20

51 of 53 new or added lines in 3 files covered. (96.23%)

914 of 920 relevant lines covered (99.35%)

0.99 hits per line