MITLibraries / hrqb-client / 8901372230

93%

main: 96%

push

github

Base luigi Task and Targets for HRQB

Why these changes are being introduced:

This application will be oriented around Extract, Transform, and Load luigi
Tasks.  Each of these Tasks will likely either pickle pandas objects to disk,
or upsert data to Quickbase.  The base Task and Targets here will be used
throughout the actual ETL luigi tasks defined.

How this addresses that need:
* HRQBLocalTarget extends luigi.LocalTarget to have a table name property
* PandasPickleTarget extends HRQBLocalTarget to read/write pandas objects to disk
* QuickbaseTableTarget extends HRQBLocalTarget to read/write Quickbase API writes
* HRQBTask extends luigi.Task to provide some convenience methods used by many Tasks
* PandasPickleTask extends HRQBTask to provide PandasPickleTarget defaults
* QuickbaseTableUpsert is similar to PandasPickleTask
* HRQBPipelineTask is a special luigi wrapper class meant to trigger other tasks but
not have output.  Will be used for defining a pipeline (set of tasks).
* extract, transform, and load files scaffolded for future Tasks

Side effects of this change:
* None

Relevant ticket(s):
* https://mitlibraries.atlassian.net/browse/HRQB-10

0 of 71 new or added lines in 4 files covered. (0.0%)

41 of 112 relevant lines covered (36.61%)

0.37 hits per line