MITLibraries / timdex-dataset-api / 12162317877

99%

main: 93%

Pull #9

github

Move dataset location parsing to init

Why these changes are being introduced:

When loading a dataset for reading, or preparing to write, the
appropriate pyarrow filesystem must be set and the location normalized
(e.g. removing s3:// prefix).

How this addresses that need:
* Move call of self.parse_location to init as always required
* An invalid location will fail early before an attempt to read or write
* Previously missing, add filesystem to write method to support S3
writes

Side effects of this change:
* None

Relevant ticket(s):
* https://mitlibraries.atlassian.net/browse/TIMX-415

124 of 126 new or added lines in 5 files covered. (98.41%)

125 of 127 relevant lines covered (98.43%)

0.98 hits per line