ghga-de / datahub-file-service / 21486105188

82%

push

github

DFHS Initial implementation (GSI-1993) (#2)

* Add port definition and model for file upload

* Push everything in case disaster strikes

* Still need to resolve config problems and add tests

* Add S3Cleaner entrypoint

* Add class comparison script

* Update config to use single private key and clean up class def diffs

* Change caching retry config so we don't spam retries on 500 responses

* Add cached and computed encrypted_part_count property to FileUpload model

* Add connection error translation to http module

* Fix encrypted secret encoding

* Replace direct obj storage use in S3Cleaner with S3Client

* Add error handling and further logic to S3Client class

* Rename CentralClient's storage alias param to inbox_storage_alias

* Update interrogator to use renamed s3client methods

* Update injection module

* Include 206 as a success code when downloading content range

* Add first slew of tests...

* Fix mypy errors

* Bas64-encode part md5 when generating url

* Fix crypt4gh and jwk config

* Place reencrypted chunks in upload buffer until part size is reached

* Add interrogation storage alias to CentralClient setup

* Fix offset and part range calculation on FileUpload model

* Update tests and add happy path integration test

* Fix S3 tests and md5 handling

* Refactor S3 error handling and update doc strings

* Add S3Client to comparison script

* Rework Interrogator error handling and update doc strings

* Rewrite part ranges function to adapt part size to segment size

* Download normal part sizes and examine those in 64k chunks

* Grab template updates

* Add download URL caching and a test

* Replace own constant with identical one from crypt4gh

* Add tests for FileUpload model

* Abort interrogation if FIS API is unreachable

* Add test for file not in inbox

* Add tests for decryption and checksum errors

* Fix httpx should mock

* Expand logging/error handling in S3Cleaner with some tests

* Fix empty bucket test

* Fix api unre... (continued)

615 of 735 new or added lines in 16 files covered. (83.67%)

624 of 763 relevant lines covered (81.78%)

0.82 hits per line