Lattice-Data / checkfiles
70%

DEFAULT BRANCH: main
Repo Added
Token hySe4euPMDFHvmUnMWmN6Ja5KIXuKRAYM regen
Build 22 Last
Files 18
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH main
branch: main
CHANGE BRANCH
x
Sync Branches

coverage: 70.336% (+0.7%) from 69.611%
18047532430

push

github

web-flow 
beta  (#9)

* Update production tag in configuration to 'base64' for consistency in Checkfiles validation service

* Enhance checkfiles command and S3 upload functionality

- Added support for uploading validation logs to S3 directly from the checkfiles command, improving the workflow for log management.
- Introduced a new function `upload_validation_log_to_s3` to handle the S3 upload process, including error handling and logging.
- Updated the `run_checkfiles_command` to include the new S3 upload logic and to verify the presence of the S3 upload info file.
- Enhanced the `upload_report_to_slack` function to retrieve the report from S3, ensuring that the latest validation report is used for uploads.
- Added comprehensive tests for the new upload functionality, ensuring robust error handling and validation of S3 interactions.
- Updated IAM policies in the CDK stack to allow necessary S3 actions for the Lambda functions involved in the checkfiles process.

* Refactor runner.py to streamline state machine parameters

- Removed unnecessary parameters 's3_upload_status' and 's3_key' from the state machine input mapping in the RunCheckfilesStepFunction class, simplifying the data structure and improving clarity.
- This change enhances the maintainability of the code by reducing complexity in the state machine configuration.

* Enhance log upload process in checkfiles.py

- Added a delay to allow the file system to flush writes before uploading logs to S3, improving reliability of log uploads.
- Implemented a subprocess call to force file system synchronization, ensuring all pending writes are committed prior to reading the log file.
- Enhanced logging to capture the sync process and any potential errors, contributing to better error handling and debugging capabilities.

* Enhance fetch_files_from_backend function in checkfiles.py

- Refactored the `fetch_files_from_backend` function to accept an optional `session` parameter, allowing for better session mana... (continued)

1674 of 2380 relevant lines covered (70.34%)

0.7 hits per line

Relevant lines Covered
2380 RELEVANT LINES 1674 COVERED LINES
0.7 HITS PER LINE
Source Files on main

Recent builds

Builds Branch Commit Type Ran Committer Via Coverage
18047532430 main beta (#9) * Update production tag in configuration to 'base64' for consistency in Checkfiles validation service * Enhance checkfiles command and S3 upload functionality - Added support for uploading validation logs to S3 directly from the chec... push web-flow github
70.34
15005471458 main fixing gzipped FASTQ validation (#8) * Refactor validation logic for gzipped files and add comprehensive tests - Updated `validate_local_file` and `validate_s3_file` functions to explicitly set `is_gzipped` parameter for stream validation, impro... push web-flow github
69.61
14940299599 main adding more documentation (#7) * Update configuration and documentation for Checkfiles validation service - Changed the production tag in `config.py` to 'v1.0.0-alpha.2' for better versioning clarity. - Revised the Docker README to enhance clari... push web-flow github
69.47
14933786091 main Merge pull request #6 from Lattice-Data/add-docs additional tests and documentation push web-flow github
69.47
14886396182 main Update production tag in configuration - Changed the 'checkfiles_tag_production' from 'move-test' to 'v1.0.0-alpha' in the configuration file to reflect the new versioning scheme. - This update ensures consistency with the project's release manag... push gabdank github
65.4
14886237217 main Merge pull request #5 from Lattice-Data/move-test adding tests push web-flow github
65.4
14869994543 main Merge pull request #4 push web-flow github
36.69
14802164586 main Merge pull request #3 from Lattice-Data/cdk-step-function integrating step function push web-flow github
76.17
See All Builds (22)

Badge your Repo: checkfiles

We detected this repo isn’t badged! Grab the embed code to the right, add it to your repo to show off your code coverage, and when the badge is live hit the refresh button to remove this message.
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

Refresh