Lattice-Data / checkfiles / 18047532430

Build # 18047532430

Build Type

push

github

Committed by web-flow

Commit Message

beta  (#9)

* Update production tag in configuration to 'base64' for consistency in Checkfiles validation service

* Enhance checkfiles command and S3 upload functionality

- Added support for uploading validation logs to S3 directly from the checkfiles command, improving the workflow for log management.
- Introduced a new function `upload_validation_log_to_s3` to handle the S3 upload process, including error handling and logging.
- Updated the `run_checkfiles_command` to include the new S3 upload logic and to verify the presence of the S3 upload info file.
- Enhanced the `upload_report_to_slack` function to retrieve the report from S3, ensuring that the latest validation report is used for uploads.
- Added comprehensive tests for the new upload functionality, ensuring robust error handling and validation of S3 interactions.
- Updated IAM policies in the CDK stack to allow necessary S3 actions for the Lambda functions involved in the checkfiles process.

* Refactor runner.py to streamline state machine parameters

- Removed unnecessary parameters 's3_upload_status' and 's3_key' from the state machine input mapping in the RunCheckfilesStepFunction class, simplifying the data structure and improving clarity.
- This change enhances the maintainability of the code by reducing complexity in the state machine configuration.

* Enhance log upload process in checkfiles.py

- Added a delay to allow the file system to flush writes before uploading logs to S3, improving reliability of log uploads.
- Implemented a subprocess call to force file system synchronization, ensuring all pending writes are committed prior to reading the log file.
- Enhanced logging to capture the sync process and any potential errors, contributing to better error handling and debugging capabilities.

* Enhance fetch_files_from_backend function in checkfiles.py

- Refactored the `fetch_files_from_backend` function to accept an optional `session` parameter, allowing for better session mana... (continued)

Run Details

1674 of 2380 relevant lines covered (70.34%)

0.7 hits per line

Source File

Press 'n' to go to next uncovered line, 'b' for previous

Source Not Available