dtcenter / METplus / 35256090102

93%

Build Type

push

github

Committed by web-flow

Commit Message

Feature #3218 warn overwrite, #2110 fail if output exists (#3332)

* Per #3218, track output file regex strings and warn when a path is being overwritten, indicating that a misconfiguration is present. Pass list of outputs to the next wrapper in the process list so it can properly check if multiple instances are writing to the same path. Move logic to run processes into a function that can be called by unit tests to test behavior from running multiple processes, e.g. multiple instances of the same wrapper.

* add pytest fixture to handle checking of logs for warnings that report a duplicate file path is being overwritten

* Per #3218, add unit tests for function that checks/warns when paths are being overwritten

* Per #3218, add unit tests for app that writes multiple output files to a directory

* Per #3218, add unit tests to app that writes a single output file, testing various scenarios such as: 2 fields processed separately, runtime freq doesn't match output template, custom loop list, and multiple instances. This includes updated test logic to properly test multiple PROCESS_LIST items instead of creating/calling a single wrapper/process as many test do. Consider adding a fixture to easily handle the config and process list initialization that can be resused in many tests

* add documentation to better describe how output paths are created from templates for each of the RUNTIME_FREQ settings

* add log message to suggest how to resolve duplicate file overwrite issues by changing the output template and/or prefix

* add option to override default value to use if config wrapper or generic variables are not set

* add config option to turn off output overwrite checks to prevent warnings if desired

* bump pillow to fix security vulnerability

* Enhance helper script to generate steps to 1) add config support for multiple wrappers at once and 2) skip steps specific to MET config variables

* Per #3218, do not produce warnings if SKIP_IF_OUTPUT_EXISTS... (continued)

Coverage Stats

114 of 119 new or added lines in 17 files covered. (95.8%)

1 existing line in 1 file now uncovered.

9802 of 10639 relevant lines covered (92.13%)

1.84 hits per line