ThreeSixtyGiving / dataquality

81%

LAST BUILD ON BRANCH main

branch: main

CHANGE BRANCH

x

Reset

Sync Branches

• main
• bc/update-address-and-charity-number
• bc/update-insights-to-grantvis-maps
• bc/update-libcoveweb-v0.31.2
• ca/xlsx-robustness-fix
• issue-140
• issue-140-diff-branch
• mariongalley-patch-1
• mm-replace-copy
• mm/change-default-schema-branch
• mw/TestType_change
• mw/add_coverage_btn
• mw/collect_stats_on_data_issues
• mw/country_code_checks
• mw/ensure_check_duration_not_none
• mw/fix_truthy_classes_list
• mw/footer-year
• mw/grants_individuals_skip_recipient_location
• mw/update_tools_for_new_lib
• mw/various_post_new_dqt_fixes
• tw/use-local-schema
• update-terms-and-conditions

Build # 25726671580

Build Type

push

github

Committed by chrisarridge

Commit Message

cove: add check for large number of XLSX sheet rows

This commit fixes an issue where XLSX spreadsheets with a malformed
number of rows generate very large unflattened JSON files.  The fix
simply checks for an excessive number of rows before passing off
to cove/flattentool.  The number of rows considered excessive is
controllable through the MAX_XLSX_ROWS environment variable and set
by default to 50000.

Coverage Stats

64 of 79 relevant lines covered (81.01%)

0.81 hits per line