ThreeSixtyGiving / datastore

91%

master: 91%

LAST BUILD ON BRANCH mw/update_dqt_lib

branch: mw/update_dqt_lib

CHANGE BRANCH

x

Reset

• mw/update_dqt_lib
• 102-prom
• add-region-country-to-geo-lookups
• add_coverage
• bc/add_test_coverage
• bc/remove-comment-ftc-downloader
• bc/rename-dashboard-to-status
• dashboard-dev-aug
• dependabot/pip/black-24.3.0
• dependabot/pip/django-2.2.18
• dependabot/pip/django-2.2.20
• dependabot/pip/django-2.2.21
• dependabot/pip/django-2.2.22
• dependabot/pip/django-2.2.24
• dependabot/pip/django-2.2.27
• dependabot/pip/django-2.2.28
• dependabot/pip/django-3.2.15
• dependabot/pip/django-3.2.17
• dependabot/pip/django-3.2.18
• dependabot/pip/django-3.2.19
• dependabot/pip/django-3.2.20
• dependabot/pip/django-4.2.17
• dependabot/pip/django-4.2.18
• dependabot/pip/django-4.2.22
• dependabot/pip/django-4.2.24
• dependabot/pip/django-filter-2.4.0
• dependabot/pip/djangorestframework-3.11.2
• dependabot/pip/requests-2.32.4
• dependabot/pip/sqlparse-0.4.2
• dependabot/pip/sqlparse-0.4.4
• dependabot/pip/urllib3-2.5.0
• dk-fix-issue-73
• dk/113-update-region-codes
• dk/update-findthatcharity
• docker-instructions
• eld/add_sentry
• fix-tests
• github-actions
• grantnav-issue-862
• grantnav-issue-943
• isort
• live
• master
• master-pre-mw/dash_extensions
• mm/change-schema-branch-name
• mw/50pc_external_quality_data
• mw/add_countryCode_codelist_support
• mw/add_funder_count
• mw/add_timestamp_output
• mw/additional_data_codelist_sources
• mw/additional_data_codelist_sources_add_geoCodeType_support
• mw/additional_data_codelists
• mw/additional_data_org_types_geo_locations
• mw/badge_name_typo
• mw/best_grant_impact_location
• mw/codelist-field-correction
• mw/dash_extensions
• mw/dashboard_aug
• mw/dashboard_changes
• mw/dashboard_jan
• mw/data_run_rm-f
• mw/deprecate_old_canonical_recipient
• mw/dont_update_prom_metrics_if_busy
• mw/dqt_upgrade_2
• mw/fix-publisher-recipient-stats
• mw/fix_delete_org_data_confirm_bug
• mw/fix_quality_grants_overview_stats
• mw/grantnav_geo_improvements
• mw/grants_ind_aggs2
• mw/grants_to_ind_aggs
• mw/handle_quality_fails
• mw/improve_weird_chars_id_output
• mw/include_additional_data_gn_package
• mw/individialsdetails
• mw/load_data_package
• mw/location_standard_changes
• mw/location_ward_info
• mw/new-aggregate-export-data
• mw/org-rework
• mw/pass-if-no-django-toolbar
• mw/pf/add_new_view
• mw/problem_char_v2
• mw/quality_dash_api_updates
• mw/quality_dashboard_recipient_ind
• mw/quality_data
• mw/quality_data_context_aware
• mw/readme
• mw/readme-coverage-branch
• mw/recipientLocation_add_country
• mw/recipient_type_additional_data
• mw/relative_path_datagetter
• mw/remove_datagetter_days
• mw/require_specific_version_datagetter
• mw/security_upgrades
• mw/settings
• mw/source_file_failure
• mw/temp-sources-fix
• mw/update-drf
• mw/update-reqs
• mw/update-requirements
• mw/update_datagetter
• mw/update_datagetter_2024
• mw/update_datagetter_2025
• mw/update_datagetter_version
• mw/update_req
• mw/update_tsg_org_types
• mw/use_latest_publisher_ob
• mw/use_lsoa21
• mw/workflow-updates
• org-output
• rg/add-code-list-name-migration
• rg/add-debug-toolbar
• rg/add-experimental-org-api-to-api-index
• rg/additional-data-imd-snapshot
• rg/additional-data-updates
• rg/api-4-tests
• rg/api-fix-grantsview-404-behaviour
• rg/api-fix-org-detail-performance
• rg/api-grant-data-license
• rg/api-non-primary-org-ids
• rg/api-org-grants-data
• rg/api-performance-improvements
• rg/api-schema-1
• rg/api-schema-2
• rg/api-schema-3
• rg/api-schema-and-docs
• rg/api-v1-urls
• rg/better-list-datagetter-runs
• rg/dev-settings
• rg/exclude-grants-with-bad-data
• rg/experimental-org-api
• rg/export-non-primary-orgids-in-data-package
• rg/fix-delete-all-not-in-use
• rg/fix-new-grants-convenience-fields
• rg/fix-publisher-non-primary-matching
• rg/ftc-parameter
• rg/geocode-lookup-metric
• rg/getterrun-swap-last-for-latest
• rg/grant-indexes
• rg/hotpatch-not-bad-grants-fix
• rg/import-org-additional-data-atomic
• rg/make-load-all-org-data-executable
• rg/monitoring-data-run
• rg/monitoring-fixes-2
• rg/monitoring-fuzzy-days
• rg/monitoring-import-export
• rg/monitoring-per-publisher-downtime
• rg/monitoring-sf-access-url-and-downtime
• rg/org-api-tests
• rg/public-api-new-ftc-linked-orgs-field
• rg/public-api-rate-limit
• rg/publisher-etc-monitoring
• rg/publisher-monitoring-tests
• rg/quickfix-ftc-requests-user-agent
• rg/rewrite-additional-data-memory
• rg/smart-getterrun-retention
• rg/update-datagetter
• rg/update-deprecated-unique-constraint
• rg/update-docker-compose
• rg/update-entities-transaction
• rg/upgrade-certifi
• rg/upgrade-django-4
• rg/upgrade-python-3.12
• rhiaro/213-licenceinfo
• rollback_pre_dqt_datagetter_upgrade
• tw/ext_org_ids
• update-flattentool
• update-python
• upgrade-django

Build # 15614875144

Build Type

Pull #285

github

Committed by michaelwood

Commit Message

rewrite_quality_data: Avoid thread hanging and error condition failure

Although we handle exceptions in the process_source_file this isn't
sufficient to prevent:
1) "None" being added to the array of results
2) The thread never being able to rejoin the parent due to the exception
   not getting handled a second time.

This is "belt and braces" to protect the data pipeline in doing its
core feature of outputting the data packages.

Pull Request Pull Request #285: Mw/update dqt lib and add resiliance for process sourcefile

Run Details

3 of 7 new or added lines in 1 file covered. (42.86%)

3501 of 3838 relevant lines covered (91.22%)

0.91 hits per line