ThreeSixtyGiving / datastore

91%

master: 91%

LAST BUILD ON BRANCH mw/fix_quality_grants_overview_stats

branch: mw/fix_quality_grants_overview_stats

CHANGE BRANCH

x

Reset

• mw/fix_quality_grants_overview_stats
• 102-prom
• add-region-country-to-geo-lookups
• add_coverage
• bc/add_test_coverage
• bc/remove-comment-ftc-downloader
• bc/rename-dashboard-to-status
• dashboard-dev-aug
• dependabot/pip/black-24.3.0
• dependabot/pip/django-2.2.18
• dependabot/pip/django-2.2.20
• dependabot/pip/django-2.2.21
• dependabot/pip/django-2.2.22
• dependabot/pip/django-2.2.24
• dependabot/pip/django-2.2.27
• dependabot/pip/django-2.2.28
• dependabot/pip/django-3.2.15
• dependabot/pip/django-3.2.17
• dependabot/pip/django-3.2.18
• dependabot/pip/django-3.2.19
• dependabot/pip/django-3.2.20
• dependabot/pip/django-4.2.17
• dependabot/pip/django-4.2.18
• dependabot/pip/django-4.2.22
• dependabot/pip/django-4.2.24
• dependabot/pip/django-filter-2.4.0
• dependabot/pip/djangorestframework-3.11.2
• dependabot/pip/requests-2.32.4
• dependabot/pip/sqlparse-0.4.2
• dependabot/pip/sqlparse-0.4.4
• dependabot/pip/urllib3-2.5.0
• dk-fix-issue-73
• dk/113-update-region-codes
• dk/update-findthatcharity
• docker-instructions
• eld/add_sentry
• fix-tests
• github-actions
• grantnav-issue-862
• grantnav-issue-943
• isort
• live
• master
• master-pre-mw/dash_extensions
• mm/change-schema-branch-name
• mw/50pc_external_quality_data
• mw/add_countryCode_codelist_support
• mw/add_funder_count
• mw/add_timestamp_output
• mw/additional_data_codelist_sources
• mw/additional_data_codelist_sources_add_geoCodeType_support
• mw/additional_data_codelists
• mw/additional_data_org_types_geo_locations
• mw/badge_name_typo
• mw/best_grant_impact_location
• mw/codelist-field-correction
• mw/dash_extensions
• mw/dashboard_aug
• mw/dashboard_changes
• mw/dashboard_jan
• mw/data_run_rm-f
• mw/deprecate_old_canonical_recipient
• mw/dont_update_prom_metrics_if_busy
• mw/dqt_upgrade_2
• mw/fix-publisher-recipient-stats
• mw/fix_delete_org_data_confirm_bug
• mw/grantnav_geo_improvements
• mw/grants_ind_aggs2
• mw/grants_to_ind_aggs
• mw/handle_quality_fails
• mw/improve_weird_chars_id_output
• mw/include_additional_data_gn_package
• mw/individialsdetails
• mw/load_data_package
• mw/location_standard_changes
• mw/location_ward_info
• mw/new-aggregate-export-data
• mw/org-rework
• mw/pass-if-no-django-toolbar
• mw/pf/add_new_view
• mw/problem_char_v2
• mw/quality_dash_api_updates
• mw/quality_dashboard_recipient_ind
• mw/quality_data
• mw/quality_data_context_aware
• mw/readme
• mw/readme-coverage-branch
• mw/recipientLocation_add_country
• mw/recipient_type_additional_data
• mw/relative_path_datagetter
• mw/remove_datagetter_days
• mw/require_specific_version_datagetter
• mw/security_upgrades
• mw/settings
• mw/source_file_failure
• mw/temp-sources-fix
• mw/update-drf
• mw/update-reqs
• mw/update-requirements
• mw/update_datagetter
• mw/update_datagetter_2024
• mw/update_datagetter_2025
• mw/update_datagetter_version
• mw/update_dqt_lib
• mw/update_req
• mw/update_tsg_org_types
• mw/use_latest_publisher_ob
• mw/use_lsoa21
• mw/workflow-updates
• org-output
• rg/add-code-list-name-migration
• rg/add-debug-toolbar
• rg/add-experimental-org-api-to-api-index
• rg/additional-data-imd-snapshot
• rg/additional-data-updates
• rg/api-4-tests
• rg/api-fix-grantsview-404-behaviour
• rg/api-fix-org-detail-performance
• rg/api-grant-data-license
• rg/api-non-primary-org-ids
• rg/api-org-grants-data
• rg/api-performance-improvements
• rg/api-schema-1
• rg/api-schema-2
• rg/api-schema-3
• rg/api-schema-and-docs
• rg/api-v1-urls
• rg/better-list-datagetter-runs
• rg/dev-settings
• rg/exclude-grants-with-bad-data
• rg/experimental-org-api
• rg/export-non-primary-orgids-in-data-package
• rg/fix-delete-all-not-in-use
• rg/fix-new-grants-convenience-fields
• rg/fix-publisher-non-primary-matching
• rg/ftc-parameter
• rg/geocode-lookup-metric
• rg/getterrun-swap-last-for-latest
• rg/grant-indexes
• rg/hotpatch-not-bad-grants-fix
• rg/import-org-additional-data-atomic
• rg/make-load-all-org-data-executable
• rg/monitoring-data-run
• rg/monitoring-fixes-2
• rg/monitoring-fuzzy-days
• rg/monitoring-import-export
• rg/monitoring-per-publisher-downtime
• rg/monitoring-sf-access-url-and-downtime
• rg/org-api-tests
• rg/public-api-new-ftc-linked-orgs-field
• rg/public-api-rate-limit
• rg/publisher-etc-monitoring
• rg/publisher-monitoring-tests
• rg/quickfix-ftc-requests-user-agent
• rg/rewrite-additional-data-memory
• rg/smart-getterrun-retention
• rg/update-datagetter
• rg/update-deprecated-unique-constraint
• rg/update-docker-compose
• rg/update-entities-transaction
• rg/upgrade-certifi
• rg/upgrade-django-4
• rg/upgrade-python-3.12
• rhiaro/213-licenceinfo
• rollback_pre_dqt_datagetter_upgrade
• tw/ext_org_ids
• update-flattentool
• update-python
• upgrade-django

Build # 12354671390

Build Type

Pull #247

github

Committed by michaelwood

Commit Message

quality_data: overview: Fix incorrect field summing passing grants

When calculating the overview stats for /api/dashboard/overview?mode=grants
the logic of fail/pass was being used to count the total number of
grants which passed by just using the total number of grants that are in
the source file.

This is only correct some of the time because it just so happens that if a
publisher is likely to be missing a field (such as company number) then they
will miss it out of all of their datasets so the calculation is coincidentally
correct. Some publishers however do add such fields for _some_ of their
grants which makes this calculation completely incorrect for certain
metrics.

This change makes sure we use the total count from the DQT (library)
rather than the aggregated data.

Updates basic test data to corrected value.

Fixes: https://github.com/ThreeSixtyGiving/datastore/issues/246

Pull Request Pull Request #247: quality_data: overview: Fix incorrect field summing passing grants

Run Details

5 of 6 new or added lines in 1 file covered. (83.33%)

2941 of 3244 relevant lines covered (90.66%)

0.91 hits per line