OpenTreeMap / otm-core

82%

develop: 84%

LAST BUILD ON BRANCH feature/jcw/speed-up-user-audit-query

branch: feature/jcw/speed-up-user-audit-query

CHANGE BRANCH

x

Reset

• feature/jcw/speed-up-user-audit-query
• 2.1.0
• 2.1.1
• 2.10.0
• 2.11.0
• 2.11.1
• 2.12.0
• 2.13.0
• 2.13.1
• 2.14.0
• 2.15.0
• 2.16.0
• 2.17.0
• 2.17.1
• 2.19.0
• 2.19.1
• 2.19.2
• 2.20.0
• 2.21.0
• 2.21.1
• 2.21.2
• 2.21.3
• 2.21.4
• 2.21.5
• 2.21.6
• 2.21.7
• 2.22.0
• 2.22.1
• 2.3.0
• 2.4.0
• 2.5.0
• 2.5.1
• 2.6.0
• 2.7.0
• 2.8.0
• 2.9.0
• add-suppport-for-inline-translations-to-field-tags
• admin-styling
• bug/sml/admin-treemap-title-wrapping
• custom-field-mgmt-ui
• dependabot/npm_and_yarn/extend-3.0.2
• dependabot/npm_and_yarn/fstream-1.0.12
• dependabot/npm_and_yarn/http-proxy-1.18.1
• dependabot/npm_and_yarn/ini-1.3.7
• dependabot/npm_and_yarn/lodash-4.17.13
• dependabot/npm_and_yarn/lodash-4.17.19
• dependabot/npm_and_yarn/node-sass-4.13.1
• dependabot/npm_and_yarn/sockjs-0.3.21
• dependabot/npm_and_yarn/websocket-extensions-0.1.4
• dependabot/npm_and_yarn/y18n-3.2.2
• dependabot/pip/django-1.11.28
• dependabot/pip/django-1.11.29
• dependabot/pip/pillow-6.2.0
• develop
• feature/hmc/bump-pyrollbar
• feature/hmc/django-site-domain
• feature/hmc/omgeo-with-requests
• feature/hmc/remove-circle-ci
• feature/hmc/remove-statsd
• feature/hmc/statsd-celery
• feature/hmc/systemd-non-ascii-output
• feature/jcw/add-favs-to-profile
• feature/jcw/api-docs-and-tool
• feature/jcw/comment-out-importer-support-for-read-only
• feature/jcw/mapfeature-updated-at
• feature/jcw/new-modeling-header
• feature/jcw/remove-gender-from-species
• feature/jcw/show-alt-no-favs-message
• feature/jcw/show-import-update-date-rather-than-id
• feature/jcw/speed-up-instance-edits-page
• feature/mw/advanced-search-style
• feature/mw/dashboard-header-fix
• feature/mw/dashboard-style-cleanup
• feature/mw/global-spinner
• feature/mw/highlight-color
• feature/mw/mgmt-cleanup
• feature/mw/mgmt-padding-issue
• feature/mw/modal-warning
• feature/mw/modeling-btn-fix
• feature/mw/modeling-num-input-fixes
• feature/mw/modeling-planlist
• feature/mw/modeling-sidebar-toolbar-fixes
• feature/mw/modeling-toolbar-small-desktops-fix
• feature/mw/modelling-cleanup
• feature/mw/multiselect-style-fix
• feature/mw/new-field-highlight
• feature/mw/plan-modal-labels
• feature/mw/scenario-protoype
• feature/mw/suspend-map-cleanup
• feature/mw/updated-spinner
• feature/provision-py3
• feature/sml/location-search-marker
• feature/sml/management-sidebar-click-targets
• feature/sml/profile-name-wrapping-bug
• feature/tnation/rollbar
• feature/tnation/rollbar-upgrade
• fix/jcw/double-wrapped-api-species-view
• fix/jcw/fix-species-import-guide-link
• fix/jcw/make-import-statuses-unique
• fix/jcw/remove-gender-fieldpermissions
• fix/jcw/remove-gender-fieldpermissions-2
• foo
• gitter
• hotfix/2.1.1
• hotfix/2.11.1
• hotfix/2.13.1
• hotfix/2.17.1
• hotfix/2.19.1
• hotfix/2.19.2
• hotfix/2.21.1
• hotfix/2.21.2
• hotfix/2.21.3
• hotfix/2.21.4
• hotfix/2.21.5
• hotfix/2.21.6
• hotfix/2.21.7
• hotfix/2.22.1
• hotfix/2.22.2
• hotfix/2.5.1
• jcw/bryant3
• jcw/catch-basemap-crash
• jcw/date-picker-bug
• jcw/delete-tree-with-hidden-id
• jcw/dependency-updates
• jcw/encode-geocode-param
• jcw/fix-basemap-crash-hotfix
• jcw/fix-basemap-restore
• jcw/fix-geocode-test
• jcw/fix-invitation-activation
• jcw/fix-modeling-csv
• jcw/fix-udf-helper
• jcw/handle-basemap-pref-not-available
• jcw/hidden-tree-id-can-be-shown
• jcw/jquery-3
• jcw/js-deps
• jcw/mobile-safari-layer-selector
• jcw/photo-export
• jcw/registration-recaptcha
• jcw/tree-edit-perm
• jcw/udf-choice-whitespace
• jcw/update-djqscsv-and-tinsel
• jcw/update-moment
• jcw/update-urllib3
• jw/export-units-fix
• jw/units-fix
• kd/clear-boundary-polygon
• kd/numeric-udf-validation
• kd/protected-udf
• kd/restrict-polygonal-features
• kd/sequential-reference-number
• kd/wm-task-priority
• kd/wm-teams
• kd/wmm-custom-fields
• kd/works-management
• list
• master
• maurizi-requires-io
• modeling-chart-labels-2
• modeling-mortality-params
• modular-pricing
• otm-bryant2
• otm-cloud
• python3
• release
• release-2.0
• release/2.1.0
• release/2.10.0
• release/2.11.0
• release/2.12.0
• release/2.13.0
• release/2.14.0
• release/2.15.0
• release/2.16.0
• release/2.17.0
• release/2.19.0
• release/2.2.0
• release/2.20.0
• release/2.21.0
• release/2.22.0
• release/2.3.0
• release/2.4.0
• release/2.5.0
• release/2.6.0
• release/2.7.0
• release/2.8.0
• release/2.9.0
• remove-unused-provisioning-items
• requires-io-develop
• requires-io-master
• revert-2233-remove-unused-provisioning-items
• revert-2483-feature/tune-rate-limiting
• revert-2485-revert-2483-feature/tune-rate-limiting
• rtb/custom-field-reuse-fix
• rtb/default-static-pages
• rtb/defer-choice-delete
• rtb/ecobenefits-totals
• rtb/export-comments-button
• rtb/fix-detail-search
• rtb/hstore
• rtb/importer-proximity
• rtb/no-double-quotes
• rtb/no-polygon-on-mobile
• rtb/skipped-bioswale-test
• rtb/static-page-edit-default
• rtb/stewardship-search-date-bug
• split-modeling
• udf-date-validation
• update-python-deps-1.8

Build # 326

Build Type

push

travis-ci

Committed by jwalgran

Commit Message

Update user audit query to be more efficient

Trying to view user profiles other than your own with a large database
was triggering a 500 response. The root cause was the fact that the ORM
was generating a 2.2MB SQL statement, causing Postgres to choke with a
stack error.

The audit query code in lib/user.py builds up the final QuerySet by
gluing together several small parts. I discovered that the same instance
was being fed into the filter multiple times, causing the ORM to
generate a lot of duplicate clauses, bloating the query. Adding a
``distinct`` took care of that.

The fast majority of users will only edit a single instance, and our
audit table is optimized for filtering by instance ID, so I added an
additional query to fetch the ids of a users edited instances and use
that to limit the scope of the query, rather than limiting it to the
larger list of visible instances.

Run Details

13929 of 17061 relevant lines covered (81.64%)

0.82 hits per line