MITLibraries / hrqb-client

99%

main: 96%

LAST BUILD ON BRANCH HRQB-32-memory-footprint

branch: HRQB-32-memory-footprint

CHANGE BRANCH

x

Reset

• HRQB-32-memory-footprint
• Add-actions
• HRQB-10-scaffold-task-and-target-classes
• HRQB-11-dw-connection-and-helpers
• HRQB-12-setup-qb-api-and-load-tasks
• HRQB-13-pipelines-and-cli
• HRQB-13-refactor-base-tasks
• HRQB-15-libhr-table
• HRQB-17-functional-tasks-and-pipelines-prep
• HRQB-17-pipeline-task-isolation
• HRQB-18-connection-tests
• HRQB-20-contd-termination-work
• HRQB-20-termination-reason-last-appt-only
• HRQB-20-termination-reasons
• HRQB-20-termination-types
• HRQB-21-employees-table
• HRQB-22-adjustments-for-retirements
• HRQB-22-employee-appointments
• HRQB-23-employee-salary-history
• HRQB-27-runtime-assertions
• HRQB-28-employee-leave
• HRQB-29-log-load-counts
• HRQB-34-base-salary-change-percent
• HRQB-35-generate-performance-reviews
• HRQB-36-md5-hash-merge-fields
• HRQB-39-remove-sensitive-logging-sentry
• HRQB-40-fix-merge-without-appointment-key
• HRQB-40-remove-volatile-key-fields
• HRQB-41-improve-dev-target-removal
• HRQB-43-production-deploy
• HRQB-44-resolve-lingering-upsert-issues
• HRQB-46-sentry-message-on-upsert-errors
• HRQB-49-historical-libhr-data
• HRQB-51-employee-leave-balances
• HRQB-52-salary-history-updates
• HRQB-54-add-employee-ethnicity
• HRQB-57-many-to-one-libhr-rows
• HRQB-7-app-scaffold
• IN-1194-merge-libhr-tables
• IN-1200-remove-2019-data-cutoff
• IN-1208-add-employee-fields
• IN-1240-pip-audit-and-maintenance
• IN-1329-2025-06-23-maintenance
• IN-1439-stale-salary-history-rows
• IN-1450-adjust-salary-history-query-and-key-logic
• bug-remove-appt-end-date-from-key
• dependabot/pip/black-24.4.2
• dependabot/pip/boto3-1.34.116
• dependabot/pip/boto3-1.34.117
• dependabot/pip/boto3-1.34.118
• dependabot/pip/boto3-1.34.119
• dependabot/pip/boto3-1.34.120
• dependabot/pip/boto3-1.34.121
• dependabot/pip/boto3-1.34.122
• dependabot/pip/boto3-1.34.123
• dependabot/pip/boto3-1.34.124
• dependabot/pip/boto3-1.34.125
• dependabot/pip/boto3-1.34.126
• dependabot/pip/boto3-1.34.127
• dependabot/pip/boto3-1.34.128
• dependabot/pip/boto3-1.34.129
• dependabot/pip/boto3-1.34.130
• dependabot/pip/boto3-1.34.131
• dependabot/pip/boto3-1.34.132
• dependabot/pip/boto3-1.34.133
• dependabot/pip/boto3-1.34.134
• dependabot/pip/boto3-1.34.135
• dependabot/pip/boto3-1.34.136
• dependabot/pip/boto3-1.34.137
• dependabot/pip/boto3-1.34.138
• dependabot/pip/boto3-1.34.139
• dependabot/pip/boto3-1.34.140
• dependabot/pip/boto3-1.34.141
• dependabot/pip/boto3-1.34.142
• dependabot/pip/boto3-1.34.143
• dependabot/pip/boto3-1.34.144
• dependabot/pip/boto3-1.34.145
• dependabot/pip/boto3-1.34.146
• dependabot/pip/boto3-1.34.147
• dependabot/pip/boto3-1.34.148
• dependabot/pip/boto3-1.34.149
• dependabot/pip/boto3-1.34.150
• dependabot/pip/boto3-1.34.151
• dependabot/pip/boto3-1.34.152
• dependabot/pip/boto3-1.34.153
• dependabot/pip/boto3-1.34.154
• dependabot/pip/boto3-1.34.155
• dependabot/pip/boto3-1.34.156
• dependabot/pip/boto3-1.34.157
• dependabot/pip/boto3-1.34.158
• dependabot/pip/boto3-1.34.159
• dependabot/pip/boto3-1.34.160
• dependabot/pip/boto3-1.34.161
• dependabot/pip/boto3-1.34.162
• dependabot/pip/boto3-1.35.0
• dependabot/pip/boto3-1.35.1
• dependabot/pip/boto3-1.35.10
• dependabot/pip/boto3-1.35.11
• dependabot/pip/boto3-1.35.12
• dependabot/pip/boto3-1.35.13
• dependabot/pip/boto3-1.35.14
• dependabot/pip/boto3-1.35.15
• dependabot/pip/boto3-1.35.16
• dependabot/pip/boto3-1.35.17
• dependabot/pip/boto3-1.35.18
• dependabot/pip/boto3-1.35.19
• dependabot/pip/boto3-1.35.2
• dependabot/pip/boto3-1.35.20
• dependabot/pip/boto3-1.35.21
• dependabot/pip/boto3-1.35.22
• dependabot/pip/boto3-1.35.23
• dependabot/pip/boto3-1.35.24
• dependabot/pip/boto3-1.35.25
• dependabot/pip/boto3-1.35.26
• dependabot/pip/boto3-1.35.27
• dependabot/pip/boto3-1.35.28
• dependabot/pip/boto3-1.35.29
• dependabot/pip/boto3-1.35.3
• dependabot/pip/boto3-1.35.30
• dependabot/pip/boto3-1.35.31
• dependabot/pip/boto3-1.35.32
• dependabot/pip/boto3-1.35.33
• dependabot/pip/boto3-1.35.34
• dependabot/pip/boto3-1.35.35
• dependabot/pip/boto3-1.35.4
• dependabot/pip/boto3-1.35.5
• dependabot/pip/boto3-1.35.6
• dependabot/pip/boto3-1.35.7
• dependabot/pip/boto3-1.35.8
• dependabot/pip/boto3-1.35.9
• dependabot/pip/certifi-2024.7.4
• dependabot/pip/coveralls-4.0.0
• dependabot/pip/coveralls-4.0.1
• dependabot/pip/cryptography-43.0.1
• dependabot/pip/ipython-8.25.0
• dependabot/pip/ipython-8.26.0
• dependabot/pip/mypy-1.10.0
• dependabot/pip/mypy-1.10.1
• dependabot/pip/mypy-1.11.0
• dependabot/pip/mypy-1.11.1
• dependabot/pip/mypy-1.11.2
• dependabot/pip/oracledb-2.2.0
• dependabot/pip/oracledb-2.2.1
• dependabot/pip/pandas-stubs-2.2.2.240514
• dependabot/pip/pandas-stubs-2.2.2.240603
• dependabot/pip/pre-commit-3.7.1
• dependabot/pip/pytest-8.2.2
• dependabot/pip/requests-2.32.0
• dependabot/pip/requests-2.32.2
• dependabot/pip/requests-2.32.3
• dependabot/pip/ruff-0.4.10
• dependabot/pip/ruff-0.4.2
• dependabot/pip/ruff-0.4.3
• dependabot/pip/ruff-0.4.4
• dependabot/pip/ruff-0.4.5
• dependabot/pip/ruff-0.4.6
• dependabot/pip/ruff-0.4.7
• dependabot/pip/ruff-0.4.8
• dependabot/pip/ruff-0.4.9
• dependabot/pip/ruff-0.5.0
• dependabot/pip/ruff-0.5.1
• dependabot/pip/ruff-0.5.2
• dependabot/pip/ruff-0.5.3
• dependabot/pip/ruff-0.5.4
• dependabot/pip/ruff-0.5.5
• dependabot/pip/ruff-0.5.6
• dependabot/pip/ruff-0.5.7
• dependabot/pip/ruff-0.6.0
• dependabot/pip/ruff-0.6.1
• dependabot/pip/ruff-0.6.2
• dependabot/pip/ruff-0.6.3
• dependabot/pip/ruff-0.6.4
• dependabot/pip/ruff-0.6.5
• dependabot/pip/ruff-0.6.6
• dependabot/pip/ruff-0.6.7
• dependabot/pip/ruff-0.6.8
• dependabot/pip/ruff-0.6.9
• dependabot/pip/sentry-sdk-2.0.1
• dependabot/pip/sentry-sdk-2.1.1
• dependabot/pip/sentry-sdk-2.10.0
• dependabot/pip/sentry-sdk-2.11.0
• dependabot/pip/sentry-sdk-2.12.0
• dependabot/pip/sentry-sdk-2.13.0
• dependabot/pip/sentry-sdk-2.14.0
• dependabot/pip/sentry-sdk-2.15.0
• dependabot/pip/sentry-sdk-2.2.0
• dependabot/pip/sentry-sdk-2.2.1
• dependabot/pip/sentry-sdk-2.3.1
• dependabot/pip/sentry-sdk-2.4.0
• dependabot/pip/sentry-sdk-2.5.0
• dependabot/pip/sentry-sdk-2.5.1
• dependabot/pip/sentry-sdk-2.6.0
• dependabot/pip/sentry-sdk-2.7.0
• dependabot/pip/sentry-sdk-2.7.1
• dependabot/pip/sentry-sdk-2.8.0
• dependabot/pip/sentry-sdk-2.9.0
• dependabot/pip/sqlalchemy-2.0.31
• dependabot/pip/tornado-6.4.1
• dependabot/pip/types-requests-2.32.0.20240602
• dependabot/pip/types-requests-2.32.0.20240622
• dependabot/pip/types-requests-2.32.0.20240712
• dependabot/pip/types-requests-2.32.0.20240905
• dependabot/pip/types-requests-2.32.0.20240907
• dependabot/pip/types-requests-2.32.0.20240914
• dependabot/pip/urllib3-2.2.2
• main
• refs/tags/v1.0.0
• refs/tags/v1.0.1
• refs/tags/v1.0.2
• stage-workflow
• update-emp-sal-hist-merge-field
• v.1.4.2
• v.1.4.3
• v1.1.0
• v1.10.0
• v1.11.0
• v1.11.1
• v1.12.0
• v1.13.0
• v1.2.0
• v1.3.0
• v1.4.0
• v1.4.1
• v1.5.0
• v1.6.0
• v1.7.0
• v1.8.0
• v1.9.0

Build # 9487213336

Build Type

push

github

Committed by ghukill

Commit Message

Optional field return on QB upsert

Why these changes are being introduced:

It was determined that large upserts could result in memory pressure
due to the response from quickbase which returned every record
and field value.  The memory footprint for upserting is reasonble,
but this JSON response was parsed into memory which is not memory
efficient.

Ultimately, the modified record field values are not needed most
of the time, so defaults to not returning.  But if the flag
is set to True, field values can be returned on upsert if needed.

How this addresses that need:
* Add return_fields parameter to preparing upsert payload

Side effects of this change:
* None

Relevant ticket(s):
* https://mitlibraries.atlassian.net/browse/HRQB-32

Run Details

789 of 793 relevant lines covered (99.5%)

0.99 hits per line