MITLibraries / hrqb-client

99%

main: 96%

LAST BUILD ON BRANCH HRQB-40-remove-volatile-key-fields

branch: HRQB-40-remove-volatile-key-fields

CHANGE BRANCH

x

Reset

• HRQB-40-remove-volatile-key-fields
• Add-actions
• HRQB-10-scaffold-task-and-target-classes
• HRQB-11-dw-connection-and-helpers
• HRQB-12-setup-qb-api-and-load-tasks
• HRQB-13-pipelines-and-cli
• HRQB-13-refactor-base-tasks
• HRQB-15-libhr-table
• HRQB-17-functional-tasks-and-pipelines-prep
• HRQB-17-pipeline-task-isolation
• HRQB-18-connection-tests
• HRQB-20-contd-termination-work
• HRQB-20-termination-reason-last-appt-only
• HRQB-20-termination-reasons
• HRQB-20-termination-types
• HRQB-21-employees-table
• HRQB-22-adjustments-for-retirements
• HRQB-22-employee-appointments
• HRQB-23-employee-salary-history
• HRQB-27-runtime-assertions
• HRQB-28-employee-leave
• HRQB-29-log-load-counts
• HRQB-32-memory-footprint
• HRQB-34-base-salary-change-percent
• HRQB-35-generate-performance-reviews
• HRQB-36-md5-hash-merge-fields
• HRQB-39-remove-sensitive-logging-sentry
• HRQB-40-fix-merge-without-appointment-key
• HRQB-41-improve-dev-target-removal
• HRQB-43-production-deploy
• HRQB-44-resolve-lingering-upsert-issues
• HRQB-46-sentry-message-on-upsert-errors
• HRQB-49-historical-libhr-data
• HRQB-51-employee-leave-balances
• HRQB-52-salary-history-updates
• HRQB-54-add-employee-ethnicity
• HRQB-57-many-to-one-libhr-rows
• HRQB-7-app-scaffold
• IN-1194-merge-libhr-tables
• IN-1200-remove-2019-data-cutoff
• IN-1208-add-employee-fields
• IN-1240-pip-audit-and-maintenance
• IN-1329-2025-06-23-maintenance
• IN-1439-stale-salary-history-rows
• IN-1450-adjust-salary-history-query-and-key-logic
• bug-remove-appt-end-date-from-key
• dependabot/pip/black-24.4.2
• dependabot/pip/boto3-1.34.116
• dependabot/pip/boto3-1.34.117
• dependabot/pip/boto3-1.34.118
• dependabot/pip/boto3-1.34.119
• dependabot/pip/boto3-1.34.120
• dependabot/pip/boto3-1.34.121
• dependabot/pip/boto3-1.34.122
• dependabot/pip/boto3-1.34.123
• dependabot/pip/boto3-1.34.124
• dependabot/pip/boto3-1.34.125
• dependabot/pip/boto3-1.34.126
• dependabot/pip/boto3-1.34.127
• dependabot/pip/boto3-1.34.128
• dependabot/pip/boto3-1.34.129
• dependabot/pip/boto3-1.34.130
• dependabot/pip/boto3-1.34.131
• dependabot/pip/boto3-1.34.132
• dependabot/pip/boto3-1.34.133
• dependabot/pip/boto3-1.34.134
• dependabot/pip/boto3-1.34.135
• dependabot/pip/boto3-1.34.136
• dependabot/pip/boto3-1.34.137
• dependabot/pip/boto3-1.34.138
• dependabot/pip/boto3-1.34.139
• dependabot/pip/boto3-1.34.140
• dependabot/pip/boto3-1.34.141
• dependabot/pip/boto3-1.34.142
• dependabot/pip/boto3-1.34.143
• dependabot/pip/boto3-1.34.144
• dependabot/pip/boto3-1.34.145
• dependabot/pip/boto3-1.34.146
• dependabot/pip/boto3-1.34.147
• dependabot/pip/boto3-1.34.148
• dependabot/pip/boto3-1.34.149
• dependabot/pip/boto3-1.34.150
• dependabot/pip/boto3-1.34.151
• dependabot/pip/boto3-1.34.152
• dependabot/pip/boto3-1.34.153
• dependabot/pip/boto3-1.34.154
• dependabot/pip/boto3-1.34.155
• dependabot/pip/boto3-1.34.156
• dependabot/pip/boto3-1.34.157
• dependabot/pip/boto3-1.34.158
• dependabot/pip/boto3-1.34.159
• dependabot/pip/boto3-1.34.160
• dependabot/pip/boto3-1.34.161
• dependabot/pip/boto3-1.34.162
• dependabot/pip/boto3-1.35.0
• dependabot/pip/boto3-1.35.1
• dependabot/pip/boto3-1.35.10
• dependabot/pip/boto3-1.35.11
• dependabot/pip/boto3-1.35.12
• dependabot/pip/boto3-1.35.13
• dependabot/pip/boto3-1.35.14
• dependabot/pip/boto3-1.35.15
• dependabot/pip/boto3-1.35.16
• dependabot/pip/boto3-1.35.17
• dependabot/pip/boto3-1.35.18
• dependabot/pip/boto3-1.35.19
• dependabot/pip/boto3-1.35.2
• dependabot/pip/boto3-1.35.20
• dependabot/pip/boto3-1.35.21
• dependabot/pip/boto3-1.35.22
• dependabot/pip/boto3-1.35.23
• dependabot/pip/boto3-1.35.24
• dependabot/pip/boto3-1.35.25
• dependabot/pip/boto3-1.35.26
• dependabot/pip/boto3-1.35.27
• dependabot/pip/boto3-1.35.28
• dependabot/pip/boto3-1.35.29
• dependabot/pip/boto3-1.35.3
• dependabot/pip/boto3-1.35.30
• dependabot/pip/boto3-1.35.31
• dependabot/pip/boto3-1.35.32
• dependabot/pip/boto3-1.35.33
• dependabot/pip/boto3-1.35.34
• dependabot/pip/boto3-1.35.35
• dependabot/pip/boto3-1.35.4
• dependabot/pip/boto3-1.35.5
• dependabot/pip/boto3-1.35.6
• dependabot/pip/boto3-1.35.7
• dependabot/pip/boto3-1.35.8
• dependabot/pip/boto3-1.35.9
• dependabot/pip/certifi-2024.7.4
• dependabot/pip/coveralls-4.0.0
• dependabot/pip/coveralls-4.0.1
• dependabot/pip/cryptography-43.0.1
• dependabot/pip/ipython-8.25.0
• dependabot/pip/ipython-8.26.0
• dependabot/pip/mypy-1.10.0
• dependabot/pip/mypy-1.10.1
• dependabot/pip/mypy-1.11.0
• dependabot/pip/mypy-1.11.1
• dependabot/pip/mypy-1.11.2
• dependabot/pip/oracledb-2.2.0
• dependabot/pip/oracledb-2.2.1
• dependabot/pip/pandas-stubs-2.2.2.240514
• dependabot/pip/pandas-stubs-2.2.2.240603
• dependabot/pip/pre-commit-3.7.1
• dependabot/pip/pytest-8.2.2
• dependabot/pip/requests-2.32.0
• dependabot/pip/requests-2.32.2
• dependabot/pip/requests-2.32.3
• dependabot/pip/ruff-0.4.10
• dependabot/pip/ruff-0.4.2
• dependabot/pip/ruff-0.4.3
• dependabot/pip/ruff-0.4.4
• dependabot/pip/ruff-0.4.5
• dependabot/pip/ruff-0.4.6
• dependabot/pip/ruff-0.4.7
• dependabot/pip/ruff-0.4.8
• dependabot/pip/ruff-0.4.9
• dependabot/pip/ruff-0.5.0
• dependabot/pip/ruff-0.5.1
• dependabot/pip/ruff-0.5.2
• dependabot/pip/ruff-0.5.3
• dependabot/pip/ruff-0.5.4
• dependabot/pip/ruff-0.5.5
• dependabot/pip/ruff-0.5.6
• dependabot/pip/ruff-0.5.7
• dependabot/pip/ruff-0.6.0
• dependabot/pip/ruff-0.6.1
• dependabot/pip/ruff-0.6.2
• dependabot/pip/ruff-0.6.3
• dependabot/pip/ruff-0.6.4
• dependabot/pip/ruff-0.6.5
• dependabot/pip/ruff-0.6.6
• dependabot/pip/ruff-0.6.7
• dependabot/pip/ruff-0.6.8
• dependabot/pip/ruff-0.6.9
• dependabot/pip/sentry-sdk-2.0.1
• dependabot/pip/sentry-sdk-2.1.1
• dependabot/pip/sentry-sdk-2.10.0
• dependabot/pip/sentry-sdk-2.11.0
• dependabot/pip/sentry-sdk-2.12.0
• dependabot/pip/sentry-sdk-2.13.0
• dependabot/pip/sentry-sdk-2.14.0
• dependabot/pip/sentry-sdk-2.15.0
• dependabot/pip/sentry-sdk-2.2.0
• dependabot/pip/sentry-sdk-2.2.1
• dependabot/pip/sentry-sdk-2.3.1
• dependabot/pip/sentry-sdk-2.4.0
• dependabot/pip/sentry-sdk-2.5.0
• dependabot/pip/sentry-sdk-2.5.1
• dependabot/pip/sentry-sdk-2.6.0
• dependabot/pip/sentry-sdk-2.7.0
• dependabot/pip/sentry-sdk-2.7.1
• dependabot/pip/sentry-sdk-2.8.0
• dependabot/pip/sentry-sdk-2.9.0
• dependabot/pip/sqlalchemy-2.0.31
• dependabot/pip/tornado-6.4.1
• dependabot/pip/types-requests-2.32.0.20240602
• dependabot/pip/types-requests-2.32.0.20240622
• dependabot/pip/types-requests-2.32.0.20240712
• dependabot/pip/types-requests-2.32.0.20240905
• dependabot/pip/types-requests-2.32.0.20240907
• dependabot/pip/types-requests-2.32.0.20240914
• dependabot/pip/urllib3-2.2.2
• main
• refs/tags/v1.0.0
• refs/tags/v1.0.1
• refs/tags/v1.0.2
• stage-workflow
• update-emp-sal-hist-merge-field
• v.1.4.2
• v.1.4.3
• v1.1.0
• v1.10.0
• v1.11.0
• v1.11.1
• v1.12.0
• v1.13.0
• v1.2.0
• v1.3.0
• v1.4.0
• v1.4.1
• v1.5.0
• v1.6.0
• v1.7.0
• v1.8.0
• v1.9.0

Build # 10042906642

Build Type

push

github

Committed by ghukill

Commit Message

Fix bugs in volatile upserts

Why these changes are being introduced:

Two issues were noted during testing:

1. Some tables had updates for rows each daily run, even
though meaningful data had not changed. This was due to data
internal warehouse values getting included in Quickbase table that
changed daily, but were not meaningful changes.

2. Performance Reviews had some clobbering due to the merge
'Key' field not being unique.

How this addresses that need:

1. Foreign keys from data warehouse were removed from Quickbase
tables.  These provided no value, and were false flags that
data was changing.

2. Performance Reviews got the related Employee Appointment ID
as part of its merge 'Key' field hashing.  This makes them globally
unique for an employee.

Side effects of this change:
* All records were wiped from Performance Reviews and reloaded.

Relevant ticket(s):
* https://mitlibraries.atlassian.net/browse/HRQB-40

Run Details

2 of 2 new or added lines in 1 file covered. (100.0%)

922 of 927 relevant lines covered (99.46%)

0.99 hits per line