MITLibraries / oai-pmh-harvester

100%

main: 99%

LAST BUILD ON BRANCH IN-1038-gracefully-handle-network-timeouts

branch: IN-1038-gracefully-handle-network-timeouts

CHANGE BRANCH

x

Reset

Sync Branches

• IN-1038-gracefully-handle-network-timeouts
• IN-1259-pip-audit
• IN-1429-2025-08-maintenance
• RDI-115-retry-logic
• RDI-95
• TIMX-258-graceful-record-skip
• actions-makefiles-update
• bugfix
• bugfix-no-records-error
• ci-updates
• dependabot-update
• dependabot/pip/bandit-1.7.5
• dependabot/pip/black-22.10.0
• dependabot/pip/black-22.12.0
• dependabot/pip/black-22.6.0
• dependabot/pip/black-22.8.0
• dependabot/pip/black-23.1.0
• dependabot/pip/black-23.3.0
• dependabot/pip/certifi-2022.12.7
• dependabot/pip/certifi-2024.7.4
• dependabot/pip/click-8.1.3
• dependabot/pip/coverage-6.3.3
• dependabot/pip/coverage-6.4
• dependabot/pip/coverage-6.4.1
• dependabot/pip/coverage-6.4.2
• dependabot/pip/coverage-6.4.3
• dependabot/pip/coverage-6.4.4
• dependabot/pip/coverage-6.5.0
• dependabot/pip/coverage-7.0.0
• dependabot/pip/coverage-7.0.1
• dependabot/pip/coverage-7.0.2
• dependabot/pip/coverage-7.0.3
• dependabot/pip/coverage-7.0.4
• dependabot/pip/coverage-7.0.5
• dependabot/pip/coverage-7.1.0
• dependabot/pip/coverage-7.2.0
• dependabot/pip/coverage-7.2.1
• dependabot/pip/coverage-7.2.2
• dependabot/pip/coverage-7.2.3
• dependabot/pip/coverage-7.2.5
• dependabot/pip/coverage-7.2.6
• dependabot/pip/coverage-7.2.7
• dependabot/pip/flake8-5.0.2
• dependabot/pip/flake8-5.0.3
• dependabot/pip/flake8-5.0.4
• dependabot/pip/flake8-6.0.0
• dependabot/pip/isort-5.11.1
• dependabot/pip/isort-5.11.2
• dependabot/pip/isort-5.11.3
• dependabot/pip/isort-5.11.4
• dependabot/pip/isort-5.12.0
• dependabot/pip/lxml-4.9.1
• dependabot/pip/mypy-0.950
• dependabot/pip/mypy-0.960
• dependabot/pip/mypy-0.961
• dependabot/pip/mypy-0.971
• dependabot/pip/mypy-0.981
• dependabot/pip/mypy-0.982
• dependabot/pip/mypy-0.990
• dependabot/pip/mypy-0.991
• dependabot/pip/mypy-1.0.0
• dependabot/pip/mypy-1.0.1
• dependabot/pip/mypy-1.1.1
• dependabot/pip/mypy-1.2.0
• dependabot/pip/mypy-1.3.0
• dependabot/pip/mypy-1.4.1
• dependabot/pip/pytest-7.1.2
• dependabot/pip/pytest-7.1.3
• dependabot/pip/pytest-7.2.0
• dependabot/pip/pytest-7.2.1
• dependabot/pip/pytest-7.2.2
• dependabot/pip/pytest-7.3.1
• dependabot/pip/pytest-7.3.2
• dependabot/pip/pytest-7.4.0
• dependabot/pip/requests-2.31.0
• dependabot/pip/ruff-0.4.10
• dependabot/pip/ruff-0.4.9
• dependabot/pip/ruff-0.5.0
• dependabot/pip/ruff-0.5.1
• dependabot/pip/ruff-0.5.2
• dependabot/pip/ruff-0.5.4
• dependabot/pip/ruff-0.5.5
• dependabot/pip/ruff-0.5.6
• dependabot/pip/ruff-0.5.7
• dependabot/pip/ruff-0.6.1
• dependabot/pip/ruff-0.6.2
• dependabot/pip/ruff-0.6.3
• dependabot/pip/safety-3.2.3
• dependabot/pip/safety-3.2.4
• dependabot/pip/safety-3.2.5
• dependabot/pip/safety-3.2.6
• dependabot/pip/safety-3.2.7
• dependabot/pip/sentry-sdk-1.10.0
• dependabot/pip/sentry-sdk-1.10.1
• dependabot/pip/sentry-sdk-1.11.0
• dependabot/pip/sentry-sdk-1.11.1
• dependabot/pip/sentry-sdk-1.12.0
• dependabot/pip/sentry-sdk-1.12.1
• dependabot/pip/sentry-sdk-1.13.0
• dependabot/pip/sentry-sdk-1.14.0
• dependabot/pip/sentry-sdk-1.15.0
• dependabot/pip/sentry-sdk-1.16.0
• dependabot/pip/sentry-sdk-1.17.0
• dependabot/pip/sentry-sdk-1.18.0
• dependabot/pip/sentry-sdk-1.19.0
• dependabot/pip/sentry-sdk-1.19.1
• dependabot/pip/sentry-sdk-1.20.0
• dependabot/pip/sentry-sdk-1.21.1
• dependabot/pip/sentry-sdk-1.22.2
• dependabot/pip/sentry-sdk-1.23.0
• dependabot/pip/sentry-sdk-1.23.1
• dependabot/pip/sentry-sdk-1.24.0
• dependabot/pip/sentry-sdk-1.25.0
• dependabot/pip/sentry-sdk-1.25.1
• dependabot/pip/sentry-sdk-1.26.0
• dependabot/pip/sentry-sdk-1.28.0
• dependabot/pip/sentry-sdk-1.5.12
• dependabot/pip/sentry-sdk-1.6.0
• dependabot/pip/sentry-sdk-1.7.0
• dependabot/pip/sentry-sdk-1.7.1
• dependabot/pip/sentry-sdk-1.7.2
• dependabot/pip/sentry-sdk-1.8.0
• dependabot/pip/sentry-sdk-1.9.0
• dependabot/pip/sentry-sdk-1.9.10
• dependabot/pip/sentry-sdk-1.9.2
• dependabot/pip/sentry-sdk-1.9.3
• dependabot/pip/sentry-sdk-1.9.4
• dependabot/pip/sentry-sdk-1.9.5
• dependabot/pip/sentry-sdk-1.9.6
• dependabot/pip/sentry-sdk-1.9.7
• dependabot/pip/sentry-sdk-1.9.8
• dependabot/pip/sentry-sdk-1.9.9
• dependabot/pip/sentry-sdk-2.10.0
• dependabot/pip/sentry-sdk-2.11.0
• dependabot/pip/sentry-sdk-2.12.0
• dependabot/pip/sentry-sdk-2.13.0
• dependabot/pip/sentry-sdk-2.5.1
• dependabot/pip/sentry-sdk-2.6.0
• dependabot/pip/sentry-sdk-2.7.1
• dependabot/pip/sentry-sdk-2.8.0
• dependabot/pip/smart-open-6.0.0
• dependabot/pip/smart-open-6.1.0
• dependabot/pip/smart-open-6.2.0
• dependabot/pip/smart-open-6.3.0
• dependabot/pip/types-requests-2.32.0.20240622
• dependabot/pip/types-requests-2.32.0.20240712
• dependabot/pip/urllib3-2.2.2
• dependabot/pip/vcrpy-4.2.0
• dependabot/pip/vcrpy-4.2.1
• dependency-updates
• main
• maintenance-week
• maintenance-week-updates
• rdi-157-update-config
• rdi-176-add-list-method-option
• rdi-updates
• refs/heads/dependabot/pip/coverage-7.2.6
• refs/heads/dependabot/pip/sentry-sdk-1.24.0
• refs/tags/v1.0
• refs/tags/v1.0.1
• refs/tags/v1.0.2
• refs/tags/v1.0.3
• refs/tags/v1.0.4
• refs/tags/v1.0.5
• refs/tags/v1.0.6
• retry-values-update
• sentry-env-var-not-required
• shared-github-workflow
• stage-action-update
• timx-189-add-skip-list
• update-caller-workflows
• v1.0.7
• workflow-makefile-updates

Build # 10494632383

Build Type

Pull #579

github

Committed by ghukill

Commit Message

OAIClient.get_record catch RequestException

Why these changes are being introduced:

A harvest recently failed when the requests library threw a
ConnectTimeout exception.  We were formerly handling only HTTPErrors,
which this exception did not inherit from.  This exception aborted
the harvest instead of gracefully skipping and reporting record,
and continuing.

How this addresses that need:
* OAIClient.get_records() now catches more base RequestException which
includes HTTP and connection errors

Side effects of this change:
* Connection errors when retrieving a record will be logged and
skipped, and will not fail the full harvest.

Relevant ticket(s):
* https://mitlibraries.atlassian.net/browse/IN-1038

Pull Request Pull Request #579: IN 1038 - handle more base RequestException

Run Details

2 of 2 new or added lines in 1 file covered. (100.0%)

202 of 202 relevant lines covered (100.0%)

1.0 hits per line