• Home
  • Features
  • Pricing
  • Docs
  • Announcements
  • Sign In

MITLibraries / s3-bagit-validator
88%
main: 90%

Build:
Build:
LAST BUILD BRANCH: v0.1.6
DEFAULT BRANCH: main
Repo Added 18 Mar 2025 01:02PM UTC
Token Tg8qBmvISmVGy9L9VvQ0SIU82pYCn4NdA regen
Build 495 Last
Files 8
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH IR-206-subtasks
branch: IR-206-subtasks
CHANGE BRANCH
x
Reset
Sync Branches
  • IR-206-subtasks
  • IN-1203-scaffold-repository
  • IN-1454-batch-operations
  • INFRA-597-input-metadata-fix
  • IR-182-athena-util
  • IR-182-port-aip-validation
  • IR-182-port-poc-code
  • IR-182-s3-util
  • IR-183-lambda-handler
  • IR-191-cli-sidecar
  • IR-192-add-sentry
  • IR-193-parallelize-checksum-generation
  • IR-207-duckdb-instead-of-athena
  • IR-209-missing-or-empty-inventory
  • IR-210-large-file-checksumming
  • IR-219-app-limitations
  • IR-220
  • IR-220-manifests-and-error-details
  • IR-222-223
  • IR-228-resume-bulk-validation
  • IR-228-small-updates-and-aws-sam-cli
  • IR-233-validate-all
  • IR-238-multi-region
  • IR-240-aip-inventory
  • IR-240-inventory-csv
  • IR-247-record-aip-id-for-all-errors
  • IR-258-cloudshell
  • bug-unit-test-invokes-lambda
  • cli-errors-in-response
  • dependabot/pip/black-25.1.0
  • dependabot/pip/boto3-1.37.29
  • dependabot/pip/boto3-1.37.30
  • dependabot/pip/boto3-1.37.31
  • dependabot/pip/boto3-1.37.32
  • dependabot/pip/boto3-1.37.33
  • dependabot/pip/boto3-1.37.34
  • dependabot/pip/boto3-1.37.35
  • dependabot/pip/boto3-1.37.36
  • dependabot/pip/boto3-1.37.37
  • dependabot/pip/boto3-1.37.38
  • dependabot/pip/boto3-1.38.0
  • dependabot/pip/boto3-1.38.1
  • dependabot/pip/boto3-1.38.10
  • dependabot/pip/boto3-1.38.11
  • dependabot/pip/boto3-1.38.12
  • dependabot/pip/boto3-1.38.13
  • dependabot/pip/boto3-1.38.14
  • dependabot/pip/boto3-1.38.15
  • dependabot/pip/boto3-1.38.16
  • dependabot/pip/boto3-1.38.17
  • dependabot/pip/boto3-1.38.18
  • dependabot/pip/boto3-1.38.19
  • dependabot/pip/boto3-1.38.2
  • dependabot/pip/boto3-1.38.20
  • dependabot/pip/boto3-1.38.21
  • dependabot/pip/boto3-1.38.22
  • dependabot/pip/boto3-1.38.23
  • dependabot/pip/boto3-1.38.24
  • dependabot/pip/boto3-1.38.25
  • dependabot/pip/boto3-1.38.26
  • dependabot/pip/boto3-1.38.27
  • dependabot/pip/boto3-1.38.28
  • dependabot/pip/boto3-1.38.29
  • dependabot/pip/boto3-1.38.3
  • dependabot/pip/boto3-1.38.30
  • dependabot/pip/boto3-1.38.32
  • dependabot/pip/boto3-1.38.33
  • dependabot/pip/boto3-1.38.34
  • dependabot/pip/boto3-1.38.35
  • dependabot/pip/boto3-1.38.36
  • dependabot/pip/boto3-1.38.37
  • dependabot/pip/boto3-1.38.38
  • dependabot/pip/boto3-1.38.39
  • dependabot/pip/boto3-1.38.4
  • dependabot/pip/boto3-1.38.40
  • dependabot/pip/boto3-1.38.41
  • dependabot/pip/boto3-1.38.42
  • dependabot/pip/boto3-1.38.43
  • dependabot/pip/boto3-1.38.44
  • dependabot/pip/boto3-1.38.45
  • dependabot/pip/boto3-1.38.46
  • dependabot/pip/boto3-1.38.5
  • dependabot/pip/boto3-1.38.6
  • dependabot/pip/boto3-1.38.7
  • dependabot/pip/boto3-1.38.8
  • dependabot/pip/boto3-1.38.9
  • dependabot/pip/boto3-1.39.0
  • dependabot/pip/boto3-1.39.1
  • dependabot/pip/boto3-1.39.10
  • dependabot/pip/boto3-1.39.11
  • dependabot/pip/boto3-1.39.12
  • dependabot/pip/boto3-1.39.13
  • dependabot/pip/boto3-1.39.15
  • dependabot/pip/boto3-1.39.16
  • dependabot/pip/boto3-1.39.17
  • dependabot/pip/boto3-1.39.2
  • dependabot/pip/boto3-1.39.3
  • dependabot/pip/boto3-1.39.4
  • dependabot/pip/boto3-1.39.7
  • dependabot/pip/boto3-1.39.8
  • dependabot/pip/boto3-1.39.9
  • dependabot/pip/boto3-1.40.0
  • dependabot/pip/boto3-1.40.10
  • dependabot/pip/boto3-1.40.12
  • dependabot/pip/boto3-1.40.13
  • dependabot/pip/boto3-1.40.15
  • dependabot/pip/boto3-1.40.16
  • dependabot/pip/boto3-1.40.17
  • dependabot/pip/boto3-1.40.18
  • dependabot/pip/boto3-1.40.19
  • dependabot/pip/boto3-1.40.2
  • dependabot/pip/boto3-1.40.20
  • dependabot/pip/boto3-1.40.21
  • dependabot/pip/boto3-1.40.22
  • dependabot/pip/boto3-1.40.23
  • dependabot/pip/boto3-1.40.24
  • dependabot/pip/boto3-1.40.25
  • dependabot/pip/boto3-1.40.26
  • dependabot/pip/boto3-1.40.27
  • dependabot/pip/boto3-1.40.28
  • dependabot/pip/boto3-1.40.29
  • dependabot/pip/boto3-1.40.3
  • dependabot/pip/boto3-1.40.30
  • dependabot/pip/boto3-1.40.31
  • dependabot/pip/boto3-1.40.32
  • dependabot/pip/boto3-1.40.33
  • dependabot/pip/boto3-1.40.34
  • dependabot/pip/boto3-1.40.36
  • dependabot/pip/boto3-1.40.38
  • dependabot/pip/boto3-1.40.39
  • dependabot/pip/boto3-1.40.4
  • dependabot/pip/boto3-1.40.40
  • dependabot/pip/boto3-1.40.41
  • dependabot/pip/boto3-1.40.42
  • dependabot/pip/boto3-1.40.43
  • dependabot/pip/boto3-1.40.44
  • dependabot/pip/boto3-1.40.45
  • dependabot/pip/boto3-1.40.46
  • dependabot/pip/boto3-1.40.47
  • dependabot/pip/boto3-1.40.48
  • dependabot/pip/boto3-1.40.49
  • dependabot/pip/boto3-1.40.5
  • dependabot/pip/boto3-1.40.50
  • dependabot/pip/boto3-1.40.51
  • dependabot/pip/boto3-1.40.52
  • dependabot/pip/boto3-1.40.53
  • dependabot/pip/boto3-1.40.54
  • dependabot/pip/boto3-1.40.55
  • dependabot/pip/boto3-1.40.56
  • dependabot/pip/boto3-1.40.57
  • dependabot/pip/boto3-1.40.58
  • dependabot/pip/boto3-1.40.59
  • dependabot/pip/boto3-1.40.6
  • dependabot/pip/boto3-1.40.60
  • dependabot/pip/boto3-1.40.61
  • dependabot/pip/boto3-1.40.62
  • dependabot/pip/boto3-1.40.8
  • dependabot/pip/boto3-1.40.9
  • dependabot/pip/boto3-stubs-1.37.20
  • dependabot/pip/boto3-stubs-1.37.21
  • dependabot/pip/boto3-stubs-1.37.22
  • dependabot/pip/boto3-stubs-1.37.23
  • dependabot/pip/boto3-stubs-1.37.24
  • dependabot/pip/boto3-stubs-1.37.25
  • dependabot/pip/boto3-stubs-1.37.26
  • dependabot/pip/boto3-stubs-1.37.27
  • dependabot/pip/boto3-stubs-1.37.28
  • dependabot/pip/boto3-stubs-1.37.29
  • dependabot/pip/boto3-stubs-1.37.30
  • dependabot/pip/boto3-stubs-1.37.31
  • dependabot/pip/boto3-stubs-1.37.32
  • dependabot/pip/boto3-stubs-1.37.33
  • dependabot/pip/boto3-stubs-1.37.34
  • dependabot/pip/boto3-stubs-1.37.35
  • dependabot/pip/boto3-stubs-1.37.36
  • dependabot/pip/boto3-stubs-1.37.37
  • dependabot/pip/boto3-stubs-1.37.38
  • dependabot/pip/boto3-stubs-1.38.0
  • dependabot/pip/boto3-stubs-1.38.1
  • dependabot/pip/boto3-stubs-1.38.10
  • dependabot/pip/boto3-stubs-1.38.11
  • dependabot/pip/boto3-stubs-1.38.12
  • dependabot/pip/boto3-stubs-1.38.13
  • dependabot/pip/boto3-stubs-1.38.14
  • dependabot/pip/boto3-stubs-1.38.15
  • dependabot/pip/boto3-stubs-1.38.16
  • dependabot/pip/boto3-stubs-1.38.17
  • dependabot/pip/boto3-stubs-1.38.18
  • dependabot/pip/boto3-stubs-1.38.19
  • dependabot/pip/boto3-stubs-1.38.2
  • dependabot/pip/boto3-stubs-1.38.20
  • dependabot/pip/boto3-stubs-1.38.21
  • dependabot/pip/boto3-stubs-1.38.22
  • dependabot/pip/boto3-stubs-1.38.23
  • dependabot/pip/boto3-stubs-1.38.24
  • dependabot/pip/boto3-stubs-1.38.25
  • dependabot/pip/boto3-stubs-1.38.26
  • dependabot/pip/boto3-stubs-1.38.27
  • dependabot/pip/boto3-stubs-1.38.29
  • dependabot/pip/boto3-stubs-1.38.3
  • dependabot/pip/boto3-stubs-1.38.30
  • dependabot/pip/boto3-stubs-1.38.31
  • dependabot/pip/boto3-stubs-1.38.32
  • dependabot/pip/boto3-stubs-1.38.33
  • dependabot/pip/boto3-stubs-1.38.34
  • dependabot/pip/boto3-stubs-1.38.35
  • dependabot/pip/boto3-stubs-1.38.36
  • dependabot/pip/boto3-stubs-1.38.38
  • dependabot/pip/boto3-stubs-1.38.39
  • dependabot/pip/boto3-stubs-1.38.4
  • dependabot/pip/boto3-stubs-1.38.40
  • dependabot/pip/boto3-stubs-1.38.41
  • dependabot/pip/boto3-stubs-1.38.42
  • dependabot/pip/boto3-stubs-1.38.43
  • dependabot/pip/boto3-stubs-1.38.44
  • dependabot/pip/boto3-stubs-1.38.45
  • dependabot/pip/boto3-stubs-1.38.46
  • dependabot/pip/boto3-stubs-1.38.5
  • dependabot/pip/boto3-stubs-1.38.6
  • dependabot/pip/boto3-stubs-1.38.7
  • dependabot/pip/boto3-stubs-1.38.8
  • dependabot/pip/boto3-stubs-1.38.9
  • dependabot/pip/boto3-stubs-1.39.0
  • dependabot/pip/boto3-stubs-1.39.1
  • dependabot/pip/boto3-stubs-1.39.10
  • dependabot/pip/boto3-stubs-1.39.11
  • dependabot/pip/boto3-stubs-1.39.12
  • dependabot/pip/boto3-stubs-1.39.13
  • dependabot/pip/boto3-stubs-1.39.15
  • dependabot/pip/boto3-stubs-1.39.16
  • dependabot/pip/boto3-stubs-1.39.17
  • dependabot/pip/boto3-stubs-1.39.2
  • dependabot/pip/boto3-stubs-1.39.3
  • dependabot/pip/boto3-stubs-1.39.4
  • dependabot/pip/boto3-stubs-1.39.7
  • dependabot/pip/boto3-stubs-1.39.8
  • dependabot/pip/boto3-stubs-1.39.9
  • dependabot/pip/boto3-stubs-1.40.0
  • dependabot/pip/boto3-stubs-1.40.10
  • dependabot/pip/boto3-stubs-1.40.12
  • dependabot/pip/boto3-stubs-1.40.13
  • dependabot/pip/boto3-stubs-1.40.14
  • dependabot/pip/boto3-stubs-1.40.15
  • dependabot/pip/boto3-stubs-1.40.16
  • dependabot/pip/boto3-stubs-1.40.17
  • dependabot/pip/boto3-stubs-1.40.18
  • dependabot/pip/boto3-stubs-1.40.19
  • dependabot/pip/boto3-stubs-1.40.2
  • dependabot/pip/boto3-stubs-1.40.21
  • dependabot/pip/boto3-stubs-1.40.22
  • dependabot/pip/boto3-stubs-1.40.23
  • dependabot/pip/boto3-stubs-1.40.24
  • dependabot/pip/boto3-stubs-1.40.25
  • dependabot/pip/boto3-stubs-1.40.26
  • dependabot/pip/boto3-stubs-1.40.27
  • dependabot/pip/boto3-stubs-1.40.28
  • dependabot/pip/boto3-stubs-1.40.29
  • dependabot/pip/boto3-stubs-1.40.3
  • dependabot/pip/boto3-stubs-1.40.30
  • dependabot/pip/boto3-stubs-1.40.31
  • dependabot/pip/boto3-stubs-1.40.32
  • dependabot/pip/boto3-stubs-1.40.33
  • dependabot/pip/boto3-stubs-1.40.34
  • dependabot/pip/boto3-stubs-1.40.35
  • dependabot/pip/boto3-stubs-1.40.36
  • dependabot/pip/boto3-stubs-1.40.38
  • dependabot/pip/boto3-stubs-1.40.39
  • dependabot/pip/boto3-stubs-1.40.4
  • dependabot/pip/boto3-stubs-1.40.40
  • dependabot/pip/boto3-stubs-1.40.41
  • dependabot/pip/boto3-stubs-1.40.42
  • dependabot/pip/boto3-stubs-1.40.43
  • dependabot/pip/boto3-stubs-1.40.44
  • dependabot/pip/boto3-stubs-1.40.45
  • dependabot/pip/boto3-stubs-1.40.46
  • dependabot/pip/boto3-stubs-1.40.47
  • dependabot/pip/boto3-stubs-1.40.48
  • dependabot/pip/boto3-stubs-1.40.49
  • dependabot/pip/boto3-stubs-1.40.5
  • dependabot/pip/boto3-stubs-1.40.50
  • dependabot/pip/boto3-stubs-1.40.51
  • dependabot/pip/boto3-stubs-1.40.52
  • dependabot/pip/boto3-stubs-1.40.53
  • dependabot/pip/boto3-stubs-1.40.54
  • dependabot/pip/boto3-stubs-1.40.55
  • dependabot/pip/boto3-stubs-1.40.56
  • dependabot/pip/boto3-stubs-1.40.57
  • dependabot/pip/boto3-stubs-1.40.58
  • dependabot/pip/boto3-stubs-1.40.59
  • dependabot/pip/boto3-stubs-1.40.6
  • dependabot/pip/boto3-stubs-1.40.60
  • dependabot/pip/boto3-stubs-1.40.61
  • dependabot/pip/boto3-stubs-1.40.62
  • dependabot/pip/boto3-stubs-1.40.7
  • dependabot/pip/boto3-stubs-1.40.8
  • dependabot/pip/boto3-stubs-1.40.9
  • dependabot/pip/click-8.2.0
  • dependabot/pip/click-8.2.1
  • dependabot/pip/duckdb-1.2.2
  • dependabot/pip/duckdb-1.3.1
  • dependabot/pip/ipython-9.1.0
  • dependabot/pip/ipython-9.4.0
  • dependabot/pip/ipython-9.5.0
  • dependabot/pip/ipython-9.6.0
  • dependabot/pip/mypy-1.15.0
  • dependabot/pip/mypy-1.16.0
  • dependabot/pip/pandas-stubs-2.2.3.250527
  • dependabot/pip/pip-25.2
  • dependabot/pip/pre-commit-4.1.0
  • dependabot/pip/pre-commit-4.2.0
  • dependabot/pip/pytest-8.3.5
  • dependabot/pip/requests-2.32.4
  • dependabot/pip/ruff-0.11.0
  • dependabot/pip/ruff-0.11.1
  • dependabot/pip/ruff-0.11.10
  • dependabot/pip/ruff-0.11.12
  • dependabot/pip/ruff-0.11.13
  • dependabot/pip/ruff-0.11.2
  • dependabot/pip/ruff-0.11.3
  • dependabot/pip/ruff-0.11.4
  • dependabot/pip/ruff-0.11.5
  • dependabot/pip/ruff-0.11.6
  • dependabot/pip/ruff-0.11.7
  • dependabot/pip/ruff-0.11.8
  • dependabot/pip/ruff-0.11.9
  • dependabot/pip/ruff-0.12.1
  • dependabot/pip/ruff-0.12.10
  • dependabot/pip/ruff-0.12.11
  • dependabot/pip/ruff-0.12.12
  • dependabot/pip/ruff-0.12.2
  • dependabot/pip/ruff-0.12.3
  • dependabot/pip/ruff-0.12.4
  • dependabot/pip/ruff-0.12.5
  • dependabot/pip/ruff-0.12.7
  • dependabot/pip/ruff-0.12.8
  • dependabot/pip/ruff-0.12.9
  • dependabot/pip/ruff-0.13.0
  • dependabot/pip/ruff-0.13.1
  • dependabot/pip/ruff-0.13.2
  • dependabot/pip/ruff-0.13.3
  • dependabot/pip/ruff-0.14.0
  • dependabot/pip/ruff-0.14.1
  • dependabot/pip/ruff-0.14.2
  • dependabot/pip/sentry-sdk-2.24.0
  • dependabot/pip/sentry-sdk-2.24.1
  • dependabot/pip/sentry-sdk-2.25.0
  • dependabot/pip/sentry-sdk-2.25.1
  • dependabot/pip/sentry-sdk-2.26.0
  • dependabot/pip/sentry-sdk-2.26.1
  • dependabot/pip/sentry-sdk-2.27.0
  • dependabot/pip/sentry-sdk-2.28.0
  • dependabot/pip/sentry-sdk-2.29.1
  • dependabot/pip/sentry-sdk-2.31.0
  • dependabot/pip/sentry-sdk-2.32.0
  • dependabot/pip/sentry-sdk-2.33.0
  • dependabot/pip/sentry-sdk-2.33.1
  • dependabot/pip/sentry-sdk-2.33.2
  • dependabot/pip/sentry-sdk-2.34.0
  • dependabot/pip/sentry-sdk-2.34.1
  • dependabot/pip/sentry-sdk-2.35.0
  • dependabot/pip/sentry-sdk-2.35.1
  • dependabot/pip/sentry-sdk-2.35.2
  • dependabot/pip/sentry-sdk-2.36.0
  • dependabot/pip/sentry-sdk-2.37.0
  • dependabot/pip/sentry-sdk-2.37.1
  • dependabot/pip/sentry-sdk-2.38.0
  • dependabot/pip/sentry-sdk-2.39.0
  • dependabot/pip/sentry-sdk-2.40.0
  • dependabot/pip/sentry-sdk-2.41.0
  • dependabot/pip/sentry-sdk-2.42.0
  • dependabot/pip/sentry-sdk-2.42.1
  • dependabot/pip/sentry-sdk-2.43.0
  • dependabot/pip/setuptools-78.1.0
  • dependabot/pip/setuptools-80.1.0
  • dependabot/pip/setuptools-80.3.1
  • dependabot/pip/setuptools-80.8.0
  • dependabot/pip/setuptools-80.9.0
  • dependabot/pip/types-requests-2.32.4.20250611
  • dependabot/pip/urllib3-2.5.0
  • ecr-updates
  • feature-branch-readme-update
  • lambda-response-exception
  • main
  • readme-typo
  • rework-and-init-sentry
  • stage-test
  • v0.1.0
  • v0.1.1
  • v0.1.2
  • v0.1.3
  • v0.1.4
  • v0.1.6

02 May 2025 01:35PM UTC coverage: 88.092% (+0.02%) from 88.075%
14798691470

Pull #75

github

ghukill
Improve AIP Inventory SQL query

Why these changes are being introduced:

As noted Jira ticket IR-232, one "AIP" in S3 was problematic.
It was an "AIP" in the sense it had a valid UUID in an S3 key,
but this UUID showed up only for a single object which was a
.7z file.  This actually pointed at a larger issue, how we
determine "AIPs" that we have found in Inventory data.

What we lacked before was some kind of check that "AIPs" we
found from Inventory data aligned with a valid Bagit structure.

How this addresses that need:

The SQL query that looks for S3 objects with a valid UUID
in the key, after grouping by found UUID + prefix, now
also ensures that 'bagit.txt' was included for at least
one of the AIP files.  Without this, we can assume
whatever was found in S3 was not a valid Bagit AIP
structure.

Side effects of this change:
* The amount of AIPs "found" appears to be less, but is
arguably just more accurate.

Relevant ticket(s):
* https://mitlibraries.atlassian.net/browse/IR-232
Pull Request #75: IR 206 subtasks - IR 231 and IR 232

2 of 2 new or added lines in 1 file covered. (100.0%)

614 of 697 relevant lines covered (88.09%)

0.88 hits per line

Relevant lines Covered
Build:
Build:
697 RELEVANT LINES 614 COVERED LINES
0.88 HITS PER LINE
Source Files on IR-206-subtasks
  • Tree
  • List 8
  • Changed 2
  • Source Changed 2
  • Coverage Changed 1
Coverage ∆ File Lines Relevant Covered Missed Hits/Line

Recent builds

Builds Branch Commit Type Ran Committer Via Coverage
14798691470 IR-206-subtasks Improve AIP Inventory SQL query Why these changes are being introduced: As noted Jira ticket IR-232, one "AIP" in S3 was problematic. It was an "AIP" in the sense it had a valid UUID in an S3 key, but this UUID showed up only for a single object... Pull #75 02 May 2025 03:56PM UTC ghukill github
88.09
14651402171 IR-206-subtasks Improve AIP Inventory SQL query Why these changes are being introduced: As noted Jira ticket IR-232, one "AIP" in S3 was problematic. It was an "AIP" in the sense it had a valid UUID in an S3 key, but this UUID showed up only for a single object... push 24 Apr 2025 08:50PM UTC ghukill github
88.92
See All Builds (494)

Badge your Repo: s3-bagit-validator

We detected this repo isn’t badged! Grab the embed code to the right, add it to your repo to show off your code coverage, and when the badge is live hit the refresh button to remove this message.

Could not find badge in README.

Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

Refresh
  • Settings
  • Repo on GitHub
STATUS · Troubleshooting · Open an Issue · Sales · Support · CAREERS · ENTERPRISE · START FREE · SCHEDULE DEMO
ANNOUNCEMENTS · TWITTER · TOS & SLA · Supported CI Services · What's a CI service? · Automated Testing

© 2026 Coveralls, Inc