MITLibraries / browsertrix-harvester

100%

main: 100%

LAST BUILD ON BRANCH align-btrix-v0.12.0-release

branch: align-btrix-v0.12.0-release

CHANGE BRANCH

x

Reset

• align-btrix-v0.12.0-release
• IN-1240-pip-audit
• IN-1524-maintenance
• TIMX-247-initial-build
• TIMX-542-output-jsonlines
• TIMX-557-and-misc
• TIMX-562-incremental-runs
• USE-86-align-with-etl-step-function
• USE-93-sitemap-arg-streamline-and-flexibility
• USE-93-sitemap-parsing-and-url-file
• USE-97-generate-delete-records
• dependabot/pip/black-23.11.0
• dependabot/pip/black-23.12.0
• dependabot/pip/black-23.12.1
• dependabot/pip/mypy-1.6.1
• dependabot/pip/mypy-1.7.0
• dependabot/pip/mypy-1.7.1
• dependabot/pip/mypy-1.8.0
• dependabot/pip/pandas-2.1.3
• dependabot/pip/pandas-2.1.4
• dependabot/pip/pre-commit-3.5.0
• dependabot/pip/ruff-0.1.0
• dependabot/pip/ruff-0.1.1
• dependabot/pip/ruff-0.1.11
• dependabot/pip/ruff-0.1.3
• dependabot/pip/ruff-0.1.4
• dependabot/pip/ruff-0.1.5
• dependabot/pip/ruff-0.1.6
• dependabot/pip/ruff-0.1.7
• dependabot/pip/ruff-0.1.8
• dependabot/pip/ruff-0.1.9
• dependabot/pip/safety-2.3.5
• dependabot/pip/urllib3-1.26.18
• in-1500-new-workflows
• main
• maintenance
• maintenance-09-2024
• pr4-ci-aws
• refs/tags/v1.0.0
• refs/tags/v1.1.0
• v1.1.1
• v1.2

Build # 6746009289

Build Type

push

github

Committed by ghukill

Commit Message

Align btrix CLI arguments for v0.12.0 release

Why these changes are being introduced:

Browsertrix had a [minor release v0.12.0](https://github.com/webrecorder/browsertrix-crawler/releases/tag/v0.12.0) that changed the
optional fulltext CLI argument --text, now requiring where the fulltext should be stored (where previously defaulted to pages JSON
files).

In time, we may want to leverage storing fulltext in WARC which is now possible from this change, but okay to continue
looking for fulltext in pages JSON files.

How this addresses that need:

Updates the example config YAML from 'text: true' to 'text: to-pages'

Side effects of this change:

None at this time.

Relevant ticket(s):

None

Run Details

416 of 416 relevant lines covered (100.0%)

1.0 hits per line