MITLibraries / browsertrix-harvester

100%

main: 100%

LAST BUILD ON BRANCH TIMX-542-output-jsonlines

branch: TIMX-542-output-jsonlines

CHANGE BRANCH

x

Reset

• TIMX-542-output-jsonlines
• IN-1240-pip-audit
• IN-1524-maintenance
• TIMX-247-initial-build
• TIMX-557-and-misc
• TIMX-562-incremental-runs
• USE-240-staff-directory-in-mitlibwebsite
• USE-258-html-vs-metadata-parsing
• USE-272-response-headers-in-records
• USE-86-align-with-etl-step-function
• USE-91-handle-empty-crawls
• USE-93-sitemap-arg-streamline-and-flexibility
• USE-93-sitemap-parsing-and-url-file
• USE-97-generate-delete-records
• align-btrix-v0.12.0-release
• dependabot/pip/black-23.11.0
• dependabot/pip/black-23.12.0
• dependabot/pip/black-23.12.1
• dependabot/pip/mypy-1.6.1
• dependabot/pip/mypy-1.7.0
• dependabot/pip/mypy-1.7.1
• dependabot/pip/mypy-1.8.0
• dependabot/pip/pandas-2.1.3
• dependabot/pip/pandas-2.1.4
• dependabot/pip/pre-commit-3.5.0
• dependabot/pip/ruff-0.1.0
• dependabot/pip/ruff-0.1.1
• dependabot/pip/ruff-0.1.11
• dependabot/pip/ruff-0.1.3
• dependabot/pip/ruff-0.1.4
• dependabot/pip/ruff-0.1.5
• dependabot/pip/ruff-0.1.6
• dependabot/pip/ruff-0.1.7
• dependabot/pip/ruff-0.1.8
• dependabot/pip/ruff-0.1.9
• dependabot/pip/safety-2.3.5
• dependabot/pip/urllib3-1.26.18
• in-1500-new-workflows
• main
• maintenance
• maintenance-09-2024
• pr4-ci-aws
• refs/tags/v1.0.0
• refs/tags/v1.1.0
• v1.1.1
• v1.2

Build # 17042528256

Build Type

Pull #42

github

Committed by ghukill

Commit Message

Support JSONLines as output format

Why these changes are being introduced:

When this harvester was first created, Transmogrifier would only accept XML as input.
With the creation of the GeoHarvester, Transmog now supports JSONLines as an input
format.  Writing JSONLines is a handy option for this harvester as well.

How this addresses that need:
* Write JSONLines if a `.jsonl` output file is passed.  Even simpler than the XML
output, we already have the data in a dataframe that converts nicely to
JSONLines.

Side effects of this change:
* None

Relevant ticket(s):
* https://mitlibraries.atlassian.net/browse/TIMX-542

Pull Request Pull Request #42: TIMX 542 - support JSONLines output

Run Details

12 of 12 new or added lines in 2 files covered. (100.0%)

403 of 403 relevant lines covered (100.0%)

1.0 hits per line