16059426820
97%
main: 93%

Ran 03 Jul 2025 07:58PM UTC

Jobs 1

Files 6

Run time 1min

Badge

Embed ▾

Committed 03 Jul 2025 07:54PM UTC coverage: 96.594%. Remained the same

Build # 16059426820

Build Type

push

github

Committed by

ghukill

Commit Message

Add 003 row group size data migration

Why these changes are being introduced:

It was discovered that some parquet files, mostly those before
2025-06-27, have potentially a single, very large row group instead
of maximum rows per group of 1k.

This has been addressed in recent work, resulting in no files after
2025-06-28 having a single row group if total rows exceed 1k.

How this addresses that need:

Migration script that re-packs row groups (rewrites) when a single
row group is found for a number of rows that should be split up.

Side effects of this change:
* None

Relevant ticket(s):
* https://mitlibraries.atlassian.net/browse/TIMX-512

Run Details

312 of 323 relevant lines covered (96.59%)

0.97 hits per line

Jobs

ID	Job ID	Ran	Files	Coverage
1	16059426820.1	03 Jul 2025 07:58PM UTC	6	96.59	GitHub Action Run

MITLibraries / timdex-dataset-api / 16059426820
97%
main: 93%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 16059426820

MITLibraries / timdex-dataset-api / 16059426820 97% main: 93%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 16059426820

MITLibraries / timdex-dataset-api / 16059426820
97%
main: 93%

README BADGES
x