1
94%
main: 93%

Ran 09 Dec 2025 03:59PM UTC

Files 8

Run time 0s

Badge

Embed ▾

Committed 09 Dec 2025 03:53PM UTC coverage: 93.861% (+0.4%) from 93.503%

Job # 20069786007.1

Build Type

Pull #179

github

Committed by

ghukill

Commit Message

Join embeddings queries on record metadata

Why these changes are being introduced:

Two nice-to-have functionalities were missing from the first pass of embeddings
reading:

1. filter by record metadata columns now in embeddings schema
2. retrieve record metadata columns in embeddings read methods

There was a deliberate choice to keep embeddings read methods simple in the first
pass.  This builds on that work.

How this addresses that need:

For TIMDEXEmbeddings.read_batches_iter(), the base read method for all embeddings
read methods, perform a join to record metadata via the composite key (timdex_record_id,
run_id, run_record_offset).  Given that record metadata is very fast and memory safe, this
join is too.  By performing this join, we can expose record metadata columns that
intentionally don't exist in the embeddings schema -- e.g. 'source' or 'run_timestamp' --
for filtering and selecting.

Side effects of this change:
* Read methods for TIMDEXEmbeddings can filter and return columns only found in
records metadata tables/views.

Relevant ticket(s):
* https://mitlibraries.atlassian.net/browse/USE-143

Pull Request Pull Request #179: Join embeddings queries on record metadata

Coverage Stats

688 of 733 relevant lines covered (93.86%)

0.94 hits per line

MITLibraries / timdex-dataset-api / 20069786007 / 1
94%
main: 93%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job 20069786007.1

MITLibraries / timdex-dataset-api / 20069786007 / 1 94% main: 93%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job 20069786007.1

MITLibraries / timdex-dataset-api / 20069786007 / 1
94%
main: 93%

README BADGES
x