MITLibraries / timdex-dataset-api / 19442228121

94%

main: 93%

Pull #175

github

New embeddings dataset source and write functionality

Why these changes are being introduced:

We will begin storing embeddings associated with TIMDEX records in
the TIMDEX dataset, where read and write functionality is managed by
this library.

We will need to ability to write and read embeddings,
but will start with establishing the new data source structure and
write methods.  Read methods are to follow, and may require some
additional linkages with the `TIMDEXDataset` class and read methods.

How this addresses that need:

This commit adds a new data source for the dataset in the form of
the new `TIMDEXEmbeddings` class.  This class will encapsulate
write and read methods for embeddings, with a composite key of
(timdex_record_id, run_id, run_record_offset) tethering the embeddings
to specific TIMDEX record versions in the dataset.

At this time only a write() method exists, writing embeddings to
a `/embeddings` folder in the dataset.

Side effects of this change:
* None

Relevant ticket(s):
* https://mitlibraries.atlassian.net/browse/USE-142

64 of 64 new or added lines in 2 files covered. (100.0%)

588 of 627 relevant lines covered (93.78%)

0.94 hits per line