IBM / unitxt / 26499736672

81%

push

github

fix: Compatibility with huggingface_hub 1.16, numpy 2.0, and pandas 3.0 (#1971)

* fix: Use namespaced HF dataset paths for huggingface_hub >= 1.16 compatibility

huggingface_hub 1.16+ enforces that dataset repository IDs must use the
'namespace/name' format. Bare dataset names (e.g., 'hellaswag') are no
longer accepted, causing HfUriError in CI.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Yoav Katz <katz@il.ibm.com>

* fix: Regenerate catalog JSONs with namespaced HF dataset paths

Run all prepare/cards scripts to update the catalog JSON files with the
full namespace/name format for HuggingFace dataset paths.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Yoav Katz <katz@il.ibm.com>

* fix: Replace evaluate library with direct sklearn calls in F1MultiLabel

The evaluate library's cached f1.py uses `float(score)` on numpy arrays,
which raises TypeError with numpy >= 2.0. Bypass the evaluate wrapper and
call sklearn's f1_score/precision_score/recall_score directly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Yoav Katz <katz@il.ibm.com>

* fix: Numpy 2.0 and pandas 3.0 compatibility fixes

- F1 metric: replace evaluate library wrapper with direct sklearn calls
  to avoid numpy 2.0 float() TypeError on 0-d arrays
- text2sql: cast DataFrame to str before sorting to avoid pandas 3.0
  TypeError when assigning string values to int64 columns
- wiki_bio: use namespaced HF dataset path (michaelauli/wiki_bio)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Yoav Katz <katz@il.ibm.com>

* fix: Regenerate wiki_bio catalog JSON with namespaced HF path

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Yoav Katz <katz@il.ibm.com>

* fix: Use namespaced HF path for winogrande (allenai/winogrande)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Yoav Katz <katz@il.ibm.com>

---------

Signed-off-by: Yoav Katz <katz@il.ibm... (continued)

1607 of 2008 branches covered (80.03%)

Branch coverage included in aggregate %.

10961 of 13536 relevant lines covered (80.98%)

0.81 hits per line