IBM / unitxt / 12534862993

81%

push

github

Benjams/add bioasq miniwiki datasets (#1460)

* Add code for generating to catalog bioasq and miniwiki

* changed by hook pre-commit

* add the catalog for bioasq and miniwikipedia

* Update prepare/cards/rag/end_to_end/bioasq.py

Co-authored-by: Elron Bandel <elronbandel@gmail.com>

* Update prepare/cards/rag/end_to_end/miniwikipedia.py

Co-authored-by: Elron Bandel <elronbandel@gmail.com>

* Update prepare/cards/rag/end_to_end/miniwikipedia.py

Co-authored-by: Elron Bandel <elronbandel@gmail.com>

* Update prepare/cards/rag/end_to_end/miniwikipedia.py

Co-authored-by: Elron Bandel <elronbandel@gmail.com>

* Refactor bioasq and miniwikipedia processing: replace ListFieldValues with Wrap, update preprocess steps, and enhance dataset descriptions

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Add license tags to BioASQ dataset JSON files

Signed-off-by: Elron Bandel <elron.bandel@ibm.com>

* Refactor arena_hard processing steps: replace Apply with Cast and Lower for improved type handling

Signed-off-by: Elron Bandel <elron.bandel@ibm.com>

---------

Signed-off-by: elronbandel <elronbandel@gmail.com>
Signed-off-by: Elron Bandel <elron.bandel@ibm.com>
Co-authored-by: Elron Bandel <elronbandel@gmail.com>
Co-authored-by: Elron Bandel <elron.bandel@ibm.com>

1337 of 1666 branches covered (80.25%)

Branch coverage included in aggregate %.

8446 of 10558 relevant lines covered (80.0%)

0.8 hits per line