IBM / unitxt / 13101814348

81%

push

github

Revisit huggingface cache policy (#1564)

* Revisit huggingface cache policy

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Enable streaming for LoadFromHFSpace and clean up commented code

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Disable Hugging Face datasets cache in CatalogPreparationTestCase

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Enable streaming for wiki_bio loader in TaskCard and update JSON configuration

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Add conditional test card execution for 'doqa_travel' subset in chat_rag_bench

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Enhance memory and performance logging in catalog preparation tests

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Return parallel execution to 1 and adjust modulo for deterministic test runs

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Try 1

Signed-off-by: elronbandel <elronbandel@gmail.com>

* try 1 fixed

Signed-off-by: elronbandel <elronbandel@gmail.com>

* trial 2

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Stop testing social iqa until problem resolved

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Update social iqa card to use specific revision and enable testing

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Refactor translation card testing logic and remove unused dataset loading

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Update head_qa card loader path and streamline dataset configuration

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Enable streaming for websrc card loader in configuration

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Add revision reference to Winogrande card loaders

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Add revision reference to PIQA card loader

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Update

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Another trial

Signed-off-by: elronbandel <e... (continued)

1451 of 1823 branches covered (79.59%)

Branch coverage included in aggregate %.

9163 of 11561 relevant lines covered (79.26%)

0.79 hits per line