IBM / unitxt / 12765217246

81%

push

github

Add Tables Understanding Benchmark (#1506)

* init commit bench

Signed-off-by: ShirApp <shirashury@gmail.com>

* merge updates

* Make tables benchmark

Signed-off-by: elronbandel <elronbandel@gmail.com>

* modify prompts (instruction once)

* modify prompts (instruction once) in generation template

* change llm as judge metric for scigen (Yifan's code)

* updated recipes

* add table augmenter

* update table benchmark files

* delete some files from branch

* fix typo of augmeter list in benchmark code + update recipes to include loader limit

* fix typos

* drop personal scripts

* create updated json cards (tab fact+turl)

* updated cards (tab fact+turl)

* add tablebench visualization json file

* delete old file

* update df serializer test

* drop table bench visualization since it is not a part of the benchmark, and we are not sure about its evaluation metric

---------

Signed-off-by: ShirApp <shirashury@gmail.com>
Signed-off-by: elronbandel <elronbandel@gmail.com>
Co-authored-by: elronbandel <elronbandel@gmail.com>

1387 of 1735 branches covered (79.94%)

Branch coverage included in aggregate %.

8742 of 11023 relevant lines covered (79.31%)

0.79 hits per line