IBM / unitxt / 13395178397

81%

push

github

add ragbench faithfulness cards (#1598)

* allow multi reference in hhem metric
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* add ragbench faithfulness cards
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* add "mistral-large-instruct" to provider
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* add "mistral-large-instruct" classification engines
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* add rag judges that use mistral-large-instruct
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* fix hhem multi reference
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* Revert "fix hhem multi reference"

This reverts commit 0abc51808.

* fix hhem multi reference
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* catch openai.BadRequestError in inference
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* fix answer correctness template
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* remove code added by error
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* bugfix in llm_as_judge_from_template
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* add comment
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* fix typo
Signed-off-by: lilacheden <lilach.edel@gmail.com>

---------

Co-authored-by: elronbandel <elronbandel@gmail.com>

1498 of 1841 branches covered (81.37%)

Branch coverage included in aggregate %.

9500 of 11725 relevant lines covered (81.02%)

0.81 hits per line