IBM / unitxt / 12749236191

81%

push

github

Fixes and adjustment in rag metrics and related inference engines (#1466)

* add new classification engines and remove deleted llama-3-70b-instruct in bam
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* allow proprietary data on RITSInferenceEngine
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* propagate score_prefix from metricPipeline to its metric
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* Adjust autorag metrics to unitxt flow with rag.response_generation task
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* Adjust granite_guardian to unitxt flow
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* Add AzureOpenAIInferenceEngine
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* lowercase engine label
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* move to azure openai classification engines
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* set chat_api format for openai
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* update secrets
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* delete old inference engine
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* update secrets
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* add numeric and verbal postprocessors
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* fix rag judge example
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* add numeric and verbal rag judge templates
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* add rag judges that use the new templates
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* fix import
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* rename metrics with correct template
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* avoid import from prepare
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* remove old metrics
Signed-off-by: lilacheden <lilach.edel@gmail.com>

* add token overlap based context relevance and answer relevance metrics... (continued)

1383 of 1731 branches covered (79.9%)

Branch coverage included in aggregate %.

8725 of 11004 relevant lines covered (79.29%)

0.79 hits per line