IBM / unitxt / 17639482433

81%

push

github

Add ReflectionToolCallingMetric and update related metrics (#1931)

* Add ReflectionToolCallingMetric and update related metrics

- Introduced ReflectionToolCallingMetric for assessing syntactic and semantic validity of tool calls.
- Updated MultiTurnToolCallingMetric description for clarity.
- Added reflection.json to catalog with appropriate descriptions.
- Enhanced test coverage for ReflectionToolCallingMetric and its reduction logic.

* removed redundant import which makes tests fail.

* Minor fix for mock provider name in llmevalkit

* Update descriptions for ReflectionToolCallingMetric and ReflectionToolCallingMetricSyntactic; enhance clarity and detail on evaluation criteria and installation instructions.

* Refactor ReflectionToolCallingMetric to use settings directly instead of unitxt.settings; update provider name format for watsonx.

* Fixed minor bugs to support different tasks.

* Fixed requirements issue, general import bug, and added some guards for the provider.

* Fixed pre-commit issues.

* made sure that we reinstall libraries from git.

* Add logging for installation URL and version info in internal pip action

* minor change

* fixed assignment of mock provider

* Update .github/actions/install-internal-pip/action.yml

Co-authored-by: Elron Bandel <elronbandel@gmail.com>

* removed two unittests that were causing problems and fixed assertEqual to assertFalse/assertTrue.

---------

Co-authored-by: Koren Lazar <koren.lazar@ibm.com>
Co-authored-by: Elron Bandel <elronbandel@gmail.com>

1607 of 2005 branches covered (80.15%)

Branch coverage included in aggregate %.

10944 of 13508 relevant lines covered (81.02%)

1.62 hits per line