IBM / unitxt / 17129746456
81%

DEFAULT BRANCH: main
Ran
Jobs 1
Files 64
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 80.81% (+0.04%) from 80.769%
17129746456

push

github

web-flow 
Add ReflectionToolCallingMetricSyntactic for evaluating tool call predictions referenceless (#1923)

* Add ReflectionToolCallingMetricSyntactic for evaluating tool call predictions

- Implemented ReflectionToolCallingMetricSyntactic to assess tool calls without references using static checks.
- Added corresponding JSON schema for the metric.
- Created example usage in evaluate_tool_calling_with_reflection.py.
- Updated test_metrics.py to include comprehensive tests for the new metric.

* Install llmevalkit before tests

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Update

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Another try

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Another try

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Another try

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Another try

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Another try

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Another try

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Another try

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Another try

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Update

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Fix style

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Fix examples

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Update the message

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Improve docs

Signed-off-by: elronbandel <elronbandel@gmail.com>

* Update

Signed-off-by: elronbandel <elronbandel@gmail.com>

---------

Signed-off-by: elronbandel <elronbandel@gmail.com>
Co-authored-by: Koren Lazar <koren.lazar@ibm.com>
Co-authored-by: elronbandel <elronbandel@gmail.com>

1595 of 1992 branches covered (80.07%)

Branch coverage included in aggregate %.

10840 of 13396 relevant lines covered (80.92%)

0.81 hits per line

Uncovered Existing Lines

Lines Coverage ∆ File
1
73.16
 -0.04% unitxt/metric_utils.py
1
86.62
 -0.07% unitxt/operators.py
551
75.25
 0.11% unitxt/metrics.py
Jobs
ID Job ID Ran Files Coverage
1 17129746456.1 64
80.81
 GitHub Action Run
Source Files on build 17129746456