IBM / unitxt / 13399374718

81%

push

github

Text2sql execution accuracy metric updates (#1604)

* revised text2sql execution accuracy metric

Signed-off-by: Oktie Hassanzadeh <hassanzadeh@us.ibm.com>

* sql execution accuracy catalog card

Signed-off-by: Oktie Hassanzadeh <hassanzadeh@us.ibm.com>

* get_sql processor card update

Signed-off-by: Oktie Hassanzadeh <hassanzadeh@us.ibm.com>

* adding text2sql non-execution accuracy metric

Signed-off-by: Oktie Hassanzadeh <hassanzadeh@us.ibm.com>

* ruff fix

Signed-off-by: Oktie Hassanzadeh <hassanzadeh@us.ibm.com>

* adding sqlparse dependency for tests

Signed-off-by: Oktie Hassanzadeh <hassanzadeh@us.ibm.com>

* moving text2sql metrics functions to sql_utils

Signed-off-by: Oktie Hassanzadeh <hassanzadeh@us.ibm.com>

* fixing sql_utils ruff issues

Signed-off-by: Oktie Hassanzadeh <hassanzadeh@us.ibm.com>

* text2sql metrics: empty string is not SQL

Signed-off-by: Oktie Hassanzadeh <hassanzadeh@us.ibm.com>

* text2sql metric: check if dfs are non-empty before comparison

Signed-off-by: Oktie Hassanzadeh <hassanzadeh@us.ibm.com>

* text2sql metrics: fixing is_subset for output dfs

Signed-off-by: Oktie Hassanzadeh <hassanzadeh@us.ibm.com>

* sql_utils SQL API fixes

Signed-off-by: Oktie Hassanzadeh <hassanzadeh@us.ibm.com>

* text2sql metric: returning gold sql exec error

Signed-off-by: Oktie Hassanzadeh <hassanzadeh@us.ibm.com>

* cleanup, change info message

Signed-off-by: Yotam Perlitz <y.perlitz@ibm.com>

---------

Signed-off-by: Oktie Hassanzadeh <hassanzadeh@us.ibm.com>
Signed-off-by: Yotam Perlitz <y.perlitz@ibm.com>
Co-authored-by: Yotam Perlitz <y.perlitz@ibm.com>

1534 of 1889 branches covered (81.21%)

Branch coverage included in aggregate %.

9656 of 11940 relevant lines covered (80.87%)

0.81 hits per line