IBM / unitxt / 14365314700

81%

push

github

Add general formatter for chat api, with chat template based (#1728)

Add general formatter for chat api, with chat template based on the model tokenizer

The model_name should be overried at runtime (e.g. chat_api_with_tokenizer_chat_template[model_name=ibm-granite/granite-3.1-2b-instruct])

Currently, we already have HFSystemFormat based formats (formats.models.llava_interleave)
which may be used in the same why. But since the name is very confusing, I
add this format).

Co-authored-by: Elron Bandel <elronbandel@gmail.com>

1584 of 1968 branches covered (80.49%)

Branch coverage included in aggregate %.

9911 of 12361 relevant lines covered (80.18%)

0.8 hits per line