rendezqueue / rendezllama / 21236314882

86%

trunk: 87%

Build Type

push

github

Committed by grencez

Commit Message

Fix assistant_cli: rewrite to use direct llama.h API and fix tokenization

Rewrote eg/assistant_cli/main.cc to use the direct llama.h C API, implementing a standard inference loop with KV cache management and sampling.
Addressed a critical issue where special tokens generated by the chat template (e.g., `<s>`, `</s>`) were being tokenized as plain text, causing garbage output.
This was fixed by invoking `llama_tokenize` with `parse_special=true`.
Verified that the output is now coherent and comparable to `llama-completion`.

Run Details

81 of 95 new or added lines in 1 file covered. (85.26%)

8 existing lines in 3 files now uncovered.

2133 of 2422 relevant lines covered (88.07%)

111.75 hits per line