rendezqueue / rendezllama / 21216901197

88%

trunk: 87%

Build Type

push

github

Committed by google-labs-jules[bot]

Commit Message

Fix assistant_cli: rewrite to use direct llama.h API and fix tokenization

Rewrote eg/assistant_cli/main.cc to use the direct llama.h C API, implementing a standard inference loop with KV cache management and sampling.
Addressed a critical issue where special tokens generated by the chat template (e.g., `<s>`, `</s>`) were being tokenized as plain text, causing garbage output.
This was fixed by invoking `llama_tokenize` with `parse_special=true`.
Verified that the output is now coherent and comparable to `llama-completion`.

Run Details

81 of 95 new or added lines in 1 file covered. (85.26%)

8 existing lines in 3 files now uncovered.

2133 of 2422 relevant lines covered (88.07%)

111.78 hits per line