rendezqueue / rendezllama

88%

trunk: 87%

LAST BUILD ON BRANCH assistant_cli_fix-4664542843336424534

branch: assistant_cli_fix-4664542843336424534

CHANGE BRANCH

x

Reset

• assistant_cli_fix-4664542843336424534
• add-inference-test
• alpaca
• assistant-cli-cleanup-719797123915082777
• assistant_cli
• assistant_cli-context-length-16482133921599447173
• assistant_cli_example-16863409140435454080
• batch
• batch-token-generation-16868365038061602641
• blas
• chat-tests-1130249731353493222
• chat-trajectory-cc-target-5413662329449254524
• context_base
• contributing-docs
• cov
• coverage-fix-14520834744988857627
• decode
• defaults
• delim
• dep-rope
• deterministic
• dflt
• display-test-8337381805460483581
• enable-blas-9284002834420640955
• eom
• eosinsert
• fallback-boundary-prefix-9540685450522850437
• fix-inference-batch-pos-10855130293914255684
• fix-localserv-determinism-3141128589317180522
• fix-localserv-test-windows-9603322172943378209
• fix-regen-seed-10117453100447507096
• fix-vulkan-oom-remove-openblas-15898943122701797407
• fix-windows-build-error-4343694421170532716
• gen
• http
• increase-ci-timeout-16256491997374578865
• inference-test-sampling-coverage-5719570704248527310
• inference-test-simplify-3779826128337906892
• jules-12111104592075292061-8f5973c0
• lint-check-indent-10360893227382872155
• localserv
• localserv-404-fix-10230615575214195740
• localserv-determinism-16728826537529082726
• localserv-flakiness-183015914988436779
• localserv-openai-integration-4266068947578723984
• localserv-serve-static-files-14364234858281320394
• makefile-ggml-defaults-3859380595056245199
• miku
• orca
• prefac_gguf
• refactor-localserv-compat-5517632047937205306
• remove-redundant-libs-13700672902199106792
• renovate/major-all-dependencies
• rewrite
• schema
• sstream
• stage
• suffix
• test-parse-options-17607259460580744177
• tinystories-test-3346048418049655079
• tmp
• tmp_sx
• trunk
• twitch_sentiment
• uncrustify-lint-6053234437761304018
• update
• update-assistant-cli-12167333632533804948
• update-localserv-chat-17281043732855938939
• vulkan

Build # 21216901197

Build Type

push

github

Committed by google-labs-jules[bot]

Commit Message

Fix assistant_cli: rewrite to use direct llama.h API and fix tokenization

Rewrote eg/assistant_cli/main.cc to use the direct llama.h C API, implementing a standard inference loop with KV cache management and sampling.
Addressed a critical issue where special tokens generated by the chat template (e.g., `<s>`, `</s>`) were being tokenized as plain text, causing garbage output.
This was fixed by invoking `llama_tokenize` with `parse_special=true`.
Verified that the output is now coherent and comparable to `llama-completion`.

Run Details

81 of 95 new or added lines in 1 file covered. (85.26%)

8 existing lines in 3 files now uncovered.

2133 of 2422 relevant lines covered (88.07%)

111.78 hits per line