• Home
  • Features
  • Pricing
  • Docs
  • Announcements
  • Sign In

kubernetes-sigs / inference-perf / 41889509256
87%

Build:
DEFAULT BRANCH: main
Ran 04 Nov 2026 06:51PM UTC
Jobs 1
Files 137
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

04 Nov 2026 06:40PM UTC coverage: 85.694% (+0.03%) from 85.668%
41889509256

push

github

web-flow
fix: keep the response text verbatim in multi-turn history (#1010)

Part of #649. The token-id half (a model's ids that don't re-encode to
themselves, harmony channel tokens) stays with #971.

**Old behavior.** After each turn the session stored `prompt + " " +
response`. The completion text already starts with whatever whitespace
the model generated, so the next prompt carried a space the server never
generated. With a Llama-style tokenizer it becomes a lone `▁` token
where the generated tokens start; with a byte-level BPE it changes the
first generated token. Either way the next turn's ids leave the cached
ones at the first generated token, and none of the generated KV blocks
are reused. Sessions that keep history (conversation_replay) also
`strip()`ed each stored turn, which cut a leading space of the turn's
question and trailing newlines of the response, so those turns diverged
earlier, inside the previous prompt.

#796 pinned the shared-prefix boundary with tests that call
`update_context(prompt + response)` directly. The request path passes
`prompt + " " + response`, so shared-prefix sessions still got the extra
space in a run.

**New behavior.** Every turn's prompt starts with the previous turn's
prompt followed by the response text as the server returned it.

Changes:
- `UserSessionCompletionAPIData.process_response` and
`_ConversationReplayAPIData.process_response` pass `self.prompt +
self.model_response`.
- `LocalUserSession.update_context`, history without a preserved
boundary: drop only the space that joined the turn to the history,
instead of `strip()`.
- New `tests/required/apis/test_user_session_turn_boundary.py`: four
session kinds (no tokenizer, history with and without a system prompt,
shared prefix) through both request classes for four turns, checking
that turn N+1 starts with turn N's prompt plus its reply. One more test
loads a Metaspace tokenizer from a local `tokenizer.json` and checks
that turn 1's ids start with turn 0's prom... (continued)

9 of 9 new or added lines in 2 files covered. (100.0%)

1 existing line in 1 file now uncovered.

13220 of 15427 relevant lines covered (85.69%)

0.86 hits per line

Coverage Regressions

Lines Coverage ∆ File
1
79.17
-1.04% inference_perf/datagen/base.py
Jobs
ID Job ID Ran Files Coverage
1 41889509256.1 04 Nov 2026 06:51PM UTC 137
85.69
GitHub Action Run
Source Files on build 41889509256
  • Tree
  • List 137
  • Changed 3
  • Source Changed 2
  • Coverage Changed 3
Coverage ∆ File Lines Relevant Covered Missed Hits/Line
  • Back to Repo
  • Github Actions Build #41889509256
  • b9192e32 on github
  • Prev Build on main (#41703960940)
  • Next Build on main (#42088289944)
  • Delete
STATUS · Troubleshooting · Open an Issue · Sales · Support · CAREERS · ENTERPRISE · START FREE TRIAL · SCHEDULE DEMO
ANNOUNCEMENTS · TWITTER · TOS & SLA · Supported CI Services · What's a CI service? · Automated Testing

© 2026 Coveralls, Inc