kubernetes-sigs / inference-perf / 36158412668

81%

push

github

fix(datagen): preserve shared-prefix multi-turn sessions (#796)

## Summary

- register configured shared-prefix sessions and rebuild them cleanly
after per-stage registry resets
- send only the new question on each round while preserving the exact
jointly decoded prefix/question boundary
- add max_model_len support so accumulated histories are bounded,
including empty-prefix sessions
- isolate generated session IDs across generator instances and document
the multi-turn behavior

Fixes #616

## Validation

- `python -m pytest tests -q -o log_cli=false --tb=short` — 1511 passed,
7 skipped
- focused session/shared-prefix/conversation replay regressions — 24
passed
- Ruff check and format check on all changed Python files
- strict mypy on both changed source modules
- CLI documentation sync check
- independent review of session lifecycle, zero-prefix history,
tokenizer boundaries, and ConversationReplay compatibility

No live model/GPU endpoint was required; the reported behavior is
reproduced and validated through the in-process request-building and
response-session lifecycle.

---------

Signed-off-by: git-jxj <65210887+git-jxj@users.noreply.github.com>

66 of 67 new or added lines in 4 files covered. (98.51%)

10623 of 13133 relevant lines covered (80.89%)

0.81 hits per line