kubernetes-sigs / inference-perf / 36058120120
81%

DEFAULT BRANCH: main
Ran
Jobs 1
Files 125
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 80.798% (+0.04%) from 80.76%
36058120120

push

github

web-flow 
test: cover multi-turn session turn serialization and transcript semantics (#677)

Addresses: #332

## Problem

The multi-turn path has no test that puts two turns in contention on the
same
session, so the waiter queue in `LocalUserSession.get_context` is never
entered.
The existing tests drive a session sequentially (acquire, release,
acquire),
which exercises the lock but not the handoff.

That gap matters because losing the handoff does not raise. A
conversation whose
turns stop serializing degrades into concurrent single-turn requests
that happen
to share a session id, and the run still reports clean numbers. There is
no
failure to notice.

## Change

Test-only: 220 lines added to `tests/required/apis/test_user_session.py`
in three
new classes.

**`TestUserSessionTurnSerialization`** puts turns in genuine contention:

- `test_concurrent_turns_never_overlap`: five turns gathered on one
session, each
yielding while holding the slot so an unserialized turn would
interleave.
  Asserts the peak number of concurrent holders is 1.
- `test_queued_turns_are_served_in_arrival_order`: queued turns are
FIFO, not
  reordered.
- `test_sessions_do_not_block_each_other`: serialization is per session.
A slow
conversation must not stall unrelated load, or reported latency folds in
  queueing that no server ever saw.
- `test_concurrent_turns_build_one_ordered_transcript`: four turns
dispatched
concurrently through the real `to_request_body` / `process_response`
path still
  produce `"SYSTEM Q0 A0 Q1 A1 Q2 A2 Q3 A3"`.

**`TestMultiTurnConversationSemantics`** covers what the workload is
for:

- `test_each_turn_carries_the_prior_transcript`: asserts the exact
growing prefix
(`"SYSTEM Q0"`, `"SYSTEM Q0 A0 Q1"`, ...). Each turn being a longer
prefill than
the last, and a cached prefix the server can reuse, is the point of the
workload.
- `test_turns_are_tagged_with_session_and_round`:
`extra_info["user_session"]` and
`extra_info["chat_round"]` advance 1, 2, 3. Reporting... (continued)

10574 of 13087 relevant lines covered (80.8%)

0.81 hits per line

Jobs
ID Job ID Ran Files Coverage
1 36058120120.1 125
80.8
GitHub Action Run
Source Files on build 36058120120