• Home
  • Features
  • Pricing
  • Docs
  • Announcements
  • Sign In

kubernetes-sigs / inference-perf / 40406160516
87%

Build:
DEFAULT BRANCH: main
Ran 26 Oct 2026 09:48PM UTC
Jobs 1
Files 136
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

26 Oct 2026 09:39PM UTC coverage: 84.754% (+0.004%) from 84.75%
40406160516

push

github

web-flow
fix: keep one session object per id in conversation_replay (#701)

Rebased on `main` (`10a29e2`). Tests, ruff, `mypy --strict` and the
doc-sync checks all pass.

Fixes the hang reported in #700. Likely the same cause as #469, and
possibly llm-d-benchmark#1688.

## Problem

In `conversation_replay`, one session id could map to two different
`LocalUserSession` objects.

Requests don't hold a session reference.
`UserSessionCompletionAPIData.user_session` looks up
`get_instance(self.user_session_id)` when it's used, and
`_new_session()` overwrites the registry unconditionally. So if an id is
rebuilt while one of its requests is in flight, that request reads its
context from the old object and calls `update_context()` on the new one.

**Where the collision comes from: the stage boundary.** `get_data()`
restarts `data_index` at 0 every stage, but `self.user_sessions` keeps
the id each slot rolled over to in the previous stage
(`slot_0_convo_1`). In stage 1 the slot is re-primed under that id, and
its first rollover computes `slot_0_convo_1` *again* while the slot's
earlier turns are still in flight. Within a single stage, every rollover
id is new and nothing collides.

(An earlier revision of this PR said the trigger was `load_lazy_data()`
being called repeatedly for the same `data_index`. That was wrong: each
index is materialized once.)

Consequences:

- **Hang.** Turns parked on the old object's future are never woken.
Only `update_context()` resolves them, and every later turn calls it on
the new object. `finished_requests_counter` never reaches
`num_requests`, so the stage loop spins forever with no traceback.
Reproduced with the real CLI, a 3-stage run, and a stub server: stage 2
never completes on `main`.
- **A straggler's response lands in the wrong conversation's history,
mangled.**
- **A lock is released that was never acquired.** That breaks the
serialization `LocalUserSession` exists to provide.

The last two don't hang. The run completes and wr... (continued)

3 of 3 new or added lines in 1 file covered. (100.0%)

12552 of 14810 relevant lines covered (84.75%)

0.85 hits per line

Jobs
ID Job ID Ran Files Coverage
1 40406160516.1 26 Oct 2026 09:48PM UTC 136
84.75
GitHub Action Run
Source Files on build 40406160516
  • Tree
  • List 136
  • Changed 1
  • Source Changed 1
  • Coverage Changed 1
Coverage ∆ File Lines Relevant Covered Missed Hits/Line
  • Back to Repo
  • Github Actions Build #40406160516
  • 4db22f31 on github
  • Prev Build on main (#40405571998)
  • Next Build on main (#40409384465)
  • Delete
STATUS · Troubleshooting · Open an Issue · Sales · Support · CAREERS · ENTERPRISE · START FREE TRIAL · SCHEDULE DEMO
ANNOUNCEMENTS · TWITTER · TOS & SLA · Supported CI Services · What's a CI service? · Automated Testing

© 2026 Coveralls, Inc