• Home
  • Features
  • Pricing
  • Docs
  • Announcements
  • Sign In

kubernetes-sigs / inference-perf / 44140647921
87%

Build:
DEFAULT BRANCH: main
Ran 17 Nov 2026 05:58PM UTC
Jobs 1
Files 144
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

17 Nov 2026 05:46PM UTC coverage: 86.778% (+0.03%) from 86.75%
44140647921

push

github

web-flow
fix(datagen): cover full conversation_replay pool below concurrency (#1051)

Fixes #1048

## Old behavior

#924 made `concurrency_level` count active conversations and, as a
documented breaking change, listed
that with `concurrency_level: C < num_conversations: N` only the first C
blueprints are ever replayed. A ready
slot stayed on its own blueprint, so blueprints C..N-1 were generated at
startup and then never dispatched —
silently, with no warning and no signal in the report.

## New behavior

A slot still only takes a new conversation once its previous
conversation has finished all of its turns, so
#924's meaning of concurrency is unchanged: one outstanding request per
slot, and simulated tool latency still
occupies the slot. What changes is which blueprint the slot continues
on: instead of always its own, it takes the
next blueprint its worker's token stream named. With enough completed
requests the whole pool is exercised, and
once every blueprint a worker can reach is active the slot recycles its
own blueprint exactly as before, so
`C == N` behaves as it does today.

This reverses #924's first-C call rather than fixing a regression: that
behavior was documented, not accidental,
and #540 item 3 asked for the opposite — a larger pool sampled at a
lower concurrency.

**Not comparable with earlier runs.** Results from a stage with
`concurrency_level` below `num_conversations` are
not comparable with runs made since #924. Those runs replayed only the
first C blueprints; here every blueprint
of the worker's pool is eligible at every concurrency, so the workload
is a different mix. Runs at
`concurrency_level == num_conversations` are unchanged and stay
comparable.

## Changes

- `conversation_replay_datagen.py`: slot state records its current
blueprint, next turn and conversation number.
A per-worker queue holds the blueprints that worker's own tokens named
and that no slot is running; a finished
conversation takes the next one and returns its previou... (continued)

33 of 33 new or added lines in 1 file covered. (100.0%)

14432 of 16631 relevant lines covered (86.78%)

0.87 hits per line

Jobs
ID Job ID Ran Files Coverage
1 44140647921.1 17 Nov 2026 05:58PM UTC 144
86.78
GitHub Action Run
Source Files on build 44140647921
  • Tree
  • List 144
  • Changed 2
  • Source Changed 1
  • Coverage Changed 2
Coverage ∆ File Lines Relevant Covered Missed Hits/Line
  • Back to Repo
  • Github Actions Build #44140647921
  • fca4217a on github
  • Prev Build on main (#44014323210)
  • Next Build on main (#44142103159)
  • Delete
STATUS · Troubleshooting · Open an Issue · Sales · Support · CAREERS · ENTERPRISE · START FREE TRIAL · SCHEDULE DEMO
ANNOUNCEMENTS · TWITTER · TOS & SLA · Supported CI Services · What's a CI service? · Automated Testing

© 2026 Coveralls, Inc