• Home
  • Features
  • Pricing
  • Docs
  • Announcements
  • Sign In

kubernetes-sigs / inference-perf / 43156269305
87%

Build:
DEFAULT BRANCH: main
Ran 12 Nov 2026 04:53AM UTC
Jobs 1
Files 141
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

12 Nov 2026 04:39AM UTC coverage: 86.405% (-0.006%) from 86.411%
43156269305

push

github

web-flow
fix(reportgen): calculate achieved rate using send intervals (#1055)

Part of #821

Old behavior: `achieved_rate` was the request count divided by
`send_duration`, the span from the first send to the last. n sends span
only n - 1 gaps, so it overstated the sustained rate by n / (n - 1).
Five requests sent 1.0s apart reported 1.25/s.

New behavior: `achieved_rate` is `(n - 1) / send_duration`, so the same
five requests report 1.0/s. A zero-width window (one request, or a burst
at one instant) still reports 0.0. That is part 2 of #821, which needs a
schema decision, so it is not changed here.

Changes:
- `reportgen/base.py`: divide the gaps, not the count, by
`send_duration`.
- `e2e/utils/load_shape.py`: `observed_send_rate` uses the same
arithmetic, since the e2e check compares the two.
- Tests from #733 that pinned the old value now pin the new one: 1.25 ->
1.0, 4/1.5 -> 3/1.5, and 4/3 -> 1.0 in the e2e helper test. The burst
test now uses 401 requests so it still reads exactly 40 qps.
- `docs/loadgen.md`: the sample's `achieved_rate` recomputed from its
own count and `send_duration` (29 / 29.604 = 0.980).

Failing on main before the fix:

FAILED
tests/required/reportgen/test_load_summary_accuracy.py::test_achieved_rate_counts_gaps_over_send_duration
    E       assert 1.25 == 1.0 ± 1.0e-06
FAILED
tests/required/reportgen/test_load_summary_accuracy.py::test_achieved_rate_tracks_send_duration_not_completion_times
    E       assert 2.6666666666666665 == 2.0 ± 2.0e-06
FAILED
e2e/tests/test_load_shape_accuracy_sim.py::test_observed_send_rate_matches_reportgen_arithmetic
    E       assert 1.3333333333333333 == 1.0 ± 1.0e-06
FAILED
e2e/tests/test_load_shape_accuracy_sim.py::test_arrivals_spread_rejects_a_burst_the_rate_check_accepts
    E       assert 40.1 == 40.0 ± 4.0e-05

Testing: unit tests, ruff, mypy --strict, the type-escape check and the
doc-sync checks pass. I did not run the llm-d-inference-sim e2e tests,
including test_achieved_rate_matches_con... (continued)

1 of 1 new or added line in 1 file covered. (100.0%)

1 existing line in 1 file now uncovered.

13741 of 15903 relevant lines covered (86.41%)

0.86 hits per line

Coverage Regressions

Lines Coverage ∆ File
1
79.17
-1.04% inference_perf/datagen/base.py
Jobs
ID Job ID Ran Files Coverage
1 43156269305.1 12 Nov 2026 04:53AM UTC 141
86.41
GitHub Action Run
Source Files on build 43156269305
  • Tree
  • List 141
  • Changed 2
  • Source Changed 1
  • Coverage Changed 2
Coverage ∆ File Lines Relevant Covered Missed Hits/Line
  • Back to Repo
  • Github Actions Build #43156269305
  • 92fd2b28 on github
  • Prev Build on main (#43155405771)
  • Next Build on main (#43260477553)
  • Delete
STATUS · Troubleshooting · Open an Issue · Sales · Support · CAREERS · ENTERPRISE · START FREE TRIAL · SCHEDULE DEMO
ANNOUNCEMENTS · TWITTER · TOS & SLA · Supported CI Services · What's a CI service? · Automated Testing

© 2026 Coveralls, Inc