• Home
  • Features
  • Pricing
  • Docs
  • Announcements
  • Sign In

kubernetes-sigs / inference-perf / 42133375366
87%

Build:
DEFAULT BRANCH: main
Ran 06 Nov 2026 12:10AM UTC
Jobs 1
Files 139
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

05 Nov 2026 11:58PM UTC coverage: 85.895% (+0.08%) from 85.818%
42133375366

push

github

web-flow
fix: record a 200 that carries an error payload or nothing as a failure (#744)

Fixes #713

Old behavior: a 200 counted as a success whatever its body held. vLLM
and SGLang emit `data: {"error": {...}}` when generation fails
mid-stream. A proxy can return the OpenAI error object whole. Both
landed in the success bucket with `output_tokens: 0`, an empty
`response` and no `error`.

New behavior: two rules, raised from `process_response` and handled by
the client the way a broken stream already is (`StreamInterruptedError`,
#530). The request is recorded as failed and the body is kept as
`response_data`.

* `InBandError`: the body, or any SSE frame, carries a top-level
`error`. `error_msg` is the JSON of the body or frame that carries it,
the same shape as a non-200 body. For the OpenAI dict form the summary
message is the server's `message`. For TGI's string form it is the whole
JSON, since `parse_error_message` only unwraps a dict.
* `EmptyResponseError`: the body yields no content, no reasoning and no
`usage`. A well-formed empty completion always carries `usage`. It is
mandatory in a unary body and every stream asks for it with
`stream_options.include_usage`. This fires only on bodies that are not
completions: an error page or an empty stream behind a 200, or `{}`.

The empty rule is unconditional, not opt-in. A server that ignores
`include_usage` and sends an empty completion would be recorded as
`EmptyResponseError`. I could not name a current server that does that.
The same holds for a stream that carries only `delta.tool_calls` and no
`usage`. #914 adds `has_tool_calls` to the parsed stream. Whichever of
the two lands second adds it to the empty check. If this is too strict,
a config toggle is the alternative.

Changes:
* `inference_perf/apis/response_errors.py`: `InvalidResponseError`,
`InBandError`, `EmptyResponseError` and `in_band_error()`.
* `parse_sse_stream` drains the stream, then raises `InBandError` on an
error frame. An error frame fol... (continued)

61 of 61 new or added lines in 10 files covered. (100.0%)

13355 of 15548 relevant lines covered (85.9%)

0.86 hits per line

Jobs
ID Job ID Ran Files Coverage
1 42133375366.1 06 Nov 2026 12:10AM UTC 139
85.9
GitHub Action Run
Source Files on build 42133375366
  • Tree
  • List 139
  • Changed 9
  • Source Changed 9
  • Coverage Changed 9
Coverage ∆ File Lines Relevant Covered Missed Hits/Line
  • Back to Repo
  • Github Actions Build #42133375366
  • 0f041deb on github
  • Prev Build on main (#42133139974)
  • Next Build on main (#42297162286)
  • Delete
STATUS · Troubleshooting · Open an Issue · Sales · Support · CAREERS · ENTERPRISE · START FREE TRIAL · SCHEDULE DEMO
ANNOUNCEMENTS · TWITTER · TOS & SLA · Supported CI Services · What's a CI service? · Automated Testing

© 2026 Coveralls, Inc