• Home
  • Features
  • Pricing
  • Docs
  • Announcements
  • Sign In

kubernetes-sigs / inference-perf / 42133139974
87%

Build:
DEFAULT BRANCH: main
Ran 06 Nov 2026 12:08AM UTC
Jobs 1
Files 138
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

05 Nov 2026 11:56PM UTC coverage: 85.818% (+0.02%) from 85.803%
42133139974

push

github

web-flow
test: catch SGLang and TGI metric-name drift (#737)

Part of #669, SGLang and TGI half (vLLM was covered in #697). Tracked in
#606.

If a model server renames a metric, we keep querying the old name. The
query returns nothing and the field in `summary_prometheus_metrics.json`
is empty or 0, same as if the server had nothing to report. #382 fixed
one of these by hand (`sglang:cache_hit_rate` -> `sglang:token_usage`).
This adds tests that catch them automatically, and they found three
more:
* `sglang:time_per_output_token_seconds` doesn't exist in SGLang
v0.5.17, so the `time_per_output_token` section is empty on every SGLang
run. Replacement is `sglang:inter_token_latency_seconds`. Not fixed here
since this PR is test-only, the `KNOWN_UNRESOLVED` entry is the repro.
* `vllm:prompt_tokens` and `vllm:generation_tokens` were queried without
the `_total` suffix vLLM v0.26.0 uses, so `prompt_len` and `output_len`
read 0. Fixed since in #568, both now pass.

Changes:
* Added `e2e/tests/test_sglang_tgi_metric_names.py`, five checks per
server:
* Without a server: every name we query exists in the saved list for
that server (`e2e/testdata/server_metric_families/<server>.txt`), every
`KNOWN_UNRESOLVED` name is still queried and still missing, and each
saved list records its server, version, source and date.
* Against a live server: the server publishes the same names as the
saved list, and every name we query exists.
* Added `Metric.candidate_names()`, which returns the names a metric's
own queries will match. Previously the vLLM check copied the Prometheus
naming rules into the test (counter `X` can show up as `X_total`), and
that copy drifted from the query code, which is how the two vLLM names
passed #697's check while matching nothing. Both checks now ask the
metric instead. A unit test asserts every name a metric returns actually
appears in its query.
* Reworked `e2e/utils/metric_families.py`, which both checks share, to
resolve through `candidate_names()`... (continued)

18 of 18 new or added lines in 4 files covered. (100.0%)

13295 of 15492 relevant lines covered (85.82%)

0.86 hits per line

Jobs
ID Job ID Ran Files Coverage
1 42133139974.1 06 Nov 2026 12:08AM UTC 138
85.82
GitHub Action Run
Source Files on build 42133139974
  • Tree
  • List 138
  • Changed 5
  • Source Changed 4
  • Coverage Changed 5
Coverage ∆ File Lines Relevant Covered Missed Hits/Line
  • Back to Repo
  • Github Actions Build #42133139974
  • 17e4bfec on github
  • Prev Build on main (#42111912680)
  • Next Build on main (#42133375366)
  • Delete
STATUS · Troubleshooting · Open an Issue · Sales · Support · CAREERS · ENTERPRISE · START FREE TRIAL · SCHEDULE DEMO
ANNOUNCEMENTS · TWITTER · TOS & SLA · Supported CI Services · What's a CI service? · Automated Testing

© 2026 Coveralls, Inc