kubernetes-sigs / inference-perf / 39337572130
84%

DEFAULT BRANCH: main
Ran
Jobs 1
Files 136
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 83.638%. Remained the same
39337572130

push

github

web-flow 
[URGENT] skip known-bad vLLM releases in the e2e tests (#908)

vLLM v0.32.0 shipped on Oct 18 without the fix for its CPU slot-mapping
fallback (vllm-project/vllm#61131, fixed on main in #61992 after the
release branch was cut). Every vllm-openai-cpu:v0.32.0 server dies on
its
first request, so the floating `latest` entry in both release tables has
been red since, on main and on every PR that pushed after the release.

Before, the only way around a release that is broken upstream was to pin
the floating entry to an older tag and remember to unpin. Now
e2e/vllm_known_bad.txt lists such releases, one `<version> <reason>` per
line. After a server starts, the job asks it for its version and skips
the pass with a warning annotation and a step-summary line when that
version is listed, unless the tag itself names the version: a pin at a
broken release still goes red, only floating tags route around it. The
match is on the exact `/version` string, so the entry goes inert the
moment `latest` floats to 0.32.1 and nothing needs flipping back.

Changes:
* e2e/known_bad.py: reads the table, asks a server for its
  version, decides. Stdlib only, strict-typed, with a two-command CLI
  the workflow calls.
* Both workflow loops call it after start instead of curling /version
  inline.
* e2e/vllm_known_bad.txt: 0.32.0, pointing at the upstream issue.
* e2e/tests/test_known_bad.py: pins the module. Runs in the serverless
  e2e step, ahead of the live loop.
* pyproject: type-check covers the new module.

Other changes:
* One sentence in each release table header and in docs/e2e_tests.md
  pointing at the file.

12360 of 14778 relevant lines covered (83.64%)

0.84 hits per line

Jobs
ID Job ID Ran Files Coverage
1 39337572130.1 136
83.64
GitHub Action Run
Source Files on build 39337572130