• Home
  • Features
  • Pricing
  • Docs
  • Announcements
  • Sign In

kubernetes-sigs / inference-perf
87%

Build:
DEFAULT BRANCH: main
Repo Added 04 Sep 2026 08:13PM UTC
Token NLLSCEurPmbo3BH5LnhZ5rwbNBvlYheVq regen
Build 143 Last
Files 150
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH main
branch: SELECT
CHANGE BRANCH
x
Sync Branches
  • No branch selected
  • main

17 Nov 2026 10:08PM UTC coverage: 87.189% (+0.4%) from 86.752%
44187044576

push

github

web-flow
feat(tools): convert vllm bench and aiperf argv into configs (#814)

Part of #755.

Adds `inference-perf-convert`, which reads a `vllm bench serve` or
`aiperf profile` argument list at a caller-supplied peer version and
emits an inference-perf config. The comment block records the source
argv, assumptions, and dropped workload-neutral flags. Unsupported
conversions refuse with a named reason and exit 2 without creating or
overwriting the output file; invalid peer arguments exit 1.

Changes:

- Mirror the verified vLLM v0.10.0 and AIPerf v0.12.0 flag surfaces,
including aliases, declared boolean negations, multi-value arguments,
and sweep refusals. Unknown flags and unverified versions refuse.
- Emit streaming and ignore-EOS explicitly. Match the peers'
request-count and dataset-pool rules, and map vLLM's rescaled random
arrival gaps to `load.type: constant`.
- Refuse AIPerf chat/messages and media input flags because the emitted
random generator only supports text completions. Remove the unused
multimodal mappings. The former image/video success tests now expect
refusals because those configs silently discarded media.
- Refuse `--apply-chat-template`: AIPerf compensates synthetic lengths
for template overhead while sending bare completions text;
inference-perf's setting wraps the wire prompt instead. Explicitly
disabling the flag still converts.
- Correct the warmup entry-reuse and BOS explanations, remove the stale
claim that chat/messages convert, and update the comparability
documentation. API keys and credential headers remain omitted from
configs and redacted in recorded argv.
- Use the in-tree parity cases from #736 as the acceptance oracle. Merge
current main and replace the converter's `Any` annotations with typed
parser options and Pydantic-validated configuration dictionaries to pass
the new type-escape check.

Validation:

- Before the payload fixes, the regression selection reported `12
failed, 1 passed`; the CLI cases failed with `Asserti... (continued)

1036 of 1106 new or added lines in 6 files covered. (93.67%)

15476 of 17750 relevant lines covered (87.19%)

0.87 hits per line

Relevant lines Covered
Build:
Build:
17750 RELEVANT LINES 15476 COVERED LINES
0.87 HITS PER LINE
Source Files on main
  • Tree
  • List 150
  • Changed 1
  • Source Changed 0
  • Coverage Changed 1
Coverage ∆ File Lines Relevant Covered Missed Hits/Line

Recent builds

Builds Branch Commit Type Ran Committer Via Coverage
44187044576 main feat(tools): convert vllm bench and aiperf argv into configs (#814) Part of #755. Adds `inference-perf-convert`, which reads a `vllm bench serve` or `aiperf profile` argument list at a caller-supplied peer version and emits an inference-perf con... push 17 Nov 2026 10:23PM UTC web-flow github
87.19
44168932103 main Add one-token TTFT tests (#944) Fixes #942 #914 fixed the bug: a streamed response with one output token now gets a TTFT from its single token timestamp. The native summary and per-request paths already have unit tests for it. This PR adds the t... push 17 Nov 2026 08:38PM UTC web-flow github
86.75
44142103159 main fix(utils): allow inference_perf to import on Windows (#1058) Fixes #1053 ## Summary Allow the package to import on Windows without changing Linux/macOS forkserver behavior. ## Root cause PR #526 unconditionally imported `ForkServerContext` a... push 17 Nov 2026 06:08PM UTC web-flow github
86.75
44140647921 main fix(datagen): cover full conversation_replay pool below concurrency (#1051) Fixes #1048 ## Old behavior #924 made `concurrency_level` count active conversations and, as a documented breaking change, listed that with `concurrency_level: C < num_... push 17 Nov 2026 05:58PM UTC web-flow github
86.78
44014323210 main Re-tokenize the shared-prefix word-start candidates in one batched call (#1097) Follow-up to #1094, for #1093. `build_word_start_token_ids` called `count_tokens` once per whitespace-prefixed id, so startup grew with the vocab. With a Hugging Fac... push 17 Nov 2026 04:17AM UTC web-flow github
86.75
43985176825 main Validate unary response bodies (#542) Part of #531 A unary 200 whose body was not a completion was read through `.get()` chains. A choice with no message or text, next to a valid usage, was recorded as a zero-token success. Unary chat and compl... push 17 Nov 2026 12:21AM UTC web-flow github
86.75
43980778874 main Log the first failed startup probe at info (#1091) Fixes #1032 Both startup wait loops logged every failed probe at debug, so with a wrong `base_url` nothing showed at the default log level until the deadline error, up to 600 s later. Now the fi... push 16 Nov 2026 11:50PM UTC web-flow github
86.73
43978425555 main Keep only single-token ids in the shared-prefix word-start pool (#1094) Fixes #1093 Hi @Bslabe123, this takes the fix you suggested in the issue. A length-1 question is only the word-start token, and the convergence loop never drops that first ... push 16 Nov 2026 11:35PM UTC web-flow github
86.72
43961807348 main Report the stage drop rate in the BR0.2 partial (#1090) Part of #799 (item 2 of the 11-13 list). Old behavior: the BR0.2 partial had no `results.observability.drop_rate`, though each stage counts the requests it generated but never dispatched (`d... push 16 Nov 2026 09:49PM UTC web-flow github
86.72
43955908636 main feat(datagen): add support for replaying TraceLab agentic traces (#857) ## Summary Issue kubernetes-sigs/inference-perf#813 asks for replaying the UW TraceLab coding-agent traces (Claude Code / Codex, https://tracelab.cs.washington.edu, https://a... push 16 Nov 2026 09:15PM UTC web-flow github
86.72
See All Builds (143)
  • Settings
  • Repo on GitHub
STATUS · Troubleshooting · Open an Issue · Sales · Support · CAREERS · ENTERPRISE · START FREE TRIAL · SCHEDULE DEMO
ANNOUNCEMENTS · TWITTER · TOS & SLA · Supported CI Services · What's a CI service? · Automated Testing

© 2026 Coveralls, Inc