• Home
  • Features
  • Pricing
  • Docs
  • Announcements
  • Sign In

kubernetes-sigs / inference-perf / 42093717096
87%

Build:
DEFAULT BRANCH: main
Ran 05 Nov 2026 07:37PM UTC
Jobs 1
Files 137
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

05 Nov 2026 07:25PM UTC coverage: 85.736% (+0.01%) from 85.725%
42093717096

push

github

web-flow
Fix exact-length prompt generation oscillating when tokens merge (#1007)

Fixes #1006

**Old behavior:** `generate_random_exact_length_text` pads a short
prompt with random tokens and trims a long one. When an appended token
merges with its neighbour on re-tokenization, the count jumps by 2 and
the loop bounces between N-1 and N+1 until it raises a misleading
"tokenizer mismatch" error. With `ibm-granite/granite-3.3-8b-instruct`
at 16,000 tokens this hits roughly 1 prompt in 8,000.

**New behavior:** when a length repeats, the tail token is dropped and
fresh random tokens are drawn in its place, so the loop escapes. Runs
that never repeat a length consume the same random numbers as before, so
seeded output is unchanged. The error message also names non-convergence
as a possible cause.

Changes:
- `datagen_utils.py`: track seen lengths in
`generate_random_exact_length_text`; on a repeat, drop the last token
before padding or trimming. Reword the `ValueError`.
- `test_datagen_utils.py`: a fake tokenizer that merges across a token
divisible by 5, run for seeds 3, 4 and 14 at `target_len=8`.

**Failing on main before the fix:**
```
E       ValueError: Could not generate a prompt of exactly 8 tokens after 20 attempts (got 7). This usually means tokenizer.pretrained_model_name_or_path does not match the model the server is running.
```
(3 failed, 4 passed.) With the fix all 7 pass.

**Real tokenizer:** the issue's reproduction with seed 14611
(granite-3.3-8b-instruct, 16,000 tokens) fails with `got 15999` before
the change and returns exactly 16000 after.

**Not run:** `pdm run validate` and `pdm run test:e2e` (no pdm or
`llm-d-inference-sim` here). ruff is clean and mypy reports nothing for
the touched files; `tests/required/datagen` passes.

14 of 14 new or added lines in 2 files covered. (100.0%)

13254 of 15459 relevant lines covered (85.74%)

0.86 hits per line

Jobs
ID Job ID Ran Files Coverage
1 42093717096.1 05 Nov 2026 07:37PM UTC 137
85.74
GitHub Action Run
Source Files on build 42093717096
  • Tree
  • List 137
  • Changed 2
  • Source Changed 2
  • Coverage Changed 2
Coverage ∆ File Lines Relevant Covered Missed Hits/Line
  • Back to Repo
  • Github Actions Build #42093717096
  • 722075bd on github
  • Prev Build on main (#42088289944)
  • Next Build on main (#42096251049)
  • Delete
STATUS · Troubleshooting · Open an Issue · Sales · Support · CAREERS · ENTERPRISE · START FREE TRIAL · SCHEDULE DEMO
ANNOUNCEMENTS · TWITTER · TOS & SLA · Supported CI Services · What's a CI service? · Automated Testing

© 2026 Coveralls, Inc