• Home
  • Features
  • Pricing
  • Docs
  • Announcements
  • Sign In

kubernetes-sigs / inference-perf / 43955908636
87%

Build:
DEFAULT BRANCH: main
Ran 16 Nov 2026 09:15PM UTC
Jobs 1
Files 143
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

16 Nov 2026 09:02PM UTC coverage: 86.722% (+0.2%) from 86.553%
43955908636

push

github

web-flow
feat(datagen): add support for replaying TraceLab agentic traces (#857)

## Summary
Issue kubernetes-sigs/inference-perf#813 asks for replaying the UW
TraceLab coding-agent traces
(Claude Code / Codex, https://tracelab.cs.washington.edu,
https://arxiv.org/abs/2606.30560) in
inference-perf, matching the replay support that already exists in
AIPerf. This PR adds a new
`data.type: tracelab_trace_replay` end to end: config, generator,
factory wiring, docs, CLI-flags sync,
and tests.

## Root cause
There was no TraceLab loader or generator anywhere in the tree:
- `DataGenType` (`inference_perf/config/datagen/config.py:31`) only knew
`otel_trace_replay` /
  `weka_trace_replay`, so a TraceLab config failed validation
- `main_cli` (`inference_perf/main.py`) had no factory branch,
mp-manager, tokenizer-gate, or
  session-metrics wiring for it
- `inference_perf/datagen/replay/` only shipped Conversation / OTel /
Weka generators

## Fix
- New `TraceLabTraceReplayDataGenerator`
(`inference_perf/datagen/replay/tracelab_trace_replay_datagen.py:123`,
`ReplayGraphSessionGeneratorBase` subclass, same runtime as OTel/Weka
replay): rows are grouped by
`session_id`, ordered by `round_index`, and each round becomes one LLM
call in a linear session chain
built via `build_graph`. Edges exist for ordering and wait timing, but
each prompt is the
offline-reconstructed one, so live model output is never substituted
into later prompts.
- The public trace is sanitized (no raw prompts/completions/tool
payloads), so prompts are
deterministically reconstructed to the recorded `input_tokens_total` /
`newly_append_tokens` from
corpus slices (`_text_for_tokens`,
`tracelab_trace_replay_datagen.py:321`), outputs are sized to
`output_tokens`, and inter-round tool latency (sum of
`tools[].tool_wall_latency_ms`) is preserved as
event spacing, capped by `trace_idle_gap_cap_seconds` (or zeroed by
`ignore_trace_delays`, which
  overrides the cap).
- New `TraceLabTraceReplayConfig`
(`inference_... (continued)

451 of 491 new or added lines in 5 files covered. (91.85%)

14362 of 16561 relevant lines covered (86.72%)

0.87 hits per line

Uncovered Changes

Lines Coverage ∆ File
38
91.83
inference_perf/datagen/replay/tracelab_trace_replay_datagen.py
2
19.76
-0.16% inference_perf/main.py
Jobs
ID Job ID Ran Files Coverage
1 43955908636.1 16 Nov 2026 09:15PM UTC 143
86.72
GitHub Action Run
Source Files on build 43955908636
  • Tree
  • List 143
  • Changed 6
  • Source Changed 4
  • Coverage Changed 6
Coverage ∆ File Lines Relevant Covered Missed Hits/Line
  • Back to Repo
  • Github Actions Build #43955908636
  • 111d0dce on github
  • Prev Build on main (#43937092050)
  • Next Build on main (#43961807348)
  • Delete
STATUS · Troubleshooting · Open an Issue · Sales · Support · CAREERS · ENTERPRISE · START FREE TRIAL · SCHEDULE DEMO
ANNOUNCEMENTS · TWITTER · TOS & SLA · Supported CI Services · What's a CI service? · Automated Testing

© 2026 Coveralls, Inc