kubernetes-sigs / inference-perf / 39638850955

85%

push

github

fix(synthetic_agentic): wire theme system prompt, rotate tools, vary filler (#907)

#### What type of PR is this?
/kind bug

#### What this PR does / why we need it
Fixes three realism issues in the synthetic agentic workload generator:

1. **Theme `system_prompt` is now used.** `Theme.system_prompt` was
declared (and set by the
bundled themes) but never reached `_render_system_head`, so every root
agent got a generic
`ROOT_SYSTEM_PROMPTS` head. The root agent's head now opens with the
theme's prompt when one
is set, falling back to the generic pool otherwise. Sub-agents are
unchanged and keep using
   `SUBAGENT_SYSTEM_PROMPTS`.
2. **Tool selection no longer resets each round.** The tool loop indexed
`tool_defs[(t + j) % len(tool_defs)]`, and every round builds a fresh
root agent at `t=0`,
so each round's first call was always `tool_defs[0]`. Each agent now
starts at a seeded
   offset into the catalog (new per-agent sub-seed `12`).
3. **Assistant output no longer repeats across turns.** `fit_filler`
accepted an `rng` but never
used it, and always cycled the word pool from index 0, so answers/acks
were byte-identical
between rounds. It now uses `rng` to pick a seeded start offset
(`rng=None` keeps the old
start-at-0 behavior). The system head's filler is left at a fixed start.

All changes stay deterministic: the same `(config, session_index)` still
yields a byte-identical
graph. Note that graphs generated from a given seed will differ from
those produced before this
change.

#### Which issue(s) this PR fixes
Fixes #790

#### Testing
- Updated `test_agent_first_call_carries_role_appropriate_system_head`
(root now expects the
  theme prompt; sub-agents still from `SUBAGENT_SYSTEM_PROMPTS`).
- New regression tests (fail before / pass after):
  - `test_tool_rotation_varies_across_rounds`
  - `test_answer_text_varies_across_rounds`
  - `test_fit_filler_rng_offsets_word_cycle_deterministically`
- New `test_root_head_falls_back_to_root_pool_without_theme_pr... (continued)

8 of 8 new or added lines in 1 file covered. (100.0%)

12369 of 14787 relevant lines covered (83.65%)

0.84 hits per line