dcdpr / jp / 26806547805

67%

push

github

fix(llm, anthropic): Don't abort or break long tool runs (#717)

Long agentic conversations against Anthropic models could fail in two
ways. The idle timeout aborted streaming mid-tool-call whenever the
model spent a long time emitting a large argument (Anthropic sends one
key/value at a time, with silent gaps between events), and the stream
then retried until the budget was exhausted. Separately, once a turn was
resumed or replayed, the rebuilt request re-sent native `thinking`
blocks whose signatures no longer matched a single generation, so the
API rejected it with `thinking blocks ... cannot be modified` and the
conversation became permanently unsendable.

The Anthropic provider now injects a synthetic `KeepAlive` every five
seconds while a tool call is mid-stream, via the new
`with_tool_call_keepalive` combinator, and surfaces real `ping` events
as `KeepAlive` as well. The generic `with_idle_timeout` stays
provider-agnostic, so a genuine stall on any other provider is still
surfaced as a timeout.

For resumed turns, the request builder rewrites `thinking` blocks in the
trailing continuation-target assistant message as `<think>` text
(reusing the shared `think_tag_text` fallback) and drops redacted
thinking. Mid-turn thinking that precedes a tool result stays native, as
the API requires. This also heals an already-wedged conversation on the
next request, without rewriting any stored events.

The default `stream_idle_timeout_secs` is raised from 30 to 60 seconds,
and a new `Validator` trait (implemented across the resolved config
tree) rejects non-zero values below ten, keeping the five-second
keepalive comfortably below the idle window.

Finally, `peek_partial_content` returns only partial message text, so an
interrupted or retried turn no longer commits partial reasoning as
assistant answer text.

BREAKING CHANGE: `assistant.request.stream_idle_timeout_secs` rejects
values between 1 and 9.

Use `0` to disable the idle timeout, or a value of `10` ... (continued)

90 of 99 new or added lines in 12 files covered. (90.91%)

33406 of 50224 relevant lines covered (66.51%)

264.04 hits per line