dcdpr / jp / 24366146360

67%

push

github

fix(llm, anthropic): Add forced-tool fallback for reasoning models (#549)

Anthropic's API does not allow forced `tool_choice` when extended
thinking is enabled. The previous behaviour silently downgraded to
`auto` + a system prompt nudge, but offered no guarantee that the model
would actually call the required tool.

This commit adds a `ForcedToolFallback` mechanism. When a reasoning
model is asked to call a specific tool, `create_request` still
downgrades `tool_choice` to `auto` for the initial request, but now
captures the original forced choice in a `ForcedToolFallback` value. The
`call()` function tracks which tool names were invoked during streaming.
If the model finishes without satisfying the constraint,
`force_tool_retry` is invoked: it appends the assistant's response as a
message, adds a user message instructing the model to call the required
tool, then re-issues the request with thinking disabled and the original
forced `tool_choice` restored.

The retry path passes `None` as its own fallback to prevent infinite
retry loops. Reasoning content emitted by a (unexpected) thinking block
in the retry response is silently dropped.

---------

Signed-off-by: Jean Mertz <git@jeanmertz.com>

65 of 121 new or added lines in 1 file covered. (53.72%)

21663 of 33160 relevant lines covered (65.33%)

197.65 hits per line