stacklok / toolhive / 39439729051

73%

Build Type

push

github

Committed by web-flow

Commit Message

Enable Claude Code gateway model discovery (#6862)

* Enable Claude Code gateway model discovery

Claude Code's /model picker went stale against an LLM gateway: setup
pinned the three Bedrock tiers to model IDs hardcoded in pkg/llm, so a
gateway that added or retired a model forced users to re-run setup.
Customers reported this as a recurring annoyance.

Claude Code can discover a gateway's models itself (GET /v1/models on
each startup), and the gateway now serves the Anthropic models list, so:

- Set CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1 for Claude Code so new
  models the gateway advertises appear in the picker without re-running
  setup.
- Replace the hardcoded Bedrock tier defaults with per-family selection
  from the gateway's model list: for haiku/opus/sonnet/fable, pin the
  newest Bedrock-form ID, ranked by the version parsed from the ID (the
  gateway's created_at is identical across models and unusable). A
  --models entry still overrides its tier; an unpinned tier is left for
  Claude Code to resolve.
- Add a fourth tier (fable) and its ANTHROPIC_DEFAULT_FABLE_MODEL key.
- Warn, and leave tiers unpinned, when discovery fails or setup ran in
  lazy mode.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Harden Bedrock tier discovery for Claude Code

Address review feedback on the gateway-derived Bedrock pins:

- Accept version-first legacy Bedrock IDs. The old regex only matched
  the family-first layout (claude-haiku-4-5), so Claude 3.x IDs such as
  us.anthropic.claude-3-5-haiku-20241022-v1:0 and the major-only
  claude-sonnet-4-20250514-v1:0 were silently dropped, leaving a tier
  unpinned on gateways that serve only those. Replace the regex with a
  tokenizing parser that handles both layouts and ranks on
  [major, minor, releaseDate].

- Refuse --lazy --bedrock-compat for Claude Code unless --models is
  passed. Lazy mode never logs in, so it cannot discover the tiers;
  rather than write an empty, broken config it now ... (continued)

Coverage Stats

138 of 151 new or added lines in 3 files covered. (91.39%)

8 existing lines in 4 files now uncovered.

88133 of 123368 relevant lines covered (71.44%)

127.21 hits per line