stacklok / toolhive / 27712366822

push

github

Wire the tool optimizer onto the Serve path (#5543)

* Wire the tool optimizer onto the Serve path

Implements changes for issue #5538:
- Build a per-session optimizer over core.ListTools on the Serve path
  and advertise only find_tool/call_tool (sourced from the core)
- Route call_tool's inner invocation through core.CallTool by its real
  name, closing the deferred inner-target admission gap
- Skip the session factory's optimizer decorator on the Serve path
  (FactoryConfig.AdvertiseFromCore) to avoid double-indexing the store
- Surface the resolved optimizer factory via Manager.OptimizerFactory;
  store/cleanup ownership stays in sessionmanager

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Guard nil find_tool output and document optimizer re-index cost

Addresses stacklok/toolhive#5543 review comments:
- MEDIUM serve_optimizer.go (3428801576): add the output == nil guard to
  optimizerFindToolHandler for parity with the legacy/sibling handlers, so a
  nil result cannot marshal to "null" and surface as a success
- LOW serve_optimizer.go (3428801603): document that optimizerSessionTools
  re-upserts on every registration and cross-pod rehydration (idempotent by
  PK, repeated work not a leak; optimization deferred to #5445)

* Strengthen Serve optimizer tests

Addresses stacklok/toolhive#5543 review comments:
- LOW serve_optimizer_test.go (3428801613): assert find_tool returns exactly
  the core's advertised tool names by decoding into FindToolOutput, instead of
  substring-matching the whole marshalled body; note ranking is out of scope
- LOW serve_optimizer.go (3428801607): add a test that optimizerToolHandler
  rejects an unknown meta-tool name, locking in the defensive default branch
- LOW serve_optimizer_test.go (3428801618): assert the cross-pod rehydration
  re-aggregates via a fresh core.ListTools and rebuilds the optimizer (the
  half of AC5 the test did not previously prove)

* Enforce the AdvertiseFromCore doubl... (continued)

139 of 167 new or added lines in 7 files covered. (83.23%)

69179 of 103170 relevant lines covered (67.05%)

64.06 hits per line