lsm / neokai / 26007705881

81%

push

github

feat(web): custom endpoints settings panel + presets (#1936)

* feat(providers): add custom OpenAI-compatible endpoint support

Adds backend support for user-defined OpenAI-compatible endpoints (LM Studio,
vLLM, LiteLLM, self-hosted deployments, etc.) without requiring an external
proxy.

Each endpoint is persisted as JSON under `settings.customEndpoints` and
registered in the provider registry as `custom:<endpointId>`. Traffic is
routed through an embedded Anthropic Messages ↔ OpenAI Chat Completions
bridge that handles message translation, streaming tool_calls accumulation,
graceful image dropping when vision isn't supported, and per-model
capability gating.

Backend only — UI for endpoint management ships in a follow-up PR.

Acceptance criteria covered:
- Tool use works end-to-end via tool_calls deltas → Anthropic tool_use blocks
- Streaming works for both Anthropic and OpenAI-compatible providers
- Unsupported features (vision, tool use) are silently dropped per model
- Built-in Anthropic provider behaviour is unchanged
- Existing Ollama provider keeps working; new bridge is for generic OAI-compat

* fix(providers): bind custom-endpoint bridge to loopback + honour live thinkingModes

Addresses review feedback on #1929:

P1 — Security: the OpenAI Chat Completions bridge was binding to 0.0.0.0
because `Bun.serve({ port: 0 })` defaults the hostname. On multi-user hosts
another local user could probe ephemeral ports and reach the bridge with the
configured upstream API key. Now explicitly binds to 127.0.0.1.

P1 — Correctness: `addSessionStateOptions` resolved provider thinking mode
exclusively from the static `PROVIDER_THINKING_MODES` map, which doesn't list
`custom:*` providers and therefore defaulted to `'granular'`. That emitted
`thinking: { type: 'enabled', ... }` for endpoints whose capabilities report
`thinkingModes: 'off'`. The bridge silently dropped the field, but the SDK
could error expecting matching reasoning blocks back. Now prefers the ... (continued)

8326 of 11892 branches covered (70.01%)

Branch coverage included in aggregate %.

93 of 222 new or added lines in 6 files covered. (41.89%)

66597 of 78343 relevant lines covered (85.01%)

300.03 hits per line