dcdpr / jp / 18347113204

58%

Build Type

push

github

Committed by web-flow

Commit Message

refactor(llm)!: Overhaul provider system and streaming (#271)

Completely rewrites the LLM provider infrastructure with a new
streaming architecture and enhanced model management capabilities.
This change introduces breaking changes to the `Provider` trait while
adding significant new functionality.

The streaming system now uses a centralized `Accumulator` with proper
text buffering and configurable line length limits, replacing the
previous per-provider implementations. A new `stream` module provides
reusable `Delta`, `StreamEvent`, and `CompletionChunk` abstractions.

The `Provider` trait now requires a `model_details(&Name)` method and
all completion methods accept `&ModelDetails` instead of
`&ModelIdConfig`, providing richer context about model capabilities,
deprecation status, and provider-specific features.

For Anthropic specifically, this adds automatic request chaining when
models hit token limits (configurable via `chain_on_max_tokens`) and
beta features support including interleaved thinking, context editing,
and 1M context windows. The system intelligently merges responses to
avoid content duplication during chaining.

All six providers (Anthropic, Google, LlamaCpp, Ollama, OpenAI,
OpenRouter) have been updated with comprehensive model mappings that
include deprecation tracking and feature flags. The CLI now calls
`model_details()` before requests and properly handles the new event
stream structure.

---------

Signed-off-by: Jean Mertz <git@jeanmertz.com>

Run Details

814 of 1023 new or added lines in 18 files covered. (79.57%)

119 existing lines in 6 files now uncovered.

6605 of 14470 relevant lines covered (45.65%)

15.71 hits per line