dcdpr / jp / 27555700404

67%

Build Type

push

github

Committed by web-flow

Commit Message

feat(llm, anthropic): Respect `CachePolicy` in Anthropic cache annotations (#753)

Previously, the Anthropic provider hardcoded `CacheControl::default()`
at every breakpoint site, meaning caching was always enabled with no way
to opt out or select a different TTL.

This change wires the configured `CachePolicy` through to every point
where a `cache_control` annotation is emitted. A new
`resolve_cache_control` helper maps the policy to the correct Anthropic
TTL: `Off` suppresses all annotations (and zeroes the cache budget),
`Short` requests the 5-minute TTL, `Long` requests the 1-hour TTL, and
`Custom(d)` rounds to the nearest supported option using 30 minutes as
the threshold.

The `cache` field docs in `RequestConfig` are also expanded to list all
accepted values and their semantics.

Fixes #716

Signed-off-by: Jean Mertz <git@jeanmertz.com>

Coverage Stats

32 of 42 new or added lines in 1 file covered. (76.19%)

1 existing line in 1 file now uncovered.

34573 of 51981 relevant lines covered (66.51%)

250.81 hits per line