agentjido / req_llm / f6963a1d775a56ee52bf65890f7a66301d42e765

49%

Build Type

push

github

Committed by web-flow

Commit Message

Add Anthropic prompt caching script and related functionality (#132)

- Introduced a new script `anthropic_prompt_caching.exs` to demonstrate the use of prompt caching with Anthropic models, including options for model selection, cache TTL, and logging levels.
- Enhanced the `ReqLLM.Providers.Anthropic` module to support prompt caching, including the addition of `anthropic_prompt_cache` and `anthropic_prompt_cache_ttl` options.
- Updated the request preparation process to inject cache control metadata into tools and system messages.
- Implemented usage tracking for cached tokens in the response handling, allowing for cost savings analysis.
- Added comprehensive unit tests for the new caching functionality, covering various scenarios including header injection and cache control handling.

These changes improve the efficiency and cost-effectiveness of using Anthropic models by leveraging prompt caching capabilities.

Run Details

34 of 49 new or added lines in 4 files covered. (69.39%)

4 existing lines in 3 files now uncovered.

3617 of 6912 relevant lines covered (52.33%)

435.16 hits per line