agentjido / req_llm / a377c25d71d9e982bbd2eb2a57ebb8403c95b7f1-PR-132

52%

main: 49%

Build Type

Pull #132

github

Committed by mikehostetler

Commit Message

Add Anthropic prompt caching script and related functionality

- Introduced a new script `anthropic_prompt_caching.exs` to demonstrate the use of prompt caching with Anthropic models, including options for model selection, cache TTL, and logging levels.
- Enhanced the `ReqLLM.Providers.Anthropic` module to support prompt caching, including the addition of `anthropic_prompt_cache` and `anthropic_prompt_cache_ttl` options.
- Updated the request preparation process to inject cache control metadata into tools and system messages.
- Implemented usage tracking for cached tokens in the response handling, allowing for cost savings analysis.
- Added comprehensive unit tests for the new caching functionality, covering various scenarios including header injection and cache control handling.

These changes improve the efficiency and cost-effectiveness of using Anthropic models by leveraging prompt caching capabilities.

Pull Request Pull Request #132: Add Anthropic prompt caching support

Run Details

34 of 49 new or added lines in 4 files covered. (69.39%)

4 existing lines in 3 files now uncovered.

3608 of 6904 relevant lines covered (52.26%)

434.68 hits per line