agentjido / req_llm / 945b7128c4955aab6b401241ca7d785556e25a59-PR-193

49%

main: 49%

Build Type

Pull #193

github

Committed by neilberkman

Commit Message

feat: Add Google Context Caching support for Gemini models

Adds explicit context caching API for Gemini models to reduce costs by up to
90% when reusing large amounts of content.

- Add ReqLLM.Providers.Google.CachedContent module for cache CRUD operations
- Support for both Google AI Studio and Vertex AI (when Gemini support is added)
- Add cached_content provider option to reference existing caches
- Comprehensive tests for cache creation, listing, updating, and deletion
- Documentation and examples in Google provider moduledoc
- Updated CHANGELOG

```elixir
{:ok, cache} = ReqLLM.Providers.Google.CachedContent.create(
  provider: :google,
  model: "gemini-2.5-flash",
  api_key: System.get_env("GOOGLE_API_KEY"),
  contents: [%{role: "user", parts: [%{text: large_document}]}],
  ttl: "3600s"
)

{:ok, response} = ReqLLM.generate_text(
  "google:gemini-2.5-flash",
  "Question about the document?",
  provider_options: [cached_content: cache.name]
)

IO.inspect(response.usage.cached_tokens)
```

- Gemini 2.5 Flash: 1,024 minimum tokens
- Gemini 2.5 Pro: 4,096 minimum tokens

Pull Request Pull Request #193: feat: Add Google Context Caching support for Gemini models

Run Details

6 of 159 new or added lines in 2 files covered. (3.77%)

3 existing lines in 1 file now uncovered.

3594 of 7343 relevant lines covered (48.94%)

57.29 hits per line