agentjido / req_llm

53%

main: 49%

LAST BUILD ON BRANCH feature/bedrock-prompt-caching

branch: feature/bedrock-prompt-caching

CHANGE BRANCH

x

Reset

Sync Branches

• feature/bedrock-prompt-caching
• add-bedrock-fixtures
• add-bedrock-structured-output
• add-hex-changelog-and-module-grouping
• add-json-schema-validation
• add-vertex-guide-to-docs
• add-vllm-provider
• add-vllm-support
• add_openai_responses_api_structured_responses
• add_retry_step
• allow_json_schemas
• bedrock-1.0-fixes
• bedrock-clean
• bedrock-mistral-support
• breaking/reqllm-llmdb-forced
• bug/api-return-types
• bug/codec-tool-calls
• bug/debug-stream-return
• bug/incorrect-model-spec-docs
• bug/object-array
• bug/openai-tool-calls
• bug/stream-process-linking
• bug/streaming-nil-deltas
• bug/streaming-race-condition
• bug/usage_total_cost
• bw-openai-responses-api-reasoning-effort
• cerebras
• cerebras-zai-glm-4-6
• chore/2025-10-14-update-fixtures
• chore/docs-review
• chore/model_update_2025-10-27
• chore/object-fixtures
• chore/object-fixtures-resurrected
• chore/refine-fixtures
• chore/refresh-coverage-tests
• chore/refresh-fixtures-before-1.0
• chore/update-models-2025-09-21
• config_base_url_override
• copilot/fix-32
• credo-config
• credo/code-readability-fix
• credo/fix-refactor-enum-filter
• dependabot/hex/ex_aws_auth-1.3.1
• dependabot/hex/ex_doc-0.39.1
• dependabot/hex/jsv-0.11.5
• dependabot/hex/req-0.5.16
• dependabot/hex/tidewave-0.5.1
• dependabot/hex/zoi-0.8.1
• dependabot/hex/zoi-0.8.4
• devtools
• docs/agent-tutorial
• docs/aws-event-stream-specialization
• egomes/fix-claude-multi-turn
• egomes/fix-tool-inspection-with-json-schema
• feat/bedrock-service-tiers
• feat/context-json-serialization
• feat/fixture-credential-fallback
• feat/fixture-credential-fallback-rebased
• feat/google-context-caching
• feat/google-upload-file
• feat/in-type-support
• feat/oauth2-token-cache
• feat/structured-output-openai-google
• feat/vertex-gemini-support
• feature/anthropic-structured-output
• feature/base-url-override
• feature/cerebras-provider
• feature/configurable-metadata-timeout
• feature/custom_providers
• feature/google_grounding
• feature/model-catalog
• feature/normalize-bedrock-inference-profiles
• feature/pre-release-fixes
• feature/prompt-caching
• feature/refactor-llm-api-fixtures
• feature/refined-key-management
• feature/stream-collectors
• feature/unique-model-provider-options
• feature/upgrade-ex-aws-auth
• feature/zai-fixtures
• feature/zoi-schema
• fix-anthropic-streaming
• fix-bedrock-timeout
• fix-duplicate-clause
• fix-google
• fix-groq-stream-error
• fix-mix-task-docs
• fix-openai-max-tokens-param
• fix-reasoning-overlay-pattern-match
• fix-response-process-stream
• fix-retry-delay-conflict
• fix-schema-boolean-property
• fix/bedrock-claude-exclusions
• fix/bedrock-inference-profiles
• fix/bug-119-aws-auth-credentials
• fix/capability-structure-pattern-matches
• fix/cost-calculation-in-usage
• fix/google-cached-tokens
• fix/google-file-support
• fix/google-structured-output
• fix/groq-utf8-streaming
• fix/http2-large-request-bodies
• fix/issue-65-http-status-validation
• fix/issue-96-validation-error-fields
• fix/jsv-preserve-original-data
• fix/proxy-options
• fix/registry-get-provider-nil-module
• fix/tool-budget-pattern-match
• fix/tool_calls
• fix/vertex-decode-stream-event
• google-vision
• idempotent-stream-response
• improve-metadata-provider-errors
• list_with_subtype
• main
• patch-1
• put-max-tokens-model-options
• record-bedrock-vertex-fixtures
• refactor-meta-provider-generic
• refactor/context-tools
• refactor/rename-decode-sse-event
• refactor/req-streaming
• refactor/xai-structured-objects
• remove-duplicate-google-provider-metadata
• remove-jido-keys
• response-context-append-bug
• support-openai-tool-choice-required
• vertex-gemini-rebased
• zai
• zw/fix-stream-finish-reason

Build # a8fbd60c537acd0358fb1871ca7a55d1726c1e4f-PR-138

Build Type

Pull #138

github

Committed by neilberkman

Commit Message

Add Bedrock prompt caching support

Extends Anthropic prompt caching to AWS Bedrock provider by reusing
native Anthropic caching functions and adding smart API selection.

Changes:
- Made Anthropic caching functions public for reuse
- Added tools support to Bedrock Anthropic formatter
- Auto-switch to native API when caching enabled with tools
- Added comprehensive tests for auto-switching behavior

Bedrock Converse API only caches entire system prompts, while native
API supports granular cache control on system, tools, and messages.
When both caching and tools are present, automatically use native API
with a warning that can be silenced via explicit use_converse setting.

Tested live with AWS credentials showing 90% cost savings and cache
hit behavior working correctly with tools.

Pull Request Pull Request #138: Add Bedrock Anthropic prompt caching support

Run Details

22 of 22 new or added lines in 2 files covered. (100.0%)

3640 of 6924 relevant lines covered (52.57%)

435.37 hits per line