shumengya/sproutclaw
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(ai): support prompt caching for Bedrock application inference profiles (#2346)

Browse Source 
Add AWS_BEDROCK_FORCE_CACHE=1 environment variable support. When the
model ID doesn't contain a recognizable Claude model name, users can
set this variable to force cache point injection.
This commit is contained in:
xu0o0
2026-03-18 18:21:10 +08:00
committed by GitHub
parent 8a8e2a8049
commit 31d59f8513
2 changed files with 20 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
packages/ai/src/providers/amazon-bedrock.ts
View File

@@ -430,10 +430,22 @@ function resolveCacheRetention(cacheRetention?: CacheRetention): CacheRetention
/** /**
* Check if the model supports prompt caching. * Check if the model supports prompt caching.
* Supported: Claude 3.5 Haiku, Claude 3.7 Sonnet, Claude 4.x models * Supported: Claude 3.5 Haiku, Claude 3.7 Sonnet, Claude 4.x models
*
* For base models and system-defined inference profiles the model ID / ARN
* contains the model name, so we can decide locally.
*
* For application inference profiles (whose ARNs don't contain the model name),
* set AWS_BEDROCK_FORCE_CACHE=1 to enable cache points. Amazon Nova models
* have automatic caching and don't need explicit cache points.
*/ */
function supportsPromptCaching(model: Model<"bedrock-converse-stream">): boolean { function supportsPromptCaching(model: Model<"bedrock-converse-stream">): boolean {
const id = model.id.toLowerCase(); const id = model.id.toLowerCase();
if (!id.includes("claude")) return false; if (!id.includes("claude")) {
// Application inference profiles don't contain the model name in the ARN.
// Allow users to force cache points via environment variable.
if (typeof process !== "undefined" && process.env.AWS_BEDROCK_FORCE_CACHE === "1") return true;
return false;
}
// Claude 4.x models (opus-4, sonnet-4, haiku-4) // Claude 4.x models (opus-4, sonnet-4, haiku-4)
if (id.includes("-4-") || id.includes("-4.")) return true; if (id.includes("-4-") || id.includes("-4.")) return true;
// Claude 3.7 Sonnet // Claude 3.7 Sonnet

7
packages/coding-agent/docs/providers.md
View File

@@ -147,6 +147,13 @@ Also supports ECS task roles (`AWS_CONTAINER_CREDENTIALS_*`) and IRSA (`AWS_WEB_
pi --provider amazon-bedrock --model us.anthropic.claude-sonnet-4-20250514-v1:0 pi --provider amazon-bedrock --model us.anthropic.claude-sonnet-4-20250514-v1:0
``` ```
Prompt caching is enabled automatically for Claude models whose ID contains a recognizable model name (base models and system-defined inference profiles). For application inference profiles (whose ARNs don't contain the model name), set `AWS_BEDROCK_FORCE_CACHE=1` to enable cache points:
```bash
export AWS_BEDROCK_FORCE_CACHE=1
pi --provider amazon-bedrock --model arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/abc123
```
If you are connecting to a Bedrock API proxy, the following environment variables can be used: If you are connecting to a Bedrock API proxy, the following environment variables can be used:
```bash ```bash