shumengya/sproutclaw
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(ai): add Cloudflare AI Gateway as a provider (#3856)

Browse Source 
* feat(ai): add Cloudflare AI Gateway as a provider

Routes through Cloudflare's Unified API (`/compat`) for Workers AI and
Anthropic models, and through the provider-specific `/openai` subpath
for OpenAI models so reasoning models (gpt-5.x, o-series) can hit
`/v1/responses` natively. Once `/compat` adds Responses-API support,
the OpenAI subpath can be folded back in.

Catalog layout:
  workers-ai/@cf/...  -> openai-completions, gateway/.../compat
  anthropic/...       -> openai-completions, gateway/.../compat
  <native-id>         -> openai-responses,   gateway/.../openai
                         (gpt-5.1, claude-... no, sorry: gpt-5.x and o-series only;
                          prefix stripped because the OpenAI SDK posts native ids)

Touches:
  packages/ai/src/types.ts                       add cloudflare-ai-gateway to KnownProvider
  packages/ai/src/env-api-keys.ts                map to CLOUDFLARE_API_KEY
  packages/ai/src/providers/cloudflare.ts        add CLOUDFLARE_AI_GATEWAY_COMPAT_BASE_URL
                                                 and CLOUDFLARE_AI_GATEWAY_OPENAI_BASE_URL
  packages/ai/src/providers/openai-responses.ts  one-line dispatch through resolveCloudflareBaseUrl
                                                 (matches what openai-completions.ts already does)
  packages/ai/scripts/generate-models.ts         branch openai/* vs workers-ai/anthropic/*
  packages/ai/src/models.generated.ts            spliced 34 entries
  packages/ai/test/stream.test.ts                3 e2e blocks (one per upstream)
  packages/coding-agent/*                        defaultModelPerProvider, login, env docs,
                                                 README, providers.md

Verified end-to-end against a real Cloudflare account with unified
billing: 9/9 e2e tests pass across all three upstreams (Workers AI
Kimi K2.6, OpenAI gpt-5.1 reasoning, Anthropic claude-sonnet-4-5).

* refactor(ai): move AI Gateway User-Agent and per-route session-affinity flag to catalog

Mirrors the same per-model metadata refactor done for Workers AI in the
parent branch. All cloudflare-ai-gateway entries get the User-Agent
header. Only workers-ai/* gateway entries set
`compat.sendSessionAffinityHeaders: true` because the gateway
forwards that header to the underlying Workers AI runtime; anthropic/*
upstream and openai/* (openai-responses) don't use it.

  packages/ai/scripts/generate-models.ts: emit headers (always) and
  per-upstream compat (workers-ai only) on each cloudflare-ai-gateway
  entry.
  packages/ai/src/models.generated.ts: re-spliced 35 entries with
  headers + conditional compat.

Behavior unchanged - 9/9 e2e tests pass across all three upstream
families.

* fix(ai): align AI Gateway with telemetry-aware UA helper

Adapts to badlogic/pi-mono#3851's follow-up fix ("honor telemetry for
Cloudflare attribution headers", fbb5eed) which moved the
'User-Agent: pi-coding-agent' header out of per-model catalog metadata
and into a centralized telemetry-honoring helper
(coding-agent/src/core/sdk.ts:getAttributionHeaders).

- packages/coding-agent/src/core/sdk.ts: extend the cloudflare branch of
  getAttributionHeaders to also match cloudflare-ai-gateway and
  gateway.ai.cloudflare.com.

- packages/ai/scripts/generate-models.ts and src/models.generated.ts:
  drop 'headers' from the 35 cloudflare-ai-gateway entries (constant
  CLOUDFLARE_STATIC_HEADERS no longer exists). Per-route
  compat.sendSessionAffinityHeaders is unchanged.

End-to-end behavior unchanged: 9/9 tests still pass across all three
upstream families (Workers AI, Anthropic, OpenAI Responses).

---------

Co-authored-by: Mario Zechner <badlogicgames@gmail.com>
This commit is contained in:
MC
2026-04-30 22:29:37 +01:00
committed by GitHub
parent c9ddca1968
commit 24fb6b833b
13 changed files with 755 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
packages/ai/README.md
View File

@@ -57,6 +57,7 @@ Unified LLM API with automatic model discovery, provider configuration, token an
- **Mistral**
- **Groq**
- **Cerebras**
- **Cloudflare AI Gateway**
- **Cloudflare Workers AI**
- **xAI**
- **OpenRouter**
@@ -1026,6 +1027,7 @@ In Node.js environments, you can set environment variables to avoid passing API
| Mistral | `MISTRAL_API_KEY` |
| Groq | `GROQ_API_KEY` |
| Cerebras | `CEREBRAS_API_KEY` |
| Cloudflare AI Gateway | `CLOUDFLARE_API_KEY` + `CLOUDFLARE_ACCOUNT_ID` + `CLOUDFLARE_GATEWAY_ID` |
| Cloudflare Workers AI | `CLOUDFLARE_API_KEY` + `CLOUDFLARE_ACCOUNT_ID` |
| xAI | `XAI_API_KEY` |
| Fireworks | `FIREWORKS_API_KEY` |

57
packages/ai/scripts/generate-models.ts
View File

@@ -3,7 +3,11 @@
import { writeFileSync } from "fs";
import { join, dirname } from "path";
import { fileURLToPath } from "url";
import { CLOUDFLARE_WORKERS_AI_BASE_URL } from "../src/providers/cloudflare.js";
import {
CLOUDFLARE_AI_GATEWAY_COMPAT_BASE_URL,
CLOUDFLARE_AI_GATEWAY_OPENAI_BASE_URL,
CLOUDFLARE_WORKERS_AI_BASE_URL,
} from "../src/providers/cloudflare.js";
import {
Api,
type AnthropicMessagesCompat,
@@ -404,6 +408,57 @@ async function loadModelsDevData(): Promise<Model<any>[]> {
}
}
// Process Cloudflare AI Gateway models
if (data["cloudflare-ai-gateway"]?.models) {
for (const [prefixedId, model] of Object.entries(data["cloudflare-ai-gateway"].models)) {
const m = model as ModelsDevModel;
if (m.tool_call !== true) continue;
const slashIdx = prefixedId.indexOf("/");
if (slashIdx === -1) continue;
const upstream = prefixedId.slice(0, slashIdx);
const nativeId = prefixedId.slice(slashIdx + 1);
let api: "openai-completions" | "openai-responses";
let baseUrl: string;
let id: string;
if (upstream === "openai") {
api = "openai-responses";
baseUrl = CLOUDFLARE_AI_GATEWAY_OPENAI_BASE_URL;
id = nativeId;
} else if (upstream === "workers-ai" || upstream === "anthropic") {
api = "openai-completions";
baseUrl = CLOUDFLARE_AI_GATEWAY_COMPAT_BASE_URL;
id = prefixedId;
} else {
continue;
}
// workers-ai/* through the gateway forwards x-session-affinity to
// the underlying Workers AI runtime for prefix-cache routing.
const compat = upstream === "workers-ai" ? { sendSessionAffinityHeaders: true } : undefined;
models.push({
id,
name: m.name || id,
api,
provider: "cloudflare-ai-gateway",
baseUrl,
reasoning: m.reasoning === true,
input: m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"],
cost: {
input: m.cost?.input || 0,
output: m.cost?.output || 0,
cacheRead: m.cost?.cache_read || 0,
cacheWrite: m.cost?.cache_write || 0,
},
contextWindow: m.limit?.context || 4096,
maxTokens: m.limit?.output || 4096,
...(compat ? { compat } : {}),
});
}
}
// Process xAi models
if (data.xai?.models) {
for (const [modelId, model] of Object.entries(data.xai.models)) {

1
packages/ai/src/env-api-keys.ts
View File

@@ -121,6 +121,7 @@ function getApiKeyEnvVars(provider: string): readonly string[] | undefined {
"opencode-go": "OPENCODE_API_KEY",
"kimi-coding": "KIMI_API_KEY",
"cloudflare-workers-ai": "CLOUDFLARE_API_KEY",
"cloudflare-ai-gateway": "CLOUDFLARE_API_KEY",
};
const envVar = envMap[provider];

601
packages/ai/src/models.generated.ts
View File

@@ -15654,4 +15654,605 @@ export const MODELS = {
maxTokens: 131072,
} satisfies Model<"openai-completions">,
},
"cloudflare-ai-gateway": {
"anthropic/claude-3-5-haiku": {
id: "anthropic/claude-3-5-haiku",
name: "Claude Haiku 3.5 (latest)",
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
reasoning: false,
input: ["text", "image"],
cost: {
input: 0.8,
output: 4,
cacheRead: 0.08,
cacheWrite: 1,
},
contextWindow: 200000,
maxTokens: 8192,
} satisfies Model<"openai-completions">,
"anthropic/claude-3-haiku": {
id: "anthropic/claude-3-haiku",
name: "Claude Haiku 3",
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
reasoning: false,
input: ["text", "image"],
cost: {
input: 0.25,
output: 1.25,
cacheRead: 0.03,
cacheWrite: 0.3,
},
contextWindow: 200000,
maxTokens: 4096,
} satisfies Model<"openai-completions">,
"anthropic/claude-3-opus": {
id: "anthropic/claude-3-opus",
name: "Claude Opus 3",
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
reasoning: false,
input: ["text", "image"],
cost: {
input: 15,
output: 75,
cacheRead: 1.5,
cacheWrite: 18.75,
},
contextWindow: 200000,
maxTokens: 4096,
} satisfies Model<"openai-completions">,
"anthropic/claude-3-sonnet": {
id: "anthropic/claude-3-sonnet",
name: "Claude Sonnet 3",
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
reasoning: false,
input: ["text", "image"],
cost: {
input: 3,
output: 15,
cacheRead: 0.3,
cacheWrite: 0.3,
},
contextWindow: 200000,
maxTokens: 4096,
} satisfies Model<"openai-completions">,
"anthropic/claude-3.5-haiku": {
id: "anthropic/claude-3.5-haiku",
name: "Claude Haiku 3.5 (latest)",
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
reasoning: false,
input: ["text", "image"],
cost: {
input: 0.8,
output: 4,
cacheRead: 0.08,
cacheWrite: 1,
},
contextWindow: 200000,
maxTokens: 8192,
} satisfies Model<"openai-completions">,
"anthropic/claude-3.5-sonnet": {
id: "anthropic/claude-3.5-sonnet",
name: "Claude Sonnet 3.5 v2",
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
reasoning: false,
input: ["text", "image"],
cost: {
input: 3,
output: 15,
cacheRead: 0.3,
cacheWrite: 3.75,
},
contextWindow: 200000,
maxTokens: 8192,
} satisfies Model<"openai-completions">,
"anthropic/claude-haiku-4-5": {
id: "anthropic/claude-haiku-4-5",
name: "Claude Haiku 4.5 (latest)",
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
reasoning: true,
input: ["text", "image"],
cost: {
input: 1,
output: 5,
cacheRead: 0.1,
cacheWrite: 1.25,
},
contextWindow: 200000,
maxTokens: 64000,
} satisfies Model<"openai-completions">,
"anthropic/claude-opus-4": {
id: "anthropic/claude-opus-4",
name: "Claude Opus 4 (latest)",
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
reasoning: true,
input: ["text", "image"],
cost: {
input: 15,
output: 75,
cacheRead: 1.5,
cacheWrite: 18.75,
},
contextWindow: 200000,
maxTokens: 32000,
} satisfies Model<"openai-completions">,
"anthropic/claude-opus-4-1": {
id: "anthropic/claude-opus-4-1",
name: "Claude Opus 4.1 (latest)",
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
reasoning: true,
input: ["text", "image"],
cost: {
input: 15,
output: 75,
cacheRead: 1.5,
cacheWrite: 18.75,
},
contextWindow: 200000,
maxTokens: 32000,
} satisfies Model<"openai-completions">,
"anthropic/claude-opus-4-5": {
id: "anthropic/claude-opus-4-5",
name: "Claude Opus 4.5 (latest)",
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
reasoning: true,
input: ["text", "image"],
cost: {
input: 5,
output: 25,
cacheRead: 0.5,
cacheWrite: 6.25,
},
contextWindow: 200000,
maxTokens: 64000,
} satisfies Model<"openai-completions">,
"anthropic/claude-opus-4-6": {
id: "anthropic/claude-opus-4-6",
name: "Claude Opus 4.6 (latest)",
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
reasoning: true,
input: ["text", "image"],
cost: {
input: 5,
output: 25,
cacheRead: 0.5,
cacheWrite: 6.25,
},
contextWindow: 1000000,
maxTokens: 128000,
} satisfies Model<"openai-completions">,
"anthropic/claude-opus-4-7": {
id: "anthropic/claude-opus-4-7",
name: "Claude Opus 4.7",
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
reasoning: true,
input: ["text", "image"],
cost: {
input: 5,
output: 25,
cacheRead: 0.5,
cacheWrite: 6.25,
},
contextWindow: 1000000,
maxTokens: 128000,
} satisfies Model<"openai-completions">,
"anthropic/claude-sonnet-4": {
id: "anthropic/claude-sonnet-4",
name: "Claude Sonnet 4 (latest)",
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
reasoning: true,
input: ["text", "image"],
cost: {
input: 3,
output: 15,
cacheRead: 0.3,
cacheWrite: 3.75,
},
contextWindow: 200000,
maxTokens: 64000,
} satisfies Model<"openai-completions">,
"anthropic/claude-sonnet-4-5": {
id: "anthropic/claude-sonnet-4-5",
name: "Claude Sonnet 4.5 (latest)",
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
reasoning: true,
input: ["text", "image"],
cost: {
input: 3,
output: 15,
cacheRead: 0.3,
cacheWrite: 3.75,
},
contextWindow: 200000,
maxTokens: 64000,
} satisfies Model<"openai-completions">,
"anthropic/claude-sonnet-4-6": {
id: "anthropic/claude-sonnet-4-6",
name: "Claude Sonnet 4.6",
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
reasoning: true,
input: ["text", "image"],
cost: {
input: 3,
output: 15,
cacheRead: 0.3,
cacheWrite: 3.75,
},
contextWindow: 1000000,
maxTokens: 64000,
} satisfies Model<"openai-completions">,
"workers-ai/@cf/moonshotai/kimi-k2.5": {
id: "workers-ai/@cf/moonshotai/kimi-k2.5",
name: "Kimi K2.5",
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
compat: {"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.6,
output: 3,
cacheRead: 0.1,
cacheWrite: 0,
},
contextWindow: 256000,
maxTokens: 256000,
} satisfies Model<"openai-completions">,
"workers-ai/@cf/moonshotai/kimi-k2.6": {
id: "workers-ai/@cf/moonshotai/kimi-k2.6",
name: "Kimi K2.6",
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
compat: {"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text", "image"],
cost: {
input: 0.95,
output: 4,
cacheRead: 0.16,
cacheWrite: 0,
},
contextWindow: 256000,
maxTokens: 256000,
} satisfies Model<"openai-completions">,
"workers-ai/@cf/nvidia/nemotron-3-120b-a12b": {
id: "workers-ai/@cf/nvidia/nemotron-3-120b-a12b",
name: "Nemotron 3 Super 120B",
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
compat: {"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text"],
cost: {
input: 0.5,
output: 1.5,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 256000,
maxTokens: 256000,
} satisfies Model<"openai-completions">,
"workers-ai/@cf/zai-org/glm-4.7-flash": {
id: "workers-ai/@cf/zai-org/glm-4.7-flash",
name: "GLM-4.7-Flash",
api: "openai-completions",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat",
compat: {"sendSessionAffinityHeaders":true},
reasoning: true,
input: ["text"],
cost: {
input: 0.06,
output: 0.4,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 131072,
maxTokens: 131072,
} satisfies Model<"openai-completions">,
"gpt-4": {
id: "gpt-4",
name: "GPT-4",
api: "openai-responses",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/openai",
reasoning: false,
input: ["text"],
cost: {
input: 30,
output: 60,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 8192,
maxTokens: 8192,
} satisfies Model<"openai-responses">,
"gpt-4-turbo": {
id: "gpt-4-turbo",
name: "GPT-4 Turbo",
api: "openai-responses",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/openai",
reasoning: false,
input: ["text", "image"],
cost: {
input: 10,
output: 30,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 128000,
maxTokens: 4096,
} satisfies Model<"openai-responses">,
"gpt-4o": {
id: "gpt-4o",
name: "GPT-4o",
api: "openai-responses",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/openai",
reasoning: false,
input: ["text", "image"],
cost: {
input: 2.5,
output: 10,
cacheRead: 1.25,
cacheWrite: 0,
},
contextWindow: 128000,
maxTokens: 16384,
} satisfies Model<"openai-responses">,
"gpt-4o-mini": {
id: "gpt-4o-mini",
name: "GPT-4o mini",
api: "openai-responses",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/openai",
reasoning: false,
input: ["text", "image"],
cost: {
input: 0.15,
output: 0.6,
cacheRead: 0.08,
cacheWrite: 0,
},
contextWindow: 128000,
maxTokens: 16384,
} satisfies Model<"openai-responses">,
"gpt-5.1": {
id: "gpt-5.1",
name: "GPT-5.1",
api: "openai-responses",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/openai",
reasoning: true,
input: ["text", "image"],
cost: {
input: 1.25,
output: 10,
cacheRead: 0.13,
cacheWrite: 0,
},
contextWindow: 400000,
maxTokens: 128000,
} satisfies Model<"openai-responses">,
"gpt-5.1-codex": {
id: "gpt-5.1-codex",
name: "GPT-5.1 Codex",
api: "openai-responses",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/openai",
reasoning: true,
input: ["text", "image"],
cost: {
input: 1.25,
output: 10,
cacheRead: 0.125,
cacheWrite: 0,
},
contextWindow: 400000,
maxTokens: 128000,
} satisfies Model<"openai-responses">,
"gpt-5.2": {
id: "gpt-5.2",
name: "GPT-5.2",
api: "openai-responses",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/openai",
reasoning: true,
input: ["text", "image"],
cost: {
input: 1.75,
output: 14,
cacheRead: 0.175,
cacheWrite: 0,
},
contextWindow: 400000,
maxTokens: 128000,
} satisfies Model<"openai-responses">,
"gpt-5.2-codex": {
id: "gpt-5.2-codex",
name: "GPT-5.2 Codex",
api: "openai-responses",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/openai",
reasoning: true,
input: ["text", "image"],
cost: {
input: 1.75,
output: 14,
cacheRead: 0.175,
cacheWrite: 0,
},
contextWindow: 400000,
maxTokens: 128000,
} satisfies Model<"openai-responses">,
"gpt-5.3-codex": {
id: "gpt-5.3-codex",
name: "GPT-5.3 Codex",
api: "openai-responses",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/openai",
reasoning: true,
input: ["text", "image"],
cost: {
input: 1.75,
output: 14,
cacheRead: 0.175,
cacheWrite: 0,
},
contextWindow: 400000,
maxTokens: 128000,
} satisfies Model<"openai-responses">,
"gpt-5.4": {
id: "gpt-5.4",
name: "GPT-5.4",
api: "openai-responses",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/openai",
reasoning: true,
input: ["text", "image"],
cost: {
input: 2.5,
output: 15,
cacheRead: 0.25,
cacheWrite: 0,
},
contextWindow: 1050000,
maxTokens: 128000,
} satisfies Model<"openai-responses">,
"gpt-5.5": {
id: "gpt-5.5",
name: "GPT-5.5",
api: "openai-responses",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/openai",
reasoning: true,
input: ["text", "image"],
cost: {
input: 5,
output: 30,
cacheRead: 0.5,
cacheWrite: 0,
},
contextWindow: 1050000,
maxTokens: 128000,
} satisfies Model<"openai-responses">,
"o1": {
id: "o1",
name: "o1",
api: "openai-responses",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/openai",
reasoning: true,
input: ["text", "image"],
cost: {
input: 15,
output: 60,
cacheRead: 7.5,
cacheWrite: 0,
},
contextWindow: 200000,
maxTokens: 100000,
} satisfies Model<"openai-responses">,
"o3": {
id: "o3",
name: "o3",
api: "openai-responses",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/openai",
reasoning: true,
input: ["text", "image"],
cost: {
input: 2,
output: 8,
cacheRead: 0.5,
cacheWrite: 0,
},
contextWindow: 200000,
maxTokens: 100000,
} satisfies Model<"openai-responses">,
"o3-mini": {
id: "o3-mini",
name: "o3-mini",
api: "openai-responses",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/openai",
reasoning: true,
input: ["text"],
cost: {
input: 1.1,
output: 4.4,
cacheRead: 0.55,
cacheWrite: 0,
},
contextWindow: 200000,
maxTokens: 100000,
} satisfies Model<"openai-responses">,
"o3-pro": {
id: "o3-pro",
name: "o3-pro",
api: "openai-responses",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/openai",
reasoning: true,
input: ["text", "image"],
cost: {
input: 20,
output: 80,
cacheRead: 0,
cacheWrite: 0,
},
contextWindow: 200000,
maxTokens: 100000,
} satisfies Model<"openai-responses">,
"o4-mini": {
id: "o4-mini",
name: "o4-mini",
api: "openai-responses",
provider: "cloudflare-ai-gateway",
baseUrl: "https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/openai",
reasoning: true,
input: ["text", "image"],
cost: {
input: 1.1,
output: 4.4,
cacheRead: 0.28,
cacheWrite: 0,
},
contextWindow: 200000,
maxTokens: 100000,
} satisfies Model<"openai-responses">,
},
} as const;

16
packages/ai/src/providers/cloudflare.ts
View File

@@ -1,15 +1,23 @@
import type { Model } from "../types.js";
import type { Api, Model } from "../types.js";
/** Workers AI endpoint. `{CLOUDFLARE_ACCOUNT_ID}` is substituted at request time. */
/** Workers AI direct endpoint. */
export const CLOUDFLARE_WORKERS_AI_BASE_URL =
"https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/v1";
/** AI Gateway Unified API. https://developers.cloudflare.com/ai-gateway/usage/unified-api/ */
export const CLOUDFLARE_AI_GATEWAY_COMPAT_BASE_URL =
"https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/compat";
/** AI Gateway → OpenAI passthrough. Used until /compat supports /v1/responses. */
export const CLOUDFLARE_AI_GATEWAY_OPENAI_BASE_URL =
"https://gateway.ai.cloudflare.com/v1/{CLOUDFLARE_ACCOUNT_ID}/{CLOUDFLARE_GATEWAY_ID}/openai";
export function isCloudflareProvider(provider: string): boolean {
return provider === "cloudflare-workers-ai";
return provider === "cloudflare-workers-ai" || provider === "cloudflare-ai-gateway";
}
/** Substitute `{VAR}` placeholders in a Cloudflare baseUrl from process.env. */
export function resolveCloudflareBaseUrl(model: Model<"openai-completions">): string {
export function resolveCloudflareBaseUrl(model: Model<Api>): string {
const url = model.baseUrl;
if (!url.includes("{")) return url;
return url.replace(/\{([A-Z_][A-Z0-9_]*)\}/g, (_match, name: string) => {

3
packages/ai/src/providers/openai-responses.ts
View File

@@ -16,6 +16,7 @@ import type {
} from "../types.js";
import { AssistantMessageEventStream } from "../utils/event-stream.js";
import { headersToRecord } from "../utils/headers.js";
import { isCloudflareProvider, resolveCloudflareBaseUrl } from "./cloudflare.js";
import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./github-copilot-headers.js";
import { convertResponsesMessages, convertResponsesTools, processResponsesStream } from "./openai-responses-shared.js";
import { buildBaseOptions, clampReasoning } from "./simple-options.js";
@@ -198,7 +199,7 @@ function createClient(
return new OpenAI({
apiKey,
baseURL: model.baseUrl,
baseURL: isCloudflareProvider(model.provider) ? resolveCloudflareBaseUrl(model) : model.baseUrl,
dangerouslyAllowBrowser: true,
defaultHeaders: headers,
});

3
packages/ai/src/types.ts
View File

@@ -41,7 +41,8 @@ export type KnownProvider =
| "opencode"
| "opencode-go"
| "kimi-coding"
| "cloudflare-workers-ai";
| "cloudflare-workers-ai"
| "cloudflare-ai-gateway";
export type Provider = KnownProvider | string;
export type ThinkingLevel = "minimal" | "low" | "medium" | "high" | "xhigh";

54
packages/ai/test/stream.test.ts
View File

@@ -641,6 +641,60 @@ describe("Generate E2E Tests", () => {
},
);
describe.skipIf(
!process.env.CLOUDFLARE_API_KEY || !process.env.CLOUDFLARE_ACCOUNT_ID || !process.env.CLOUDFLARE_GATEWAY_ID,
)("Cloudflare AI Gateway → Workers AI (Kimi K2.6 via /compat)", () => {
const llm = getModel("cloudflare-ai-gateway", "workers-ai/@cf/moonshotai/kimi-k2.6");
it("should complete basic text generation", { retry: 3 }, async () => {
await basicTextGeneration(llm);
});
it("should handle tool calling", { retry: 3 }, async () => {
await handleToolCall(llm);
});
it("should handle streaming", { retry: 3 }, async () => {
await handleStreaming(llm);
});
});
describe.skipIf(
!process.env.CLOUDFLARE_API_KEY || !process.env.CLOUDFLARE_ACCOUNT_ID || !process.env.CLOUDFLARE_GATEWAY_ID,
)("Cloudflare AI Gateway → OpenAI (gpt-5.1 via /openai responses)", () => {
const llm = getModel("cloudflare-ai-gateway", "gpt-5.1");
it("should complete basic text generation", { retry: 3 }, async () => {
await basicTextGeneration(llm);
});
it("should handle tool calling", { retry: 3 }, async () => {
await handleToolCall(llm);
});
it("should handle streaming", { retry: 3 }, async () => {
await handleStreaming(llm);
});
});
describe.skipIf(
!process.env.CLOUDFLARE_API_KEY || !process.env.CLOUDFLARE_ACCOUNT_ID || !process.env.CLOUDFLARE_GATEWAY_ID,
)("Cloudflare AI Gateway → Anthropic (claude-sonnet-4-5 via /compat)", () => {
const llm = getModel("cloudflare-ai-gateway", "anthropic/claude-sonnet-4-5");
it("should complete basic text generation", { retry: 3 }, async () => {
await basicTextGeneration(llm);
});
it("should handle tool calling", { retry: 3 }, async () => {
await handleToolCall(llm);
});
it("should handle streaming", { retry: 3 }, async () => {
await handleStreaming(llm);
});
});
describe.skipIf(!process.env.HF_TOKEN)("Hugging Face Provider (Kimi-K2.5 via OpenAI Completions)", () => {
const llm = getModel("huggingface", "moonshotai/Kimi-K2.5");

1
packages/coding-agent/README.md
View File

@@ -116,6 +116,7 @@ For each built-in provider, pi maintains a list of tool-capable models, updated
- Mistral
- Groq
- Cerebras
- Cloudflare AI Gateway
- Cloudflare Workers AI
- xAI
- OpenRouter

14
packages/coding-agent/docs/providers.md
View File

@@ -57,6 +57,7 @@ pi
| Mistral | `MISTRAL_API_KEY` | `mistral` |
| Groq | `GROQ_API_KEY` | `groq` |
| Cerebras | `CEREBRAS_API_KEY` | `cerebras` |
| Cloudflare AI Gateway | `CLOUDFLARE_API_KEY` (+ `CLOUDFLARE_ACCOUNT_ID`, `CLOUDFLARE_GATEWAY_ID`) | `cloudflare-ai-gateway` |
| Cloudflare Workers AI | `CLOUDFLARE_API_KEY` (+ `CLOUDFLARE_ACCOUNT_ID`) | `cloudflare-workers-ai` |
| xAI | `XAI_API_KEY` | `xai` |
| OpenRouter | `OPENROUTER_API_KEY` | `openrouter` |
@@ -169,6 +170,19 @@ export AWS_BEDROCK_SKIP_AUTH=1
export AWS_BEDROCK_FORCE_HTTP1=1
```
### Cloudflare AI Gateway
`CLOUDFLARE_API_KEY` can be set via `/login`. The account ID and gateway slug must be set as environment variables.
```bash
export CLOUDFLARE_API_KEY=... # or use /login
export CLOUDFLARE_ACCOUNT_ID=...
export CLOUDFLARE_GATEWAY_ID=... # create at dash.cloudflare.com → AI → AI Gateway
pi --provider cloudflare-ai-gateway --model "anthropic/claude-sonnet-4-5"
```
Routes to OpenAI, Anthropic, and Workers AI through Cloudflare's [Unified API](https://developers.cloudflare.com/ai-gateway/usage/unified-api/) (`/compat`). Workers AI and Anthropic models use the prefixed model id (`workers-ai/@cf/...`, `anthropic/...`). OpenAI reasoning models (gpt-5.x, o-series) need `/v1/responses` which `/compat` doesn't yet expose, so they route through the gateway's `/openai` provider-specific subpath with the native id (`gpt-5.1`). Requires BYOK or unified billing on the gateway for non-Workers-AI upstreams.
### Cloudflare Workers AI
`CLOUDFLARE_API_KEY` can be set via `/login`. `CLOUDFLARE_ACCOUNT_ID` must be set as an environment variable.

5
packages/coding-agent/src/cli/args.ts
View File

@@ -317,8 +317,9 @@ ${chalk.bold("Environment Variables:")}
MOONSHOT_API_KEY - Moonshot AI API key
OPENCODE_API_KEY - OpenCode Zen/OpenCode Go API key
KIMI_API_KEY - Kimi For Coding API key
CLOUDFLARE_API_KEY - Cloudflare API token (Workers AI)
CLOUDFLARE_ACCOUNT_ID - Cloudflare account id (required for Workers AI)
CLOUDFLARE_API_KEY - Cloudflare API token (Workers AI and AI Gateway)
CLOUDFLARE_ACCOUNT_ID - Cloudflare account id (required for both)
CLOUDFLARE_GATEWAY_ID - Cloudflare AI Gateway slug (required for AI Gateway)
AWS_PROFILE - AWS profile for Amazon Bedrock
AWS_ACCESS_KEY_ID - AWS access key for Amazon Bedrock
AWS_SECRET_ACCESS_KEY - AWS secret key for Amazon Bedrock

1
packages/coding-agent/src/core/model-resolver.ts
View File

@@ -38,6 +38,7 @@ export const defaultModelPerProvider: Record<KnownProvider, string> = {
"opencode-go": "kimi-k2.6",
"kimi-coding": "kimi-for-coding",
"cloudflare-workers-ai": "@cf/moonshotai/kimi-k2.6",
"cloudflare-ai-gateway": "workers-ai/@cf/moonshotai/kimi-k2.6",
};
export interface ScopedModel {

7
packages/coding-agent/src/core/sdk.ts
View File

@@ -141,7 +141,12 @@ function getAttributionHeaders(
};
}
if (model.provider === "cloudflare-workers-ai" || model.baseUrl.includes("api.cloudflare.com")) {
if (
model.provider === "cloudflare-workers-ai" ||
model.provider === "cloudflare-ai-gateway" ||
model.baseUrl.includes("api.cloudflare.com") ||
model.baseUrl.includes("gateway.ai.cloudflare.com")
) {
return {
"User-Agent": "pi-coding-agent",
};