shumengya/sproutclaw
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(ai): add Cloudflare AI Gateway as a provider (#3856)

Browse Source 
* feat(ai): add Cloudflare AI Gateway as a provider

Routes through Cloudflare's Unified API (`/compat`) for Workers AI and
Anthropic models, and through the provider-specific `/openai` subpath
for OpenAI models so reasoning models (gpt-5.x, o-series) can hit
`/v1/responses` natively. Once `/compat` adds Responses-API support,
the OpenAI subpath can be folded back in.

Catalog layout:
  workers-ai/@cf/...  -> openai-completions, gateway/.../compat
  anthropic/...       -> openai-completions, gateway/.../compat
  <native-id>         -> openai-responses,   gateway/.../openai
                         (gpt-5.1, claude-... no, sorry: gpt-5.x and o-series only;
                          prefix stripped because the OpenAI SDK posts native ids)

Touches:
  packages/ai/src/types.ts                       add cloudflare-ai-gateway to KnownProvider
  packages/ai/src/env-api-keys.ts                map to CLOUDFLARE_API_KEY
  packages/ai/src/providers/cloudflare.ts        add CLOUDFLARE_AI_GATEWAY_COMPAT_BASE_URL
                                                 and CLOUDFLARE_AI_GATEWAY_OPENAI_BASE_URL
  packages/ai/src/providers/openai-responses.ts  one-line dispatch through resolveCloudflareBaseUrl
                                                 (matches what openai-completions.ts already does)
  packages/ai/scripts/generate-models.ts         branch openai/* vs workers-ai/anthropic/*
  packages/ai/src/models.generated.ts            spliced 34 entries
  packages/ai/test/stream.test.ts                3 e2e blocks (one per upstream)
  packages/coding-agent/*                        defaultModelPerProvider, login, env docs,
                                                 README, providers.md

Verified end-to-end against a real Cloudflare account with unified
billing: 9/9 e2e tests pass across all three upstreams (Workers AI
Kimi K2.6, OpenAI gpt-5.1 reasoning, Anthropic claude-sonnet-4-5).

* refactor(ai): move AI Gateway User-Agent and per-route session-affinity flag to catalog

Mirrors the same per-model metadata refactor done for Workers AI in the
parent branch. All cloudflare-ai-gateway entries get the User-Agent
header. Only workers-ai/* gateway entries set
`compat.sendSessionAffinityHeaders: true` because the gateway
forwards that header to the underlying Workers AI runtime; anthropic/*
upstream and openai/* (openai-responses) don't use it.

  packages/ai/scripts/generate-models.ts: emit headers (always) and
  per-upstream compat (workers-ai only) on each cloudflare-ai-gateway
  entry.
  packages/ai/src/models.generated.ts: re-spliced 35 entries with
  headers + conditional compat.

Behavior unchanged - 9/9 e2e tests pass across all three upstream
families.

* fix(ai): align AI Gateway with telemetry-aware UA helper

Adapts to badlogic/pi-mono#3851's follow-up fix ("honor telemetry for
Cloudflare attribution headers", fbb5eed) which moved the
'User-Agent: pi-coding-agent' header out of per-model catalog metadata
and into a centralized telemetry-honoring helper
(coding-agent/src/core/sdk.ts:getAttributionHeaders).

- packages/coding-agent/src/core/sdk.ts: extend the cloudflare branch of
  getAttributionHeaders to also match cloudflare-ai-gateway and
  gateway.ai.cloudflare.com.

- packages/ai/scripts/generate-models.ts and src/models.generated.ts:
  drop 'headers' from the 35 cloudflare-ai-gateway entries (constant
  CLOUDFLARE_STATIC_HEADERS no longer exists). Per-route
  compat.sendSessionAffinityHeaders is unchanged.

End-to-end behavior unchanged: 9/9 tests still pass across all three
upstream families (Workers AI, Anthropic, OpenAI Responses).

---------

Co-authored-by: Mario Zechner <badlogicgames@gmail.com>
This commit is contained in:
MC
2026-04-30 22:29:37 +01:00
committed by GitHub
parent c9ddca1968
commit 24fb6b833b
13 changed files with 755 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files

54
packages/ai/test/stream.test.ts
View File

@@ -641,6 +641,60 @@ describe("Generate E2E Tests", () => {
},
);
describe.skipIf(
!process.env.CLOUDFLARE_API_KEY || !process.env.CLOUDFLARE_ACCOUNT_ID || !process.env.CLOUDFLARE_GATEWAY_ID,
)("Cloudflare AI Gateway → Workers AI (Kimi K2.6 via /compat)", () => {
const llm = getModel("cloudflare-ai-gateway", "workers-ai/@cf/moonshotai/kimi-k2.6");
it("should complete basic text generation", { retry: 3 }, async () => {
await basicTextGeneration(llm);
});
it("should handle tool calling", { retry: 3 }, async () => {
await handleToolCall(llm);
});
it("should handle streaming", { retry: 3 }, async () => {
await handleStreaming(llm);
});
});
describe.skipIf(
!process.env.CLOUDFLARE_API_KEY || !process.env.CLOUDFLARE_ACCOUNT_ID || !process.env.CLOUDFLARE_GATEWAY_ID,
)("Cloudflare AI Gateway → OpenAI (gpt-5.1 via /openai responses)", () => {
const llm = getModel("cloudflare-ai-gateway", "gpt-5.1");
it("should complete basic text generation", { retry: 3 }, async () => {
await basicTextGeneration(llm);
});
it("should handle tool calling", { retry: 3 }, async () => {
await handleToolCall(llm);
});
it("should handle streaming", { retry: 3 }, async () => {
await handleStreaming(llm);
});
});
describe.skipIf(
!process.env.CLOUDFLARE_API_KEY || !process.env.CLOUDFLARE_ACCOUNT_ID || !process.env.CLOUDFLARE_GATEWAY_ID,
)("Cloudflare AI Gateway → Anthropic (claude-sonnet-4-5 via /compat)", () => {
const llm = getModel("cloudflare-ai-gateway", "anthropic/claude-sonnet-4-5");
it("should complete basic text generation", { retry: 3 }, async () => {
await basicTextGeneration(llm);
});
it("should handle tool calling", { retry: 3 }, async () => {
await handleToolCall(llm);
});
it("should handle streaming", { retry: 3 }, async () => {
await handleStreaming(llm);
});
});
describe.skipIf(!process.env.HF_TOKEN)("Hugging Face Provider (Kimi-K2.5 via OpenAI Completions)", () => {
const llm = getModel("huggingface", "moonshotai/Kimi-K2.5");