shumengya/sproutclaw
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(ai): add Cloudflare Workers AI as a provider (#3851)

Browse Source 
* feat(ai): add Cloudflare Workers AI as a provider

Cloudflare Workers AI hosts open-weight LLMs (Kimi K2.6, GPT-OSS,
GLM-4.7, Llama 4, Gemma 4, Nemotron 3) on Cloudflare's GPU network with
an OpenAI-compatible endpoint. Reuses the openai-completions API
protocol; the per-account URL contains a {CLOUDFLARE_ACCOUNT_ID}
placeholder resolved at request time by a small helper.

Pi automatically sets x-session-affinity for prefix caching:
https://developers.cloudflare.com/workers-ai/features/prompt-caching/

Auth: CLOUDFLARE_API_KEY (matches pi's *_API_KEY convention) +
CLOUDFLARE_ACCOUNT_ID. The User-Agent identifies traffic as
'pi-coding-agent' in Cloudflare analytics.

Verified end-to-end against a real Cloudflare account: 17 e2e tests
pass across stream/empty/tokens/unicode/tool-call-without-result/
total-tokens against @cf/moonshotai/kimi-k2.6.

Cloudflare AI Gateway is a separate, larger change (it requires routing
through provider-specific subpaths with the matching API protocol per
upstream) and will land in a follow-up PR.

* refactor(ai): move Cloudflare User-Agent and session-affinity flag to per-model metadata

Instead of conditionally setting them in openai-completions.ts based on
provider detection, declare them as model-level fields in the catalog
(headers + compat). This is consistent with how the github-copilot and
kimi-coding entries already declare their static headers.

  packages/ai/scripts/generate-models.ts: emit headers and compat fields
  on each cloudflare-workers-ai entry (CLOUDFLARE_STATIC_HEADERS).
  packages/ai/src/providers/openai-completions.ts: drop the
  isCloudflareProvider conditional that injected User-Agent and the
  isCloudflareWorkersAI override of sendSessionAffinityHeaders.
  packages/ai/src/models.generated.ts: re-spliced 8 cloudflare-workers-ai
  entries with headers + compat.

Behavior is unchanged - verified via fetch interceptor that User-Agent
and x-session-affinity / session_id / x-client-request-id are still sent
on outbound requests. 5/5 e2e tests pass.
This commit is contained in:
MC
2026-04-27 17:41:54 -04:00
committed by GitHub
parent 31b48dec23
commit d6e08b3da0
19 changed files with 367 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
packages/ai/README.md
View File

@@ -57,6 +57,7 @@ Unified LLM API with automatic model discovery, provider configuration, token an
- **Mistral**
- **Groq**
- **Cerebras**
- **Cloudflare Workers AI**
- **xAI**
- **OpenRouter**
- **Vercel AI Gateway**
@@ -844,7 +845,7 @@ const ollamaReasoningModel: Model<'openai-completions'> = {
### OpenAI Compatibility Settings
The `openai-completions` API is implemented by many providers with minor differences. By default, the library auto-detects compatibility settings based on `baseUrl` for a small set of known OpenAI-compatible providers (Cerebras, xAI, Chutes, DeepSeek, zAi, OpenCode, etc.). For custom proxies or unknown endpoints, you can override these settings via the `compat` field. For `openai-responses` models, the compat field only supports Responses-specific flags.
The `openai-completions` API is implemented by many providers with minor differences. By default, the library auto-detects compatibility settings based on `baseUrl` for a small set of known OpenAI-compatible providers (Cerebras, xAI, Chutes, DeepSeek, zAi, OpenCode, Cloudflare Workers AI, etc.). For custom proxies or unknown endpoints, you can override these settings via the `compat` field. For `openai-responses` models, the compat field only supports Responses-specific flags.
```typescript
interface OpenAICompletionsCompat {
@@ -1028,6 +1029,7 @@ In Node.js environments, you can set environment variables to avoid passing API
| Mistral | `MISTRAL_API_KEY` |
| Groq | `GROQ_API_KEY` |
| Cerebras | `CEREBRAS_API_KEY` |
| Cloudflare Workers AI | `CLOUDFLARE_API_KEY` + `CLOUDFLARE_ACCOUNT_ID` |
| xAI | `XAI_API_KEY` |
| Fireworks | `FIREWORKS_API_KEY` |
| OpenRouter | `OPENROUTER_API_KEY` |