Files
sproutclaw/packages/coding-agent/docs/providers.md
MC 24fb6b833b feat(ai): add Cloudflare AI Gateway as a provider (#3856)
* feat(ai): add Cloudflare AI Gateway as a provider

Routes through Cloudflare's Unified API (`/compat`) for Workers AI and
Anthropic models, and through the provider-specific `/openai` subpath
for OpenAI models so reasoning models (gpt-5.x, o-series) can hit
`/v1/responses` natively. Once `/compat` adds Responses-API support,
the OpenAI subpath can be folded back in.

Catalog layout:
  workers-ai/@cf/...  -> openai-completions, gateway/.../compat
  anthropic/...       -> openai-completions, gateway/.../compat
  <native-id>         -> openai-responses,   gateway/.../openai
                         (gpt-5.1, claude-... no, sorry: gpt-5.x and o-series only;
                          prefix stripped because the OpenAI SDK posts native ids)

Touches:
  packages/ai/src/types.ts                       add cloudflare-ai-gateway to KnownProvider
  packages/ai/src/env-api-keys.ts                map to CLOUDFLARE_API_KEY
  packages/ai/src/providers/cloudflare.ts        add CLOUDFLARE_AI_GATEWAY_COMPAT_BASE_URL
                                                 and CLOUDFLARE_AI_GATEWAY_OPENAI_BASE_URL
  packages/ai/src/providers/openai-responses.ts  one-line dispatch through resolveCloudflareBaseUrl
                                                 (matches what openai-completions.ts already does)
  packages/ai/scripts/generate-models.ts         branch openai/* vs workers-ai/anthropic/*
  packages/ai/src/models.generated.ts            spliced 34 entries
  packages/ai/test/stream.test.ts                3 e2e blocks (one per upstream)
  packages/coding-agent/*                        defaultModelPerProvider, login, env docs,
                                                 README, providers.md

Verified end-to-end against a real Cloudflare account with unified
billing: 9/9 e2e tests pass across all three upstreams (Workers AI
Kimi K2.6, OpenAI gpt-5.1 reasoning, Anthropic claude-sonnet-4-5).

* refactor(ai): move AI Gateway User-Agent and per-route session-affinity flag to catalog

Mirrors the same per-model metadata refactor done for Workers AI in the
parent branch. All cloudflare-ai-gateway entries get the User-Agent
header. Only workers-ai/* gateway entries set
`compat.sendSessionAffinityHeaders: true` because the gateway
forwards that header to the underlying Workers AI runtime; anthropic/*
upstream and openai/* (openai-responses) don't use it.

  packages/ai/scripts/generate-models.ts: emit headers (always) and
  per-upstream compat (workers-ai only) on each cloudflare-ai-gateway
  entry.
  packages/ai/src/models.generated.ts: re-spliced 35 entries with
  headers + conditional compat.

Behavior unchanged - 9/9 e2e tests pass across all three upstream
families.

* fix(ai): align AI Gateway with telemetry-aware UA helper

Adapts to badlogic/pi-mono#3851's follow-up fix ("honor telemetry for
Cloudflare attribution headers", fbb5eed) which moved the
'User-Agent: pi-coding-agent' header out of per-model catalog metadata
and into a centralized telemetry-honoring helper
(coding-agent/src/core/sdk.ts:getAttributionHeaders).

- packages/coding-agent/src/core/sdk.ts: extend the cloudflare branch of
  getAttributionHeaders to also match cloudflare-ai-gateway and
  gateway.ai.cloudflare.com.

- packages/ai/scripts/generate-models.ts and src/models.generated.ts:
  drop 'headers' from the 35 cloudflare-ai-gateway entries (constant
  CLOUDFLARE_STATIC_HEADERS no longer exists). Per-route
  compat.sendSessionAffinityHeaders is unchanged.

End-to-end behavior unchanged: 9/9 tests still pass across all three
upstream families (Workers AI, Anthropic, OpenAI Responses).

---------

Co-authored-by: Mario Zechner <badlogicgames@gmail.com>
2026-04-30 23:29:37 +02:00

8.2 KiB

Providers

Pi supports subscription-based providers via OAuth and API key providers via environment variables or auth file. For each provider, pi knows all available models. The list is updated with every pi release.

Table of Contents

Subscriptions

Use /login in interactive mode, then select a provider:

  • Claude Pro/Max
  • ChatGPT Plus/Pro (Codex)
  • GitHub Copilot

Use /logout to clear credentials. Tokens are stored in ~/.pi/agent/auth.json and auto-refresh when expired.

GitHub Copilot

  • Press Enter for github.com, or enter your GitHub Enterprise Server domain
  • If you get "model not supported", enable it in VS Code: Copilot Chat → model selector → select model → "Enable"

Google Providers

  • Both free with any Google account, subject to rate limits
  • For paid Cloud Code Assist: set GOOGLE_CLOUD_PROJECT env var

OpenAI Codex

  • Requires ChatGPT Plus or Pro subscription
  • Personal use only; for production, use the OpenAI Platform API

API Keys

Environment Variables or Auth File

Use /login in interactive mode and select a provider to store an API key in auth.json, or set credentials via environment variable:

export ANTHROPIC_API_KEY=sk-ant-...
pi
Provider Environment Variable auth.json key
Anthropic ANTHROPIC_API_KEY anthropic
Azure OpenAI Responses AZURE_OPENAI_API_KEY azure-openai-responses
OpenAI OPENAI_API_KEY openai
DeepSeek DEEPSEEK_API_KEY deepseek
Google Gemini GEMINI_API_KEY google
Mistral MISTRAL_API_KEY mistral
Groq GROQ_API_KEY groq
Cerebras CEREBRAS_API_KEY cerebras
Cloudflare AI Gateway CLOUDFLARE_API_KEY (+ CLOUDFLARE_ACCOUNT_ID, CLOUDFLARE_GATEWAY_ID) cloudflare-ai-gateway
Cloudflare Workers AI CLOUDFLARE_API_KEY (+ CLOUDFLARE_ACCOUNT_ID) cloudflare-workers-ai
xAI XAI_API_KEY xai
OpenRouter OPENROUTER_API_KEY openrouter
Vercel AI Gateway AI_GATEWAY_API_KEY vercel-ai-gateway
ZAI ZAI_API_KEY zai
OpenCode Zen OPENCODE_API_KEY opencode
OpenCode Go OPENCODE_API_KEY opencode-go
Hugging Face HF_TOKEN huggingface
Fireworks FIREWORKS_API_KEY fireworks
Kimi For Coding KIMI_API_KEY kimi-coding
MiniMax MINIMAX_API_KEY minimax
MiniMax (China) MINIMAX_CN_API_KEY minimax-cn

Reference for environment variables and auth.json keys: const envMap in packages/ai/src/env-api-keys.ts.

Auth File

Store credentials in ~/.pi/agent/auth.json:

{
  "anthropic": { "type": "api_key", "key": "sk-ant-..." },
  "openai": { "type": "api_key", "key": "sk-..." },
  "deepseek": { "type": "api_key", "key": "sk-..." },
  "google": { "type": "api_key", "key": "..." },
  "opencode": { "type": "api_key", "key": "..." },
  "opencode-go": { "type": "api_key", "key": "..." }
}

The file is created with 0600 permissions (user read/write only). Auth file credentials take priority over environment variables.

Key Resolution

The key field supports three formats:

  • Shell command: "!command" executes and uses stdout (cached for process lifetime)
    { "type": "api_key", "key": "!security find-generic-password -ws 'anthropic'" }
    { "type": "api_key", "key": "!op read 'op://vault/item/credential'" }
    
  • Environment variable: Uses the value of the named variable
    { "type": "api_key", "key": "MY_ANTHROPIC_KEY" }
    
  • Literal value: Used directly
    { "type": "api_key", "key": "sk-ant-..." }
    

OAuth credentials are also stored here after /login and managed automatically.

Cloud Providers

Azure OpenAI

export AZURE_OPENAI_API_KEY=...
export AZURE_OPENAI_BASE_URL=https://your-resource.openai.azure.com
# also supported: https://your-resource.cognitiveservices.azure.com
# root endpoints are auto-normalized to /openai/v1
# or use resource name instead of base URL
export AZURE_OPENAI_RESOURCE_NAME=your-resource

# Optional
export AZURE_OPENAI_API_VERSION=2024-02-01
export AZURE_OPENAI_DEPLOYMENT_NAME_MAP=gpt-4=my-gpt4,gpt-4o=my-gpt4o

Amazon Bedrock

# Option 1: AWS Profile
export AWS_PROFILE=your-profile

# Option 2: IAM Keys
export AWS_ACCESS_KEY_ID=AKIA...
export AWS_SECRET_ACCESS_KEY=...

# Option 3: Bearer Token
export AWS_BEARER_TOKEN_BEDROCK=...

# Optional region (defaults to us-east-1)
export AWS_REGION=us-west-2

Also supports ECS task roles (AWS_CONTAINER_CREDENTIALS_*) and IRSA (AWS_WEB_IDENTITY_TOKEN_FILE).

pi --provider amazon-bedrock --model us.anthropic.claude-sonnet-4-20250514-v1:0

Prompt caching is enabled automatically for Claude models whose ID contains a recognizable model name (base models and system-defined inference profiles). For application inference profiles (whose ARNs don't contain the model name), set AWS_BEDROCK_FORCE_CACHE=1 to enable cache points:

export AWS_BEDROCK_FORCE_CACHE=1
pi --provider amazon-bedrock --model arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/abc123

If you are connecting to a Bedrock API proxy, the following environment variables can be used:

# Set the URL for the Bedrock proxy (standard AWS SDK env var)
export AWS_ENDPOINT_URL_BEDROCK_RUNTIME=https://my.corp.proxy/bedrock

# Set if your proxy does not require authentication
export AWS_BEDROCK_SKIP_AUTH=1

# Set if your proxy only supports HTTP/1.1
export AWS_BEDROCK_FORCE_HTTP1=1

Cloudflare AI Gateway

CLOUDFLARE_API_KEY can be set via /login. The account ID and gateway slug must be set as environment variables.

export CLOUDFLARE_API_KEY=...           # or use /login
export CLOUDFLARE_ACCOUNT_ID=...
export CLOUDFLARE_GATEWAY_ID=...        # create at dash.cloudflare.com → AI → AI Gateway
pi --provider cloudflare-ai-gateway --model "anthropic/claude-sonnet-4-5"

Routes to OpenAI, Anthropic, and Workers AI through Cloudflare's Unified API (/compat). Workers AI and Anthropic models use the prefixed model id (workers-ai/@cf/..., anthropic/...). OpenAI reasoning models (gpt-5.x, o-series) need /v1/responses which /compat doesn't yet expose, so they route through the gateway's /openai provider-specific subpath with the native id (gpt-5.1). Requires BYOK or unified billing on the gateway for non-Workers-AI upstreams.

Cloudflare Workers AI

CLOUDFLARE_API_KEY can be set via /login. CLOUDFLARE_ACCOUNT_ID must be set as an environment variable.

export CLOUDFLARE_API_KEY=...           # or use /login
export CLOUDFLARE_ACCOUNT_ID=...
pi --provider cloudflare-workers-ai --model "@cf/moonshotai/kimi-k2.6"

Pi automatically sets x-session-affinity for prefix caching discounts.

Google Vertex AI

Uses Application Default Credentials:

gcloud auth application-default login
export GOOGLE_CLOUD_PROJECT=your-project
export GOOGLE_CLOUD_LOCATION=us-central1

Or set GOOGLE_APPLICATION_CREDENTIALS to a service account key file.

Custom Providers

Via models.json: Add Ollama, LM Studio, vLLM, or any provider that speaks a supported API (OpenAI Completions, OpenAI Responses, Anthropic Messages, Google Generative AI). See models.md.

Via extensions: For providers that need custom API implementations or OAuth flows, create an extension. See custom-provider.md and examples/extensions/custom-provider-gitlab-duo.

Resolution Order

When resolving credentials for a provider:

  1. CLI --api-key flag
  2. auth.json entry (API key or OAuth token)
  3. Environment variable
  4. Custom provider keys from models.json