24fb6b833b
* feat(ai): add Cloudflare AI Gateway as a provider
Routes through Cloudflare's Unified API (`/compat`) for Workers AI and
Anthropic models, and through the provider-specific `/openai` subpath
for OpenAI models so reasoning models (gpt-5.x, o-series) can hit
`/v1/responses` natively. Once `/compat` adds Responses-API support,
the OpenAI subpath can be folded back in.
Catalog layout:
workers-ai/@cf/... -> openai-completions, gateway/.../compat
anthropic/... -> openai-completions, gateway/.../compat
<native-id> -> openai-responses, gateway/.../openai
(gpt-5.1, claude-... no, sorry: gpt-5.x and o-series only;
prefix stripped because the OpenAI SDK posts native ids)
Touches:
packages/ai/src/types.ts add cloudflare-ai-gateway to KnownProvider
packages/ai/src/env-api-keys.ts map to CLOUDFLARE_API_KEY
packages/ai/src/providers/cloudflare.ts add CLOUDFLARE_AI_GATEWAY_COMPAT_BASE_URL
and CLOUDFLARE_AI_GATEWAY_OPENAI_BASE_URL
packages/ai/src/providers/openai-responses.ts one-line dispatch through resolveCloudflareBaseUrl
(matches what openai-completions.ts already does)
packages/ai/scripts/generate-models.ts branch openai/* vs workers-ai/anthropic/*
packages/ai/src/models.generated.ts spliced 34 entries
packages/ai/test/stream.test.ts 3 e2e blocks (one per upstream)
packages/coding-agent/* defaultModelPerProvider, login, env docs,
README, providers.md
Verified end-to-end against a real Cloudflare account with unified
billing: 9/9 e2e tests pass across all three upstreams (Workers AI
Kimi K2.6, OpenAI gpt-5.1 reasoning, Anthropic claude-sonnet-4-5).
* refactor(ai): move AI Gateway User-Agent and per-route session-affinity flag to catalog
Mirrors the same per-model metadata refactor done for Workers AI in the
parent branch. All cloudflare-ai-gateway entries get the User-Agent
header. Only workers-ai/* gateway entries set
`compat.sendSessionAffinityHeaders: true` because the gateway
forwards that header to the underlying Workers AI runtime; anthropic/*
upstream and openai/* (openai-responses) don't use it.
packages/ai/scripts/generate-models.ts: emit headers (always) and
per-upstream compat (workers-ai only) on each cloudflare-ai-gateway
entry.
packages/ai/src/models.generated.ts: re-spliced 35 entries with
headers + conditional compat.
Behavior unchanged - 9/9 e2e tests pass across all three upstream
families.
* fix(ai): align AI Gateway with telemetry-aware UA helper
Adapts to badlogic/pi-mono#3851's follow-up fix ("honor telemetry for
Cloudflare attribution headers", fbb5eed) which moved the
'User-Agent: pi-coding-agent' header out of per-model catalog metadata
and into a centralized telemetry-honoring helper
(coding-agent/src/core/sdk.ts:getAttributionHeaders).
- packages/coding-agent/src/core/sdk.ts: extend the cloudflare branch of
getAttributionHeaders to also match cloudflare-ai-gateway and
gateway.ai.cloudflare.com.
- packages/ai/scripts/generate-models.ts and src/models.generated.ts:
drop 'headers' from the 35 cloudflare-ai-gateway entries (constant
CLOUDFLARE_STATIC_HEADERS no longer exists). Per-route
compat.sendSessionAffinityHeaders is unchanged.
End-to-end behavior unchanged: 9/9 tests still pass across all three
upstream families (Workers AI, Anthropic, OpenAI Responses).
---------
Co-authored-by: Mario Zechner <badlogicgames@gmail.com>
224 lines
8.2 KiB
Markdown
224 lines
8.2 KiB
Markdown
# Providers
|
|
|
|
Pi supports subscription-based providers via OAuth and API key providers via environment variables or auth file. For each provider, pi knows all available models. The list is updated with every pi release.
|
|
|
|
## Table of Contents
|
|
|
|
- [Subscriptions](#subscriptions)
|
|
- [API Keys](#api-keys)
|
|
- [Auth File](#auth-file)
|
|
- [Cloud Providers](#cloud-providers)
|
|
- [Custom Providers](#custom-providers)
|
|
- [Resolution Order](#resolution-order)
|
|
|
|
## Subscriptions
|
|
|
|
Use `/login` in interactive mode, then select a provider:
|
|
|
|
- Claude Pro/Max
|
|
- ChatGPT Plus/Pro (Codex)
|
|
- GitHub Copilot
|
|
|
|
Use `/logout` to clear credentials. Tokens are stored in `~/.pi/agent/auth.json` and auto-refresh when expired.
|
|
|
|
### GitHub Copilot
|
|
|
|
- Press Enter for github.com, or enter your GitHub Enterprise Server domain
|
|
- If you get "model not supported", enable it in VS Code: Copilot Chat → model selector → select model → "Enable"
|
|
|
|
### Google Providers
|
|
|
|
- Both free with any Google account, subject to rate limits
|
|
- For paid Cloud Code Assist: set `GOOGLE_CLOUD_PROJECT` env var
|
|
|
|
### OpenAI Codex
|
|
|
|
- Requires ChatGPT Plus or Pro subscription
|
|
- Personal use only; for production, use the OpenAI Platform API
|
|
|
|
## API Keys
|
|
|
|
### Environment Variables or Auth File
|
|
|
|
Use `/login` in interactive mode and select a provider to store an API key in `auth.json`, or set credentials via environment variable:
|
|
|
|
```bash
|
|
export ANTHROPIC_API_KEY=sk-ant-...
|
|
pi
|
|
```
|
|
|
|
| Provider | Environment Variable | `auth.json` key |
|
|
|----------|----------------------|------------------|
|
|
| Anthropic | `ANTHROPIC_API_KEY` | `anthropic` |
|
|
| Azure OpenAI Responses | `AZURE_OPENAI_API_KEY` | `azure-openai-responses` |
|
|
| OpenAI | `OPENAI_API_KEY` | `openai` |
|
|
| DeepSeek | `DEEPSEEK_API_KEY` | `deepseek` |
|
|
| Google Gemini | `GEMINI_API_KEY` | `google` |
|
|
| Mistral | `MISTRAL_API_KEY` | `mistral` |
|
|
| Groq | `GROQ_API_KEY` | `groq` |
|
|
| Cerebras | `CEREBRAS_API_KEY` | `cerebras` |
|
|
| Cloudflare AI Gateway | `CLOUDFLARE_API_KEY` (+ `CLOUDFLARE_ACCOUNT_ID`, `CLOUDFLARE_GATEWAY_ID`) | `cloudflare-ai-gateway` |
|
|
| Cloudflare Workers AI | `CLOUDFLARE_API_KEY` (+ `CLOUDFLARE_ACCOUNT_ID`) | `cloudflare-workers-ai` |
|
|
| xAI | `XAI_API_KEY` | `xai` |
|
|
| OpenRouter | `OPENROUTER_API_KEY` | `openrouter` |
|
|
| Vercel AI Gateway | `AI_GATEWAY_API_KEY` | `vercel-ai-gateway` |
|
|
| ZAI | `ZAI_API_KEY` | `zai` |
|
|
| OpenCode Zen | `OPENCODE_API_KEY` | `opencode` |
|
|
| OpenCode Go | `OPENCODE_API_KEY` | `opencode-go` |
|
|
| Hugging Face | `HF_TOKEN` | `huggingface` |
|
|
| Fireworks | `FIREWORKS_API_KEY` | `fireworks` |
|
|
| Kimi For Coding | `KIMI_API_KEY` | `kimi-coding` |
|
|
| MiniMax | `MINIMAX_API_KEY` | `minimax` |
|
|
| MiniMax (China) | `MINIMAX_CN_API_KEY` | `minimax-cn` |
|
|
|
|
Reference for environment variables and `auth.json` keys: [`const envMap`](https://github.com/badlogic/pi-mono/blob/main/packages/ai/src/env-api-keys.ts) in [`packages/ai/src/env-api-keys.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/ai/src/env-api-keys.ts).
|
|
|
|
#### Auth File
|
|
|
|
Store credentials in `~/.pi/agent/auth.json`:
|
|
|
|
```json
|
|
{
|
|
"anthropic": { "type": "api_key", "key": "sk-ant-..." },
|
|
"openai": { "type": "api_key", "key": "sk-..." },
|
|
"deepseek": { "type": "api_key", "key": "sk-..." },
|
|
"google": { "type": "api_key", "key": "..." },
|
|
"opencode": { "type": "api_key", "key": "..." },
|
|
"opencode-go": { "type": "api_key", "key": "..." }
|
|
}
|
|
```
|
|
|
|
The file is created with `0600` permissions (user read/write only). Auth file credentials take priority over environment variables.
|
|
|
|
### Key Resolution
|
|
|
|
The `key` field supports three formats:
|
|
|
|
- **Shell command:** `"!command"` executes and uses stdout (cached for process lifetime)
|
|
```json
|
|
{ "type": "api_key", "key": "!security find-generic-password -ws 'anthropic'" }
|
|
{ "type": "api_key", "key": "!op read 'op://vault/item/credential'" }
|
|
```
|
|
- **Environment variable:** Uses the value of the named variable
|
|
```json
|
|
{ "type": "api_key", "key": "MY_ANTHROPIC_KEY" }
|
|
```
|
|
- **Literal value:** Used directly
|
|
```json
|
|
{ "type": "api_key", "key": "sk-ant-..." }
|
|
```
|
|
|
|
OAuth credentials are also stored here after `/login` and managed automatically.
|
|
|
|
## Cloud Providers
|
|
|
|
### Azure OpenAI
|
|
|
|
```bash
|
|
export AZURE_OPENAI_API_KEY=...
|
|
export AZURE_OPENAI_BASE_URL=https://your-resource.openai.azure.com
|
|
# also supported: https://your-resource.cognitiveservices.azure.com
|
|
# root endpoints are auto-normalized to /openai/v1
|
|
# or use resource name instead of base URL
|
|
export AZURE_OPENAI_RESOURCE_NAME=your-resource
|
|
|
|
# Optional
|
|
export AZURE_OPENAI_API_VERSION=2024-02-01
|
|
export AZURE_OPENAI_DEPLOYMENT_NAME_MAP=gpt-4=my-gpt4,gpt-4o=my-gpt4o
|
|
```
|
|
|
|
### Amazon Bedrock
|
|
|
|
```bash
|
|
# Option 1: AWS Profile
|
|
export AWS_PROFILE=your-profile
|
|
|
|
# Option 2: IAM Keys
|
|
export AWS_ACCESS_KEY_ID=AKIA...
|
|
export AWS_SECRET_ACCESS_KEY=...
|
|
|
|
# Option 3: Bearer Token
|
|
export AWS_BEARER_TOKEN_BEDROCK=...
|
|
|
|
# Optional region (defaults to us-east-1)
|
|
export AWS_REGION=us-west-2
|
|
```
|
|
|
|
Also supports ECS task roles (`AWS_CONTAINER_CREDENTIALS_*`) and IRSA (`AWS_WEB_IDENTITY_TOKEN_FILE`).
|
|
|
|
```bash
|
|
pi --provider amazon-bedrock --model us.anthropic.claude-sonnet-4-20250514-v1:0
|
|
```
|
|
|
|
Prompt caching is enabled automatically for Claude models whose ID contains a recognizable model name (base models and system-defined inference profiles). For application inference profiles (whose ARNs don't contain the model name), set `AWS_BEDROCK_FORCE_CACHE=1` to enable cache points:
|
|
|
|
```bash
|
|
export AWS_BEDROCK_FORCE_CACHE=1
|
|
pi --provider amazon-bedrock --model arn:aws:bedrock:us-east-1:123456789012:application-inference-profile/abc123
|
|
```
|
|
|
|
If you are connecting to a Bedrock API proxy, the following environment variables can be used:
|
|
|
|
```bash
|
|
# Set the URL for the Bedrock proxy (standard AWS SDK env var)
|
|
export AWS_ENDPOINT_URL_BEDROCK_RUNTIME=https://my.corp.proxy/bedrock
|
|
|
|
# Set if your proxy does not require authentication
|
|
export AWS_BEDROCK_SKIP_AUTH=1
|
|
|
|
# Set if your proxy only supports HTTP/1.1
|
|
export AWS_BEDROCK_FORCE_HTTP1=1
|
|
```
|
|
|
|
### Cloudflare AI Gateway
|
|
|
|
`CLOUDFLARE_API_KEY` can be set via `/login`. The account ID and gateway slug must be set as environment variables.
|
|
|
|
```bash
|
|
export CLOUDFLARE_API_KEY=... # or use /login
|
|
export CLOUDFLARE_ACCOUNT_ID=...
|
|
export CLOUDFLARE_GATEWAY_ID=... # create at dash.cloudflare.com → AI → AI Gateway
|
|
pi --provider cloudflare-ai-gateway --model "anthropic/claude-sonnet-4-5"
|
|
```
|
|
|
|
Routes to OpenAI, Anthropic, and Workers AI through Cloudflare's [Unified API](https://developers.cloudflare.com/ai-gateway/usage/unified-api/) (`/compat`). Workers AI and Anthropic models use the prefixed model id (`workers-ai/@cf/...`, `anthropic/...`). OpenAI reasoning models (gpt-5.x, o-series) need `/v1/responses` which `/compat` doesn't yet expose, so they route through the gateway's `/openai` provider-specific subpath with the native id (`gpt-5.1`). Requires BYOK or unified billing on the gateway for non-Workers-AI upstreams.
|
|
|
|
### Cloudflare Workers AI
|
|
|
|
`CLOUDFLARE_API_KEY` can be set via `/login`. `CLOUDFLARE_ACCOUNT_ID` must be set as an environment variable.
|
|
|
|
```bash
|
|
export CLOUDFLARE_API_KEY=... # or use /login
|
|
export CLOUDFLARE_ACCOUNT_ID=...
|
|
pi --provider cloudflare-workers-ai --model "@cf/moonshotai/kimi-k2.6"
|
|
```
|
|
|
|
Pi automatically sets `x-session-affinity` for [prefix caching](https://developers.cloudflare.com/workers-ai/features/prompt-caching/) discounts.
|
|
|
|
### Google Vertex AI
|
|
|
|
Uses Application Default Credentials:
|
|
|
|
```bash
|
|
gcloud auth application-default login
|
|
export GOOGLE_CLOUD_PROJECT=your-project
|
|
export GOOGLE_CLOUD_LOCATION=us-central1
|
|
```
|
|
|
|
Or set `GOOGLE_APPLICATION_CREDENTIALS` to a service account key file.
|
|
|
|
## Custom Providers
|
|
|
|
**Via models.json:** Add Ollama, LM Studio, vLLM, or any provider that speaks a supported API (OpenAI Completions, OpenAI Responses, Anthropic Messages, Google Generative AI). See [models.md](models.md).
|
|
|
|
**Via extensions:** For providers that need custom API implementations or OAuth flows, create an extension. See [custom-provider.md](custom-provider.md) and [examples/extensions/custom-provider-gitlab-duo](../examples/extensions/custom-provider-gitlab-duo/).
|
|
|
|
## Resolution Order
|
|
|
|
When resolving credentials for a provider:
|
|
|
|
1. CLI `--api-key` flag
|
|
2. `auth.json` entry (API key or OAuth token)
|
|
3. Environment variable
|
|
4. Custom provider keys from `models.json`
|