@@ -11,6 +11,7 @@ Add custom providers and models (Ollama, vLLM, LM Studio, proxies) via `~/.pi/ag
|
||||
- [Model Configuration](#model-configuration)
|
||||
- [Overriding Built-in Providers](#overriding-built-in-providers)
|
||||
- [Per-model Overrides](#per-model-overrides)
|
||||
- [Anthropic Messages Compatibility](#anthropic-messages-compatibility)
|
||||
- [OpenAI Compatibility](#openai-compatibility)
|
||||
|
||||
## Minimal Example
|
||||
@@ -195,7 +196,7 @@ If your command is slow, expensive, rate-limited, or should keep using a previou
|
||||
| `contextWindow` | No | `128000` | Context window size in tokens |
|
||||
| `maxTokens` | No | `16384` | Maximum output tokens |
|
||||
| `cost` | No | all zeros | `{"input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0}` (per million tokens) |
|
||||
| `compat` | No | provider `compat` | OpenAI compatibility overrides. Merged with provider-level `compat` when both are set. |
|
||||
| `compat` | No | provider `compat` | Provider compatibility overrides. Merged with provider-level `compat` when both are set. |
|
||||
|
||||
Current behavior:
|
||||
- `/model` and `--list-models` list entries by model `id`.
|
||||
@@ -269,6 +270,38 @@ Behavior notes:
|
||||
- You can combine provider-level `baseUrl`/`headers` with `modelOverrides`.
|
||||
- If `models` is also defined for a provider, custom models are merged after built-in overrides. A custom model with the same `id` replaces the overridden built-in model entry.
|
||||
|
||||
## Anthropic Messages Compatibility
|
||||
|
||||
For providers or proxies using `api: "anthropic-messages"`, use `compat.supportsEagerToolInputStreaming` to control Anthropic fine-grained tool streaming compatibility.
|
||||
|
||||
By default pi sends per-tool `eager_input_streaming: true`. If a proxy or Anthropic-compatible backend rejects that field, set `supportsEagerToolInputStreaming` to `false`. Pi will omit `tools[].eager_input_streaming` and send the legacy `fine-grained-tool-streaming-2025-05-14` beta header for tool-enabled requests instead.
|
||||
|
||||
```json
|
||||
{
|
||||
"providers": {
|
||||
"anthropic-proxy": {
|
||||
"baseUrl": "https://proxy.example.com",
|
||||
"api": "anthropic-messages",
|
||||
"apiKey": "ANTHROPIC_PROXY_KEY",
|
||||
"compat": {
|
||||
"supportsEagerToolInputStreaming": false
|
||||
},
|
||||
"models": [
|
||||
{
|
||||
"id": "claude-opus-4-7",
|
||||
"reasoning": true,
|
||||
"input": ["text", "image"]
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `supportsEagerToolInputStreaming` | Whether the provider accepts per-tool `eager_input_streaming`. Default: `true`. Set to `false` to omit that field and use the legacy fine-grained tool streaming beta header on tool-enabled requests. |
|
||||
|
||||
## OpenAI Compatibility
|
||||
|
||||
For providers with partial OpenAI compatibility, use the `compat` field.
|
||||
|
||||
Reference in New Issue
Block a user