fix(ai): support Anthropic eager tool streaming compat

closes #3575
This commit is contained in:
Mario Zechner
2026-04-23 23:12:45 +02:00
parent 6af10c9c7f
commit ffa0f31239
9 changed files with 412 additions and 16 deletions

View File

@@ -11,6 +11,7 @@ Add custom providers and models (Ollama, vLLM, LM Studio, proxies) via `~/.pi/ag
- [Model Configuration](#model-configuration)
- [Overriding Built-in Providers](#overriding-built-in-providers)
- [Per-model Overrides](#per-model-overrides)
- [Anthropic Messages Compatibility](#anthropic-messages-compatibility)
- [OpenAI Compatibility](#openai-compatibility)
## Minimal Example
@@ -195,7 +196,7 @@ If your command is slow, expensive, rate-limited, or should keep using a previou
| `contextWindow` | No | `128000` | Context window size in tokens |
| `maxTokens` | No | `16384` | Maximum output tokens |
| `cost` | No | all zeros | `{"input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0}` (per million tokens) |
| `compat` | No | provider `compat` | OpenAI compatibility overrides. Merged with provider-level `compat` when both are set. |
| `compat` | No | provider `compat` | Provider compatibility overrides. Merged with provider-level `compat` when both are set. |
Current behavior:
- `/model` and `--list-models` list entries by model `id`.
@@ -269,6 +270,38 @@ Behavior notes:
- You can combine provider-level `baseUrl`/`headers` with `modelOverrides`.
- If `models` is also defined for a provider, custom models are merged after built-in overrides. A custom model with the same `id` replaces the overridden built-in model entry.
## Anthropic Messages Compatibility
For providers or proxies using `api: "anthropic-messages"`, use `compat.supportsEagerToolInputStreaming` to control Anthropic fine-grained tool streaming compatibility.
By default pi sends per-tool `eager_input_streaming: true`. If a proxy or Anthropic-compatible backend rejects that field, set `supportsEagerToolInputStreaming` to `false`. Pi will omit `tools[].eager_input_streaming` and send the legacy `fine-grained-tool-streaming-2025-05-14` beta header for tool-enabled requests instead.
```json
{
"providers": {
"anthropic-proxy": {
"baseUrl": "https://proxy.example.com",
"api": "anthropic-messages",
"apiKey": "ANTHROPIC_PROXY_KEY",
"compat": {
"supportsEagerToolInputStreaming": false
},
"models": [
{
"id": "claude-opus-4-7",
"reasoning": true,
"input": ["text", "image"]
}
]
}
}
}
```
| Field | Description |
|-------|-------------|
| `supportsEagerToolInputStreaming` | Whether the provider accepts per-tool `eager_input_streaming`. Default: `true`. Set to `false` to omit that field and use the legacy fine-grained tool streaming beta header on tool-enabled requests. |
## OpenAI Compatibility
For providers with partial OpenAI compatibility, use the `compat` field.