Support adaptive thinking for Anthropic-compatible aliases

closes #4790
This commit is contained in:
Mario Zechner
2026-05-22 18:30:04 +02:00
parent 7002c68f8b
commit d801d88a11
11 changed files with 239 additions and 33 deletions

View File

@@ -230,6 +230,8 @@ models: [{
Use `openrouter` for OpenRouter-style `reasoning: { effort }` controls. Use `together` for Together-style `reasoning: { enabled }` controls; with `supportsReasoningEffort`, it also sends `reasoning_effort`. Use `qwen-chat-template` instead for local Qwen-compatible servers that read `chat_template_kwargs.enable_thinking`.
Use `cacheControlFormat: "anthropic"` for OpenAI-compatible providers that expose Anthropic-style prompt caching via `cache_control` on the system prompt, last tool definition, and last user/assistant text content.
For Anthropic-compatible providers using `api: "anthropic-messages"`, set `compat.forceAdaptiveThinking: true` on models or providers whose upstream model requires adaptive thinking (`thinking.type: "adaptive"` plus `output_config.effort`). Built-in adaptive Claude models set this automatically.
> Migration note: Mistral moved from `openai-completions` to `mistral-conversations`.
> Use `mistral-conversations` for native Mistral models.
> If you intentionally route Mistral-compatible/custom endpoints through `openai-completions`, set `compat` flags explicitly as needed.
@@ -702,8 +704,9 @@ interface ProviderModelConfig {
/** Custom headers for this specific model. */
headers?: Record<string, string>;
/** OpenAI compatibility settings for openai-completions API. */
/** Compatibility settings for the selected API. */
compat?: {
// openai-completions
supportsStore?: boolean;
supportsDeveloperRole?: boolean;
supportsReasoningEffort?: boolean;
@@ -715,6 +718,13 @@ interface ProviderModelConfig {
requiresReasoningContentOnAssistantMessages?: boolean;
thinkingFormat?: "openai" | "openrouter" | "deepseek" | "together" | "zai" | "qwen" | "qwen-chat-template";
cacheControlFormat?: "anthropic";
// anthropic-messages
supportsEagerToolInputStreaming?: boolean;
supportsLongCacheRetention?: boolean;
sendSessionAffinityHeaders?: boolean;
supportsCacheControlOnTools?: boolean;
forceAdaptiveThinking?: boolean;
};
}
```

View File

@@ -315,10 +315,12 @@ Behavior notes:
## Anthropic Messages Compatibility
For providers or proxies using `api: "anthropic-messages"`, use `compat.supportsEagerToolInputStreaming` to control Anthropic fine-grained tool streaming compatibility.
For providers or proxies using `api: "anthropic-messages"`, use `compat` to control Anthropic-specific request compatibility.
By default pi sends per-tool `eager_input_streaming: true`. If a proxy or Anthropic-compatible backend rejects that field, set `supportsEagerToolInputStreaming` to `false`. Pi will omit `tools[].eager_input_streaming` and send the legacy `fine-grained-tool-streaming-2025-05-14` beta header for tool-enabled requests instead.
Some Anthropic models require adaptive thinking (`thinking.type: "adaptive"` plus `output_config.effort`) instead of the legacy budget-based thinking payload. Built-in models set this automatically. For custom providers or aliases that route to those models, set `forceAdaptiveThinking` to `true`.
```json
{
"providers": {
@@ -328,7 +330,8 @@ By default pi sends per-tool `eager_input_streaming: true`. If a proxy or Anthro
"apiKey": "ANTHROPIC_PROXY_KEY",
"compat": {
"supportsEagerToolInputStreaming": false,
"supportsLongCacheRetention": true
"supportsLongCacheRetention": true,
"forceAdaptiveThinking": true
},
"models": [
{
@@ -346,6 +349,9 @@ By default pi sends per-tool `eager_input_streaming: true`. If a proxy or Anthro
|-------|-------------|
| `supportsEagerToolInputStreaming` | Whether the provider accepts per-tool `eager_input_streaming`. Default: `true`. Set to `false` to omit that field and use the legacy fine-grained tool streaming beta header on tool-enabled requests. |
| `supportsLongCacheRetention` | Whether the provider accepts Anthropic long cache retention (`cache_control.ttl: "1h"`) when cache retention is `long`. Default: `true`. |
| `sendSessionAffinityHeaders` | Whether to send `x-session-affinity` from the session id when caching is enabled. Default: auto-detected for known providers. |
| `supportsCacheControlOnTools` | Whether the provider accepts Anthropic-style `cache_control` markers on tool definitions. Default: `true`. |
| `forceAdaptiveThinking` | Whether to send adaptive thinking (`thinking.type: "adaptive"` plus `output_config.effort`) for this model. Built-in adaptive models set this automatically. Default: `false`. |
## OpenAI Compatibility