shumengya/sproutclaw
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: clarify compat flags for openai-compatible local servers closes #2177

Browse Source
This commit is contained in:
Mario Zechner
2026-03-15 16:11:40 +01:00
parent e645995a4a
commit 0b48e2518f
2 changed files with 49 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

23
packages/ai/README.md
View File

@@ -729,6 +729,29 @@ const response = await stream(ollamaModel, context, {
}); });
``` ```
Some OpenAI-compatible servers do not understand the `developer` role used for reasoning-capable models. For those providers, set `compat.supportsDeveloperRole` to `false` so the system prompt is sent as a `system` message instead. If the server also does not support `reasoning_effort`, set `compat.supportsReasoningEffort` to `false` too.
This commonly applies to Ollama, vLLM, SGLang, and similar OpenAI-compatible servers. You can set `compat` at the provider level or per model.
```typescript
const ollamaReasoningModel: Model<'openai-completions'> = {
id: 'gpt-oss:20b',
name: 'GPT-OSS 20B (Ollama)',
api: 'openai-completions',
provider: 'ollama',
baseUrl: 'http://localhost:11434/v1',
reasoning: true,
input: ['text'],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 131072,
maxTokens: 32000,
compat: {
supportsDeveloperRole: false,
supportsReasoningEffort: false,
}
};
```
### OpenAI Compatibility Settings ### OpenAI Compatibility Settings
The `openai-completions` API is implemented by many providers with minor differences. By default, the library auto-detects compatibility settings based on `baseUrl` for a small set of known OpenAI-compatible providers (Cerebras, xAI, Chutes, DeepSeek, zAi, OpenCode, etc.). For custom proxies or unknown endpoints, you can override these settings via the `compat` field. For `openai-responses` models, the compat field only supports Responses-specific flags. The `openai-completions` API is implemented by many providers with minor differences. By default, the library auto-detects compatibility settings based on `baseUrl` for a small set of known OpenAI-compatible providers (Cerebras, xAI, Chutes, DeepSeek, zAi, OpenCode, etc.). For custom proxies or unknown endpoints, you can override these settings via the `compat` field. For `openai-responses` models, the compat field only supports Responses-specific flags.

26
packages/coding-agent/docs/models.md
View File

@@ -35,6 +35,32 @@ For local models (Ollama, LM Studio, vLLM), only `id` is required per model:
The `apiKey` is required but Ollama ignores it, so any value works. The `apiKey` is required but Ollama ignores it, so any value works.
Some OpenAI-compatible servers do not understand the `developer` role used for reasoning-capable models. For those providers, set `compat.supportsDeveloperRole` to `false` so pi sends the system prompt as a `system` message instead. If the server also does not support `reasoning_effort`, set `compat.supportsReasoningEffort` to `false` too.
You can set `compat` at the provider level to apply to all models, or at the model level to override a specific model. This commonly applies to Ollama, vLLM, SGLang, and similar OpenAI-compatible servers.
```json
{
"providers": {
"ollama": {
"baseUrl": "http://localhost:11434/v1",
"api": "openai-completions",
"apiKey": "ollama",
"compat": {
"supportsDeveloperRole": false,
"supportsReasoningEffort": false
},
"models": [
{
"id": "gpt-oss:20b",
"reasoning": true
}
]
}
}
}
```
## Full Example ## Full Example
Override defaults when you need specific values: Override defaults when you need specific values: