docs: clarify compat flags for openai-compatible local servers closes #2177
This commit is contained in:
@@ -35,6 +35,32 @@ For local models (Ollama, LM Studio, vLLM), only `id` is required per model:
|
||||
|
||||
The `apiKey` is required but Ollama ignores it, so any value works.
|
||||
|
||||
Some OpenAI-compatible servers do not understand the `developer` role used for reasoning-capable models. For those providers, set `compat.supportsDeveloperRole` to `false` so pi sends the system prompt as a `system` message instead. If the server also does not support `reasoning_effort`, set `compat.supportsReasoningEffort` to `false` too.
|
||||
|
||||
You can set `compat` at the provider level to apply to all models, or at the model level to override a specific model. This commonly applies to Ollama, vLLM, SGLang, and similar OpenAI-compatible servers.
|
||||
|
||||
```json
|
||||
{
|
||||
"providers": {
|
||||
"ollama": {
|
||||
"baseUrl": "http://localhost:11434/v1",
|
||||
"api": "openai-completions",
|
||||
"apiKey": "ollama",
|
||||
"compat": {
|
||||
"supportsDeveloperRole": false,
|
||||
"supportsReasoningEffort": false
|
||||
},
|
||||
"models": [
|
||||
{
|
||||
"id": "gpt-oss:20b",
|
||||
"reasoning": true
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Full Example
|
||||
|
||||
Override defaults when you need specific values:
|
||||
|
||||
Reference in New Issue
Block a user