merge: sync upstream pi-mono main into sproutclaw
Some checks failed
CI / build-check-test (push) Has been cancelled

Merge upstream/main (195 commits) while keeping sproutclaw-specific
README, webui workflow docs, and restored upstream .pi prompt templates.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
root
2026-05-22 20:05:31 +08:00
642 changed files with 18785 additions and 26436 deletions

View File

@@ -2,13 +2,78 @@
## [Unreleased]
### Fixed
- Fixed Amazon Bedrock Claude requests to send the model output token cap by default, matching Anthropic requests and avoiding Bedrock's 4096-token default truncation ([#4848](https://github.com/earendil-works/pi/issues/4848)).
## [0.75.4] - 2026-05-20
### Changed
- Changed source syntax to avoid TypeScript constructs that require JavaScript emit, keeping the package compatible with Node.js strip-only TypeScript checks.
- Removed the package-level development watch scripts now that the root TypeScript check validates strip-only-compatible sources.
### Fixed
- Fixed OpenAI-compatible `streamSimple()` requests to stop sending model-derived default output token caps, avoiding context-window reservation failures on servers such as vLLM while preserving explicit `maxTokens` and required Anthropic `max_tokens` handling ([#4675](https://github.com/earendil-works/pi/issues/4675)).
- Fixed OpenAI prompt cache keys to clamp session-derived values to the 64-character API limit across OpenAI Responses, Chat Completions, Codex Responses, and Azure OpenAI Responses ([#4720](https://github.com/earendil-works/pi/issues/4720)).
## [0.75.3] - 2026-05-18
## [0.75.2] - 2026-05-18
### Fixed
- Fixed Xiaomi MiMo generated model metadata to replay assistant tool-call messages with `reasoning_content` for thinking-mode multi-turn requests ([#4678](https://github.com/earendil-works/pi/issues/4678)).
## [0.75.1] - 2026-05-18
### Fixed
- Fixed Anthropic-compatible API-key requests to ignore unrelated `ANTHROPIC_AUTH_TOKEN` environment values, avoiding invalid bearer credentials for providers such as Xiaomi MiMo ([#4342](https://github.com/earendil-works/pi/issues/4342)).
- Fixed Amazon Bedrock message conversion to skip unknown content blocks instead of failing the stream ([#4223](https://github.com/earendil-works/pi/issues/4223)).
- Fixed Azure OpenAI Responses and OpenAI Responses error formatting to prefix HTTP status codes onto `errorMessage`, so transient 5xx and 429 errors are correctly matched by the agent-level auto-retry classifier ([#4232](https://github.com/earendil-works/pi/issues/4232)).
- Fixed Xiaomi MiMo model metadata to use the OpenAI-compatible endpoints and `openai-completions` API, restoring multi-turn thinking/tool-call sessions ([#4505](https://github.com/earendil-works/pi/issues/4505)).
- Fixed OpenCode Go Kimi reasoning replay by normalizing streamed `reasoning` fields back to `reasoning_content` for OpenCode Go only ([#4251](https://github.com/earendil-works/pi/issues/4251)).
### Removed
- Removed non-working OpenAI Codex fast model variants.
## [0.75.0] - 2026-05-17
### Breaking Changes
- Raised the minimum supported Node.js version to 22.19.0.
### Fixed
- Fixed OpenAI Codex generated model metadata to use the current upstream model list ([#4603](https://github.com/earendil-works/pi-mono/pull/4603) by [@mattiacerutti](https://github.com/mattiacerutti)).
- Fixed GitHub Copilot GPT model thinking metadata to map unsupported minimal thinking to low ([#4622](https://github.com/earendil-works/pi-mono/pull/4622) by [@mattiacerutti](https://github.com/mattiacerutti)).
- Fixed `streamSimple()` defaults for models whose advertised output limit is effectively their full context window to avoid impossible default requests ([#4614](https://github.com/earendil-works/pi/issues/4614)).
## [0.74.1] - 2026-05-16
### Added
- Added image generation APIs, image model metadata, and built-in OpenRouter image generation support ([#3887](https://github.com/earendil-works/pi-mono/pull/3887) by [@cristinaponcela](https://github.com/cristinaponcela)).
- Added Together AI as a built-in OpenAI-compatible provider with generated model metadata and `TOGETHER_API_KEY` authentication ([#3624](https://github.com/earendil-works/pi-mono/pull/3624) by [@Nutlope](https://github.com/Nutlope)).
### Fixed
- Fixed GitHub Copilot model availability to ignore generic `GH_TOKEN` and `GITHUB_TOKEN` environment variables, requiring OAuth login or `COPILOT_GITHUB_TOKEN` instead ([#4485](https://github.com/earendil-works/pi/issues/4485)).
- Fixed `openai-completions` streams to surface an error when the stream ends before any terminal `finish_reason`, so truncated responses can retry instead of being accepted as success ([#4345](https://github.com/earendil-works/pi/issues/4345)).
- Fixed Fireworks provider caching compatibility by adding session affinity headers and model metadata compat settings ([#4358](https://github.com/earendil-works/pi-mono/pull/4358) by [@yanirz](https://github.com/yanirz)).
- Fixed OpenAI Codex WebSocket transport to respect proxy environment variables under Bun ([#4354](https://github.com/earendil-works/pi-mono/pull/4354) by [@haoqixu](https://github.com/haoqixu)).
- Fixed OpenRouter cache usage normalization to preserve cached-token semantics without treating cached tokens as cache writes.
- Fixed Bedrock proxy handling to preserve `NO_PROXY` exclusions while using HTTP(S)-only proxy agents.
- Fixed compiled Bun binaries failing to start outside the repo when Bedrock proxy support tried to resolve `proxy-from-env` from external `node_modules` ([#4513](https://github.com/earendil-works/pi/issues/4513)).
- Fixed GitHub Copilot Claude test coverage to use the current Claude Sonnet 4.6 model ID.
- Fixed OpenAI Responses requests for models that support disabling reasoning to send `reasoning.effort: "none"` when thinking is off.
- Fixed Inception Mercury 2 tool calling on OpenRouter by marking `off` as unsupported in `thinkingLevelMap`, so the openai-completions provider omits the reasoning param instead of defaulting to `{reasoning:{effort:"none"}}` (which puts Mercury 2 in instant mode, disabling tool calls).
- Fixed OpenAI Codex SSE retries to honor `retry-after-ms` and `retry-after` headers before falling back to exponential backoff.
- Fixed context overflow detection for LiteLLM-wrapped OpenAI-compatible errors using `exceeds the model's maximum context length of ... tokens` wording ([#4563](https://github.com/earendil-works/pi/issues/4563)).
- Fixed `streamSimple()` defaults to respect model output limits above 32000 tokens instead of clamping provider requests to 32000 ([#4539](https://github.com/earendil-works/pi/issues/4539)).
## [0.74.0] - 2026-05-07

View File

@@ -1122,7 +1122,7 @@ In Node.js environments, you can set environment variables to avoid passing API
| Xiaomi MiMo Token Plan (China) | `XIAOMI_TOKEN_PLAN_CN_API_KEY` |
| Xiaomi MiMo Token Plan (Amsterdam) | `XIAOMI_TOKEN_PLAN_AMS_API_KEY` |
| Xiaomi MiMo Token Plan (Singapore) | `XIAOMI_TOKEN_PLAN_SGP_API_KEY` |
| GitHub Copilot | `COPILOT_GITHUB_TOKEN` or `GH_TOKEN` or `GITHUB_TOKEN` |
| GitHub Copilot | `COPILOT_GITHUB_TOKEN` |
When set, the library automatically uses these keys:

View File

@@ -1,6 +1,6 @@
{
"name": "@earendil-works/pi-ai",
"version": "0.74.0",
"version": "0.75.4",
"description": "Unified LLM API with automatic model discovery and provider configuration",
"type": "module",
"main": "./dist/index.js",
@@ -60,26 +60,22 @@
],
"scripts": {
"clean": "shx rm -rf dist",
"generate-models": "npx tsx scripts/generate-models.ts",
"generate-image-models": "npx tsx scripts/generate-image-models.ts",
"generate-models": "node scripts/generate-models.ts",
"generate-image-models": "node scripts/generate-image-models.ts",
"build": "npm run generate-models && npm run generate-image-models && tsgo -p tsconfig.build.json",
"dev": "tsgo -p tsconfig.build.json --watch --preserveWatchOutput",
"dev:tsc": "tsgo -p tsconfig.build.json --watch --preserveWatchOutput",
"test": "vitest --run",
"prepublishOnly": "npm run clean && npm run build"
},
"dependencies": {
"@anthropic-ai/sdk": "^0.91.1",
"@aws-sdk/client-bedrock-runtime": "^3.1030.0",
"@google/genai": "^1.40.0",
"@mistralai/mistralai": "^2.2.0",
"typebox": "^1.1.24",
"chalk": "^5.6.2",
"@anthropic-ai/sdk": "0.91.1",
"@aws-sdk/client-bedrock-runtime": "3.1048.0",
"@google/genai": "1.52.0",
"@mistralai/mistralai": "2.2.1",
"http-proxy-agent": "7.0.2",
"https-proxy-agent": "7.0.6",
"openai": "6.26.0",
"partial-json": "^0.1.7",
"proxy-agent": "^6.5.0",
"undici": "^7.19.1",
"zod-to-json-schema": "^3.24.6"
"partial-json": "0.1.7",
"typebox": "1.1.38"
},
"keywords": [
"ai",
@@ -99,11 +95,11 @@
"directory": "packages/ai"
},
"engines": {
"node": ">=20.0.0"
"node": ">=22.19.0"
},
"devDependencies": {
"@types/node": "^24.3.0",
"canvas": "^3.2.0",
"vitest": "^3.2.4"
"@types/node": "24.12.4",
"canvas": "3.2.3",
"vitest": "3.2.4"
}
}

View File

@@ -1,9 +1,9 @@
#!/usr/bin/env tsx
#!/usr/bin/env node
import { writeFileSync } from "fs";
import { dirname, join } from "path";
import { fileURLToPath } from "url";
import type { ImagesModel } from "../src/types.js";
import type { ImagesModel } from "../src/types.ts";
const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
@@ -109,7 +109,7 @@ function generateImageModelsFile(models: ImagesModel<"openrouter-images">[]): st
return `// This file is auto-generated by scripts/generate-image-models.ts
// Do not edit manually - run 'npm run generate-image-models' to update
import type { ImagesApi, ImagesModel } from "./types.js";
import type { ImagesApi, ImagesModel } from "./types.ts";
export const IMAGE_MODELS = {
${providerEntries}

View File

@@ -1,4 +1,4 @@
#!/usr/bin/env tsx
#!/usr/bin/env node
import { writeFileSync } from "fs";
import { join, dirname } from "path";
@@ -8,14 +8,8 @@ import {
CLOUDFLARE_AI_GATEWAY_COMPAT_BASE_URL,
CLOUDFLARE_AI_GATEWAY_OPENAI_BASE_URL,
CLOUDFLARE_WORKERS_AI_BASE_URL,
} from "../src/providers/cloudflare.js";
import {
Api,
type AnthropicMessagesCompat,
KnownProvider,
Model,
type OpenAICompletionsCompat,
} from "../src/types.js";
} from "../src/providers/cloudflare.ts";
import type { AnthropicMessagesCompat, Api, KnownProvider, Model, OpenAICompletionsCompat } from "../src/types.ts";
const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
@@ -203,6 +197,9 @@ function applyThinkingLevelMetadata(model: Model<any>): void {
) {
mergeThinkingLevelMap(model, { off: null });
}
if (model.provider === "github-copilot" && model.id.startsWith("gpt-5")) {
mergeThinkingLevelMap(model, { minimal: "low" });
}
if (
model.api === "openai-responses" &&
model.provider === "openai" &&
@@ -237,8 +234,12 @@ function applyThinkingLevelMetadata(model: Model<any>): void {
if (model.provider === "openai-codex" && supportsOpenAiXhigh(model.id)) {
mergeThinkingLevelMap(model, { minimal: "low" });
}
if (model.provider === "openai-codex" && model.id === "gpt-5.1-codex-mini") {
mergeThinkingLevelMap(model, { minimal: "medium", low: "medium", medium: "medium", high: "high" });
if (model.provider === "openrouter" && model.id.startsWith("inception/mercury-2")) {
// Mercury 2 in instant mode (reasoning_effort: "none") disables tool calling.
// Mark "off" unsupported so the openai-completions provider omits the reasoning param
// instead of defaulting to {reasoning:{effort:"none"}} (see openai-completions.ts:575).
// Pi's low/medium/high pass through verbatim; OpenRouter normalizes to Mercury's vocabulary.
mergeThinkingLevelMap(model, { off: null });
}
}
@@ -764,6 +765,16 @@ async function loadModelsDevData(): Promise<Model<any>[]> {
},
contextWindow: m.limit?.context || 4096,
maxTokens: m.limit?.output || 4096,
// Fireworks prompt caching uses automatic prefix matching + session affinity.
// x-session-affinity routes requests to the same replica for cache hits.
// cache_control on tools and eager_input_streaming are not supported.
// See: https://docs.fireworks.ai/tools-sdks/anthropic-compatibility
compat: {
sendSessionAffinityHeaders: true,
supportsEagerToolInputStreaming: false,
supportsCacheControlOnTools: false,
supportsLongCacheRetention: false,
},
});
}
}
@@ -1056,11 +1067,15 @@ async function loadModelsDevData(): Promise<Model<any>[]> {
// Built-in `xiaomi` targets the API billing endpoint (single stable URL,
// keys from platform.xiaomimimo.com). The three `xiaomi-token-plan-*`
// providers cover prepaid Token Plan endpoints in cn / ams / sgp.
const xiaomiCompat: OpenAICompletionsCompat = {
requiresReasoningContentOnAssistantMessages: true,
thinkingFormat: "deepseek",
};
const xiaomiVariants = [
{ provider: "xiaomi", baseUrl: "https://api.xiaomimimo.com/anthropic" },
{ provider: "xiaomi-token-plan-cn", baseUrl: "https://token-plan-cn.xiaomimimo.com/anthropic" },
{ provider: "xiaomi-token-plan-ams", baseUrl: "https://token-plan-ams.xiaomimimo.com/anthropic" },
{ provider: "xiaomi-token-plan-sgp", baseUrl: "https://token-plan-sgp.xiaomimimo.com/anthropic" },
{ provider: "xiaomi", baseUrl: "https://api.xiaomimimo.com/v1" },
{ provider: "xiaomi-token-plan-cn", baseUrl: "https://token-plan-cn.xiaomimimo.com/v1" },
{ provider: "xiaomi-token-plan-ams", baseUrl: "https://token-plan-ams.xiaomimimo.com/v1" },
{ provider: "xiaomi-token-plan-sgp", baseUrl: "https://token-plan-sgp.xiaomimimo.com/v1" },
] as const;
if (data.xiaomi?.models) {
@@ -1072,9 +1087,10 @@ async function loadModelsDevData(): Promise<Model<any>[]> {
models.push({
id: modelId,
name: m.name || modelId,
api: "anthropic-messages",
api: "openai-completions",
provider,
baseUrl,
compat: xiaomiCompat,
reasoning: m.reasoning === true,
input: m.modalities?.input?.includes("image") ? ["text", "image"] : ["text"],
cost: {
@@ -1480,42 +1496,6 @@ async function generateModels() {
const CODEX_CONTEXT = 272000;
const CODEX_MAX_TOKENS = 128000;
const codexModels: Model<"openai-codex-responses">[] = [
{
id: "gpt-5.1",
name: "GPT-5.1",
api: "openai-codex-responses",
provider: "openai-codex",
baseUrl: CODEX_BASE_URL,
reasoning: true,
input: ["text", "image"],
cost: { input: 1.25, output: 10, cacheRead: 0.125, cacheWrite: 0 },
contextWindow: CODEX_CONTEXT,
maxTokens: CODEX_MAX_TOKENS,
},
{
id: "gpt-5.1-codex-max",
name: "GPT-5.1 Codex Max",
api: "openai-codex-responses",
provider: "openai-codex",
baseUrl: CODEX_BASE_URL,
reasoning: true,
input: ["text", "image"],
cost: { input: 1.25, output: 10, cacheRead: 0.125, cacheWrite: 0 },
contextWindow: CODEX_CONTEXT,
maxTokens: CODEX_MAX_TOKENS,
},
{
id: "gpt-5.1-codex-mini",
name: "GPT-5.1 Codex Mini",
api: "openai-codex-responses",
provider: "openai-codex",
baseUrl: CODEX_BASE_URL,
reasoning: true,
input: ["text", "image"],
cost: { input: 0.25, output: 2, cacheRead: 0.025, cacheWrite: 0 },
contextWindow: CODEX_CONTEXT,
maxTokens: CODEX_MAX_TOKENS,
},
{
id: "gpt-5.2",
name: "GPT-5.2",
@@ -1528,18 +1508,6 @@ async function generateModels() {
contextWindow: CODEX_CONTEXT,
maxTokens: CODEX_MAX_TOKENS,
},
{
id: "gpt-5.2-codex",
name: "GPT-5.2 Codex",
api: "openai-codex-responses",
provider: "openai-codex",
baseUrl: CODEX_BASE_URL,
reasoning: true,
input: ["text", "image"],
cost: { input: 1.75, output: 14, cacheRead: 0.175, cacheWrite: 0 },
contextWindow: CODEX_CONTEXT,
maxTokens: CODEX_MAX_TOKENS,
},
{
id: "gpt-5.3-codex",
name: "GPT-5.3 Codex",
@@ -1552,6 +1520,18 @@ async function generateModels() {
contextWindow: CODEX_CONTEXT,
maxTokens: CODEX_MAX_TOKENS,
},
{
id: "gpt-5.3-codex-spark",
name: "GPT-5.3 Codex Spark",
api: "openai-codex-responses",
provider: "openai-codex",
baseUrl: CODEX_BASE_URL,
reasoning: true,
input: ["text"],
cost: { input: 1.75, output: 14, cacheRead: 0.175, cacheWrite: 0 },
contextWindow: CODEX_CONTEXT,
maxTokens: CODEX_MAX_TOKENS,
},
{
id: "gpt-5.4",
name: "GPT-5.4",
@@ -1564,6 +1544,18 @@ async function generateModels() {
contextWindow: CODEX_CONTEXT,
maxTokens: CODEX_MAX_TOKENS,
},
{
id: "gpt-5.4-mini",
name: "GPT-5.4 mini",
api: "openai-codex-responses",
provider: "openai-codex",
baseUrl: CODEX_BASE_URL,
reasoning: true,
input: ["text", "image"],
cost: { input: 0.75, output: 4.5, cacheRead: 0.075, cacheWrite: 0 },
contextWindow: CODEX_CONTEXT,
maxTokens: CODEX_MAX_TOKENS,
},
{
id: "gpt-5.5",
name: "GPT-5.5",
@@ -1576,36 +1568,36 @@ async function generateModels() {
contextWindow: CODEX_CONTEXT,
maxTokens: CODEX_MAX_TOKENS,
},
{
id: "gpt-5.4-mini",
name: "GPT-5.4 Mini",
api: "openai-codex-responses",
provider: "openai-codex",
baseUrl: CODEX_BASE_URL,
reasoning: true,
input: ["text", "image"],
cost: { input: 0.75, output: 4.5, cacheRead: 0.075, cacheWrite: 0 },
contextWindow: CODEX_CONTEXT,
maxTokens: CODEX_MAX_TOKENS,
},
{
id: "gpt-5.3-codex-spark",
name: "GPT-5.3 Codex Spark",
api: "openai-codex-responses",
provider: "openai-codex",
baseUrl: CODEX_BASE_URL,
reasoning: true,
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 128000,
maxTokens: CODEX_MAX_TOKENS,
},
];
allModels.push(...codexModels);
// Add missing Grok models
if (!allModels.some(m => m.provider === "xai" && m.id === "grok-code-fast-1")) {
allModels.push({
const missingGrokModels: Model<"openai-completions">[] = [
{
id: "grok-3",
name: "Grok 3",
api: "openai-completions",
baseUrl: "https://api.x.ai/v1",
provider: "xai",
reasoning: false,
input: ["text"],
cost: { input: 3, output: 15, cacheRead: 0.75, cacheWrite: 0 },
contextWindow: 131072,
maxTokens: 8192,
},
{
id: "grok-3-fast",
name: "Grok 3 Fast",
api: "openai-completions",
baseUrl: "https://api.x.ai/v1",
provider: "xai",
reasoning: false,
input: ["text"],
cost: { input: 5, output: 25, cacheRead: 1.25, cacheWrite: 0 },
contextWindow: 131072,
maxTokens: 8192,
},
{
id: "grok-code-fast-1",
name: "Grok Code Fast 1",
api: "openai-completions",
@@ -1621,7 +1613,12 @@ async function generateModels() {
},
contextWindow: 32768,
maxTokens: 8192,
});
},
];
for (const model of missingGrokModels) {
if (!allModels.some(m => m.provider === model.provider && m.id === model.id)) {
allModels.push(model);
}
}
// Add missing Mistral Medium 3.5 model until models.dev includes it
@@ -1860,7 +1857,7 @@ async function generateModels() {
let output = `// This file is auto-generated by scripts/generate-models.ts
// Do not edit manually - run 'npm run generate-models' to update
import type { Model } from "./types.js";
import type { Model } from "./types.ts";
export const MODELS = {
`;

View File

@@ -1,7 +1,7 @@
#!/usr/bin/env tsx
#!/usr/bin/env node
import { createCanvas } from "canvas";
import { writeFileSync } from "fs";
import { mkdirSync, writeFileSync } from "fs";
import { join, dirname } from "path";
import { fileURLToPath } from "url";
@@ -27,7 +27,6 @@ const buffer = canvas.toBuffer("image/png");
const outputPath = join(__dirname, "..", "test", "data", "red-circle.png");
// Ensure the directory exists
import { mkdirSync } from "fs";
mkdirSync(join(__dirname, "..", "test", "data"), { recursive: true });
writeFileSync(outputPath, buffer);

View File

@@ -6,7 +6,7 @@ import type {
SimpleStreamOptions,
StreamFunction,
StreamOptions,
} from "./types.js";
} from "./types.ts";
export type ApiStreamFunction = (
model: Model<Api>,

View File

@@ -1,4 +1,4 @@
import { streamBedrock, streamSimpleBedrock } from "./providers/amazon-bedrock.js";
import { streamBedrock, streamSimpleBedrock } from "./providers/amazon-bedrock.ts";
export const bedrockProviderModule = {
streamBedrock,

View File

@@ -2,8 +2,8 @@
import { createInterface } from "node:readline";
import { existsSync, readFileSync, writeFileSync } from "fs";
import { getOAuthProvider, getOAuthProviders } from "./utils/oauth/index.js";
import type { OAuthCredentials, OAuthProviderId } from "./utils/oauth/types.js";
import { getOAuthProvider, getOAuthProviders } from "./utils/oauth/index.ts";
import type { OAuthCredentials, OAuthProviderId } from "./utils/oauth/types.ts";
const AUTH_FILE = "auth.json";
const PROVIDERS = getOAuthProviders();

View File

@@ -1,4 +1,4 @@
// NEVER convert to top-level imports - breaks browser/Vite builds (web-ui)
// NEVER convert to top-level imports - breaks browser/Vite builds
let _existsSync: typeof import("node:fs").existsSync | null = null;
let _homedir: typeof import("node:os").homedir | null = null;
let _join: typeof import("node:path").join | null = null;
@@ -23,7 +23,7 @@ if (typeof process !== "undefined" && (process.versions?.node || process.version
});
}
import type { KnownProvider } from "./types.js";
import type { KnownProvider } from "./types.ts";
let _procEnvCache: Map<string, string> | null = null;

View File

@@ -1,7 +1,7 @@
// This file is auto-generated by scripts/generate-image-models.ts
// Do not edit manually - run 'npm run generate-image-models' to update
import type { ImagesApi, ImagesModel } from "./types.js";
import type { ImagesApi, ImagesModel } from "./types.ts";
export const IMAGE_MODELS = {
"openrouter": {
@@ -230,6 +230,171 @@ export const IMAGE_MODELS = {
"cacheWrite": 0
}
} satisfies ImagesModel<"openrouter-images">,
"recraft/recraft-v3": {
id: "recraft/recraft-v3",
name: "Recraft: Recraft V3",
api: "openrouter-images",
provider: "openrouter",
baseUrl: "https://openrouter.ai/api/v1",
input: ["text", "image"],
output: ["image"],
cost: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
},
} satisfies ImagesModel<"openrouter-images">,
"recraft/recraft-v4": {
id: "recraft/recraft-v4",
name: "Recraft: Recraft V4",
api: "openrouter-images",
provider: "openrouter",
baseUrl: "https://openrouter.ai/api/v1",
input: ["text", "image"],
output: ["image"],
cost: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
},
} satisfies ImagesModel<"openrouter-images">,
"recraft/recraft-v4-pro": {
id: "recraft/recraft-v4-pro",
name: "Recraft: Recraft V4 Pro",
api: "openrouter-images",
provider: "openrouter",
baseUrl: "https://openrouter.ai/api/v1",
input: ["text", "image"],
output: ["image"],
cost: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
},
} satisfies ImagesModel<"openrouter-images">,
"recraft/recraft-v4-pro-vector": {
id: "recraft/recraft-v4-pro-vector",
name: "Recraft: Recraft V4 Pro Vector",
api: "openrouter-images",
provider: "openrouter",
baseUrl: "https://openrouter.ai/api/v1",
input: ["text", "image"],
output: ["image"],
cost: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
},
} satisfies ImagesModel<"openrouter-images">,
"recraft/recraft-v4-vector": {
id: "recraft/recraft-v4-vector",
name: "Recraft: Recraft V4 Vector",
api: "openrouter-images",
provider: "openrouter",
baseUrl: "https://openrouter.ai/api/v1",
input: ["text", "image"],
output: ["image"],
cost: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
},
} satisfies ImagesModel<"openrouter-images">,
"recraft/recraft-v4.1": {
id: "recraft/recraft-v4.1",
name: "Recraft: Recraft V4.1",
api: "openrouter-images",
provider: "openrouter",
baseUrl: "https://openrouter.ai/api/v1",
input: ["text", "image"],
output: ["image"],
cost: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
},
} satisfies ImagesModel<"openrouter-images">,
"recraft/recraft-v4.1-pro": {
id: "recraft/recraft-v4.1-pro",
name: "Recraft: Recraft V4.1 Pro",
api: "openrouter-images",
provider: "openrouter",
baseUrl: "https://openrouter.ai/api/v1",
input: ["text", "image"],
output: ["image"],
cost: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
},
} satisfies ImagesModel<"openrouter-images">,
"recraft/recraft-v4.1-pro-vector": {
id: "recraft/recraft-v4.1-pro-vector",
name: "Recraft: Recraft V4.1 Pro Vector",
api: "openrouter-images",
provider: "openrouter",
baseUrl: "https://openrouter.ai/api/v1",
input: ["text", "image"],
output: ["image"],
cost: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
},
} satisfies ImagesModel<"openrouter-images">,
"recraft/recraft-v4.1-utility": {
id: "recraft/recraft-v4.1-utility",
name: "Recraft: Recraft V4.1 Utility",
api: "openrouter-images",
provider: "openrouter",
baseUrl: "https://openrouter.ai/api/v1",
input: ["text", "image"],
output: ["image"],
cost: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
},
} satisfies ImagesModel<"openrouter-images">,
"recraft/recraft-v4.1-utility-pro": {
id: "recraft/recraft-v4.1-utility-pro",
name: "Recraft: Recraft V4.1 Utility Pro",
api: "openrouter-images",
provider: "openrouter",
baseUrl: "https://openrouter.ai/api/v1",
input: ["text", "image"],
output: ["image"],
cost: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
},
} satisfies ImagesModel<"openrouter-images">,
"recraft/recraft-v4.1-vector": {
id: "recraft/recraft-v4.1-vector",
name: "Recraft: Recraft V4.1 Vector",
api: "openrouter-images",
provider: "openrouter",
baseUrl: "https://openrouter.ai/api/v1",
input: ["text", "image"],
output: ["image"],
cost: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
},
} satisfies ImagesModel<"openrouter-images">,
"sourceful/riverflow-v2-fast": {
id: "sourceful/riverflow-v2-fast",
name: "Sourceful: Riverflow V2 Fast",
@@ -305,5 +470,20 @@ export const IMAGE_MODELS = {
"cacheWrite": 0
}
} satisfies ImagesModel<"openrouter-images">,
"x-ai/grok-imagine-image-quality": {
id: "x-ai/grok-imagine-image-quality",
name: "xAI: Grok Imagine Image Quality",
api: "openrouter-images",
provider: "openrouter",
baseUrl: "https://openrouter.ai/api/v1",
input: ["text", "image"],
output: ["image"],
cost: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
},
} satisfies ImagesModel<"openrouter-images">,
},
} as const satisfies Record<string, Record<string, ImagesModel<ImagesApi>>>;

View File

@@ -1,5 +1,5 @@
import { IMAGE_MODELS } from "./image-models.generated.js";
import type { ImagesApi, ImagesModel, KnownImagesProvider } from "./types.js";
import { IMAGE_MODELS } from "./image-models.generated.ts";
import type { ImagesApi, ImagesModel, KnownImagesProvider } from "./types.ts";
const imageModelRegistry: Map<string, Map<string, ImagesModel<ImagesApi>>> = new Map();

View File

@@ -1,4 +1,4 @@
import type { AssistantImages, ImagesApi, ImagesContext, ImagesFunction, ImagesModel, ImagesOptions } from "./types.js";
import type { AssistantImages, ImagesApi, ImagesContext, ImagesFunction, ImagesModel, ImagesOptions } from "./types.ts";
export type ImagesApiFunction = (
model: ImagesModel<ImagesApi>,

View File

@@ -1,7 +1,7 @@
import "./providers/images/register-builtins.js";
import "./providers/images/register-builtins.ts";
import { getImagesApiProvider } from "./images-api-registry.js";
import type { AssistantImages, ImagesApi, ImagesContext, ImagesModel, ProviderImagesOptions } from "./types.js";
import { getImagesApiProvider } from "./images-api-registry.ts";
import type { AssistantImages, ImagesApi, ImagesContext, ImagesModel, ProviderImagesOptions } from "./types.ts";
function resolveImagesApiProvider(api: ImagesApi) {
const provider = getImagesApiProvider(api);

View File

@@ -1,34 +1,34 @@
export type { Static, TSchema } from "typebox";
export { Type } from "typebox";
export * from "./api-registry.js";
export * from "./env-api-keys.js";
export * from "./image-models.js";
export * from "./images.js";
export * from "./images-api-registry.js";
export * from "./models.js";
export type { BedrockOptions, BedrockThinkingDisplay } from "./providers/amazon-bedrock.js";
export type { AnthropicEffort, AnthropicOptions, AnthropicThinkingDisplay } from "./providers/anthropic.js";
export type { AzureOpenAIResponsesOptions } from "./providers/azure-openai-responses.js";
export * from "./providers/faux.js";
export type { GoogleOptions } from "./providers/google.js";
export type { GoogleThinkingLevel } from "./providers/google-shared.js";
export type { GoogleVertexOptions } from "./providers/google-vertex.js";
export * from "./providers/images/register-builtins.js";
export type { MistralOptions } from "./providers/mistral.js";
export * from "./api-registry.ts";
export * from "./env-api-keys.ts";
export * from "./image-models.ts";
export * from "./images.ts";
export * from "./images-api-registry.ts";
export * from "./models.ts";
export type { BedrockOptions, BedrockThinkingDisplay } from "./providers/amazon-bedrock.ts";
export type { AnthropicEffort, AnthropicOptions, AnthropicThinkingDisplay } from "./providers/anthropic.ts";
export type { AzureOpenAIResponsesOptions } from "./providers/azure-openai-responses.ts";
export * from "./providers/faux.ts";
export type { GoogleOptions } from "./providers/google.ts";
export type { GoogleThinkingLevel } from "./providers/google-shared.ts";
export type { GoogleVertexOptions } from "./providers/google-vertex.ts";
export * from "./providers/images/register-builtins.ts";
export type { MistralOptions } from "./providers/mistral.ts";
export type {
OpenAICodexResponsesOptions,
OpenAICodexWebSocketDebugStats,
} from "./providers/openai-codex-responses.js";
export type { OpenAICompletionsOptions } from "./providers/openai-completions.js";
export type { OpenAIResponsesOptions } from "./providers/openai-responses.js";
export * from "./providers/register-builtins.js";
export * from "./session-resources.js";
export * from "./stream.js";
export * from "./types.js";
export * from "./utils/diagnostics.js";
export * from "./utils/event-stream.js";
export * from "./utils/json-parse.js";
} from "./providers/openai-codex-responses.ts";
export type { OpenAICompletionsOptions } from "./providers/openai-completions.ts";
export type { OpenAIResponsesOptions } from "./providers/openai-responses.ts";
export * from "./providers/register-builtins.ts";
export * from "./session-resources.ts";
export * from "./stream.ts";
export * from "./types.ts";
export * from "./utils/diagnostics.ts";
export * from "./utils/event-stream.ts";
export * from "./utils/json-parse.ts";
export type {
OAuthAuthInfo,
OAuthCredentials,
@@ -40,7 +40,7 @@ export type {
OAuthProviderInterface,
OAuthSelectOption,
OAuthSelectPrompt,
} from "./utils/oauth/types.js";
export * from "./utils/overflow.js";
export * from "./utils/typebox-helpers.js";
export * from "./utils/validation.js";
} from "./utils/oauth/types.ts";
export * from "./utils/overflow.ts";
export * from "./utils/typebox-helpers.ts";
export * from "./utils/validation.ts";

File diff suppressed because it is too large Load Diff

View File

@@ -1,5 +1,5 @@
import { MODELS } from "./models.generated.js";
import type { Api, KnownProvider, Model, ModelThinkingLevel, Usage } from "./types.js";
import { MODELS } from "./models.generated.ts";
import type { Api, KnownProvider, Model, ModelThinkingLevel, Usage } from "./types.ts";
const modelRegistry: Map<string, Map<string, Model<Api>>> = new Map();

View File

@@ -1 +1 @@
export * from "./utils/oauth/index.js";
export * from "./utils/oauth/index.ts";

View File

@@ -20,8 +20,9 @@ import {
type ToolConfiguration,
ToolResultStatus,
} from "@aws-sdk/client-bedrock-runtime";
import { NodeHttpHandler } from "@smithy/node-http-handler";
import type { DocumentType } from "@smithy/types";
import { calculateCost } from "../models.js";
import { calculateCost } from "../models.ts";
import type {
Api,
AssistantMessage,
@@ -39,12 +40,13 @@ import type {
Tool,
ToolCall,
ToolResultMessage,
} from "../types.js";
import { AssistantMessageEventStream } from "../utils/event-stream.js";
import { parseStreamingJson } from "../utils/json-parse.js";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.js";
import { adjustMaxTokensForThinking, buildBaseOptions, clampReasoning } from "./simple-options.js";
import { transformMessages } from "./transform-messages.js";
} from "../types.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { parseStreamingJson } from "../utils/json-parse.ts";
import { createHttpProxyAgentsForTarget } from "../utils/node-http-proxy.ts";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts";
import { adjustMaxTokensForThinking, buildBaseOptions, clampReasoning } from "./simple-options.ts";
import { transformMessages } from "./transform-messages.ts";
export type BedrockThinkingDisplay = "summarized" | "omitted";
@@ -156,30 +158,15 @@ export const streamBedrock: StreamFunction<"bedrock-converse-stream", BedrockOpt
};
}
if (
process.env.HTTP_PROXY ||
process.env.HTTPS_PROXY ||
process.env.NO_PROXY ||
process.env.http_proxy ||
process.env.https_proxy ||
process.env.no_proxy
) {
const nodeHttpHandler = await import("@smithy/node-http-handler");
const proxyAgent = await import("proxy-agent");
const agent = new proxyAgent.ProxyAgent();
const proxyAgents = createHttpProxyAgentsForTarget(model.baseUrl);
if (proxyAgents) {
// Bedrock runtime uses NodeHttp2Handler by default since v3.798.0, which is based
// on `http2` module and has no support for http agent.
// Use NodeHttpHandler to support http agent.
config.requestHandler = new nodeHttpHandler.NodeHttpHandler({
httpAgent: agent,
httpsAgent: agent,
});
// Use NodeHttpHandler to support HTTP(S) proxy agents.
config.requestHandler = new NodeHttpHandler(proxyAgents);
} else if (process.env.AWS_BEDROCK_FORCE_HTTP1 === "1") {
// Some custom endpoints require HTTP/1.1 instead of HTTP/2
const nodeHttpHandler = await import("@smithy/node-http-handler");
config.requestHandler = new nodeHttpHandler.NodeHttpHandler();
config.requestHandler = new NodeHttpHandler();
}
} else {
// Non-Node environment (browser): fall back to us-east-1 since
@@ -196,12 +183,13 @@ export const streamBedrock: StreamFunction<"bedrock-converse-stream", BedrockOpt
try {
const client = new BedrockRuntimeClient(config);
const cacheRetention = resolveCacheRetention(options.cacheRetention);
const inferenceMaxTokens = options.maxTokens ?? (isAnthropicClaudeModel(model) ? model.maxTokens : undefined);
let commandInput = {
modelId: model.id,
messages: convertMessages(context, model, cacheRetention),
system: buildSystemPrompt(context.systemPrompt, model, cacheRetention),
inferenceConfig: {
...(options.maxTokens !== undefined && { maxTokens: options.maxTokens }),
...(inferenceMaxTokens !== undefined && { maxTokens: inferenceMaxTokens }),
...(options.temperature !== undefined && { temperature: options.temperature }),
},
toolConfig: convertToolConfig(context.tools, options.toolChoice),
@@ -327,8 +315,10 @@ export const streamSimpleBedrock: StreamFunction<"bedrock-converse-stream", Simp
} satisfies BedrockOptions);
}
// Undefined means the caller did not request an output cap; let the helper use the model cap.
// Do not coerce to 0 here, or the thinking budget would become the entire maxTokens value.
const adjusted = adjustMaxTokensForThinking(
base.maxTokens || 0,
base.maxTokens,
model.maxTokens,
options.reasoning,
options.thinkingBudgets,
@@ -631,24 +621,31 @@ function convertMessages(
const m = transformedMessages[i];
switch (m.role) {
case "user":
case "user": {
const content: ContentBlock[] = [];
if (typeof m.content === "string") {
content.push({ text: sanitizeSurrogates(m.content) });
} else {
for (const c of m.content) {
switch (c.type) {
case "text":
content.push({ text: sanitizeSurrogates(c.text) });
break;
case "image":
content.push({ image: createImageBlock(c.mimeType, c.data) });
break;
default:
continue;
}
}
}
if (content.length === 0) continue;
result.push({
role: ConversationRole.USER,
content:
typeof m.content === "string"
? [{ text: sanitizeSurrogates(m.content) }]
: m.content.map((c) => {
switch (c.type) {
case "text":
return { text: sanitizeSurrogates(c.text) };
case "image":
return { image: createImageBlock(c.mimeType, c.data) };
default:
throw new Error("Unknown user content type");
}
}),
content,
});
break;
}
case "assistant": {
// Skip assistant messages with empty content (e.g., from aborted requests)
// Bedrock rejects messages with empty content arrays
@@ -699,7 +696,7 @@ function convertMessages(
}
break;
default:
throw new Error("Unknown assistant content type");
continue;
}
}
// Skip if all content blocks were filtered out
@@ -758,7 +755,7 @@ function convertMessages(
break;
}
default:
throw new Error("Unknown message role");
continue;
}
}

View File

@@ -6,8 +6,8 @@ import type {
MessageParam,
RawMessageStreamEvent,
} from "@anthropic-ai/sdk/resources/messages.js";
import { getEnvApiKey } from "../env-api-keys.js";
import { calculateCost } from "../models.js";
import { getEnvApiKey } from "../env-api-keys.ts";
import { calculateCost } from "../models.ts";
import type {
AnthropicMessagesCompat,
Api,
@@ -26,16 +26,16 @@ import type {
Tool,
ToolCall,
ToolResultMessage,
} from "../types.js";
import { AssistantMessageEventStream } from "../utils/event-stream.js";
import { headersToRecord } from "../utils/headers.js";
import { parseJsonWithRepair, parseStreamingJson } from "../utils/json-parse.js";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.js";
} from "../types.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { headersToRecord } from "../utils/headers.ts";
import { parseJsonWithRepair, parseStreamingJson } from "../utils/json-parse.ts";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts";
import { resolveCloudflareBaseUrl } from "./cloudflare.js";
import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./github-copilot-headers.js";
import { adjustMaxTokensForThinking, buildBaseOptions } from "./simple-options.js";
import { transformMessages } from "./transform-messages.js";
import { resolveCloudflareBaseUrl } from "./cloudflare.ts";
import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./github-copilot-headers.ts";
import { adjustMaxTokensForThinking, buildBaseOptions } from "./simple-options.ts";
import { transformMessages } from "./transform-messages.ts";
/**
* Resolve cache retention preference.
@@ -165,9 +165,16 @@ const FINE_GRAINED_TOOL_STREAMING_BETA = "fine-grained-tool-streaming-2025-05-14
const INTERLEAVED_THINKING_BETA = "interleaved-thinking-2025-05-14";
function getAnthropicCompat(model: Model<"anthropic-messages">): Required<AnthropicMessagesCompat> {
// Auto-detect session affinity and cache control support from provider
const isFireworks = model.provider === "fireworks";
const isCloudflareAiGatewayAnthropic =
model.provider === "cloudflare-ai-gateway" && model.baseUrl.includes("anthropic");
return {
supportsEagerToolInputStreaming: model.compat?.supportsEagerToolInputStreaming ?? true,
supportsLongCacheRetention: model.compat?.supportsLongCacheRetention ?? true,
supportsEagerToolInputStreaming: model.compat?.supportsEagerToolInputStreaming ?? !isFireworks,
supportsLongCacheRetention: model.compat?.supportsLongCacheRetention ?? !isFireworks,
sendSessionAffinityHeaders:
model.compat?.sendSessionAffinityHeaders ?? !!(isFireworks || isCloudflareAiGatewayAnthropic),
supportsCacheControlOnTools: model.compat?.supportsCacheControlOnTools ?? !isFireworks,
};
}
@@ -463,6 +470,9 @@ export const streamAnthropic: StreamFunction<"anthropic-messages", AnthropicOpti
});
}
const cacheRetention = options?.cacheRetention ?? resolveCacheRetention();
const cacheSessionId = cacheRetention === "none" ? undefined : options?.sessionId;
const created = createClient(
model,
apiKey,
@@ -470,6 +480,7 @@ export const streamAnthropic: StreamFunction<"anthropic-messages", AnthropicOpti
shouldUseFineGrainedToolStreamingBeta(model, context),
options?.headers,
copilotDynamicHeaders,
cacheSessionId,
);
client = created.client;
isOAuth = created.isOAuthToken;
@@ -740,8 +751,10 @@ export const streamSimpleAnthropic: StreamFunction<"anthropic-messages", SimpleS
} satisfies AnthropicOptions);
}
// Undefined means the caller did not request an output cap; let the helper use the model cap.
// Do not coerce to 0 here, or the thinking budget would become the entire max_tokens value.
const adjusted = adjustMaxTokensForThinking(
base.maxTokens || 0,
base.maxTokens,
model.maxTokens,
options.reasoning,
options.thinkingBudgets,
@@ -766,6 +779,7 @@ function createClient(
useFineGrainedToolStreamingBeta: boolean,
optionsHeaders?: Record<string, string>,
dynamicHeaders?: Record<string, string>,
sessionId?: string,
): { client: Anthropic; isOAuthToken: boolean } {
// Adaptive thinking models (Opus 4.6, Sonnet 4.6) have interleaved thinking built-in.
// The beta header is deprecated on Opus 4.6 and redundant on Sonnet 4.6, so skip it.
@@ -847,8 +861,11 @@ function createClient(
}
// API key auth
const sessionAffinityHeaders: Record<string, string | null> =
sessionId && getAnthropicCompat(model).sendSessionAffinityHeaders ? { "x-session-affinity": sessionId } : {};
const client = new Anthropic({
apiKey,
authToken: null,
baseURL: model.baseUrl,
dangerouslyAllowBrowser: true,
defaultHeaders: mergeHeaders(
@@ -857,6 +874,7 @@ function createClient(
"anthropic-dangerous-direct-browser-access": "true",
...(betaFeatures.length > 0 ? { "anthropic-beta": betaFeatures.join(",") } : {}),
},
sessionAffinityHeaders,
model.headers,
optionsHeaders,
),
@@ -875,7 +893,7 @@ function buildParams(
const params: MessageCreateParamsStreaming = {
model: model.id,
messages: convertMessages(context.messages, model, isOAuthToken, cacheControl),
max_tokens: options?.maxTokens || (model.maxTokens / 3) | 0,
max_tokens: options?.maxTokens ?? model.maxTokens,
stream: true,
};
@@ -912,11 +930,12 @@ function buildParams(
}
if (context.tools && context.tools.length > 0) {
const compat = getAnthropicCompat(model);
params.tools = convertTools(
context.tools,
isOAuthToken,
getAnthropicCompat(model).supportsEagerToolInputStreaming,
cacheControl,
compat.supportsEagerToolInputStreaming,
compat.supportsCacheControlOnTools ? cacheControl : undefined,
);
}

View File

@@ -1,7 +1,7 @@
import { AzureOpenAI } from "openai";
import type { ResponseCreateParamsStreaming } from "openai/resources/responses/responses.js";
import { getEnvApiKey } from "../env-api-keys.js";
import { clampThinkingLevel } from "../models.js";
import { getEnvApiKey } from "../env-api-keys.ts";
import { clampThinkingLevel } from "../models.ts";
import type {
Api,
AssistantMessage,
@@ -10,11 +10,12 @@ import type {
SimpleStreamOptions,
StreamFunction,
StreamOptions,
} from "../types.js";
import { AssistantMessageEventStream } from "../utils/event-stream.js";
import { headersToRecord } from "../utils/headers.js";
import { convertResponsesMessages, convertResponsesTools, processResponsesStream } from "./openai-responses-shared.js";
import { buildBaseOptions } from "./simple-options.js";
} from "../types.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { headersToRecord } from "../utils/headers.ts";
import { clampOpenAIPromptCacheKey } from "./openai-prompt-cache.ts";
import { convertResponsesMessages, convertResponsesTools, processResponsesStream } from "./openai-responses-shared.ts";
import { buildBaseOptions } from "./simple-options.ts";
const DEFAULT_AZURE_API_VERSION = "v1";
const AZURE_TOOL_CALL_PROVIDERS = new Set(["openai", "openai-codex", "opencode", "azure-openai-responses"]);
@@ -40,6 +41,22 @@ function resolveDeploymentName(model: Model<"azure-openai-responses">, options?:
return mappedDeployment || model.id;
}
function formatAzureOpenAIError(error: unknown): string {
if (error instanceof Error) {
const status = (error as Error & { status?: unknown }).status;
const statusCode = typeof status === "number" ? status : undefined;
if (statusCode !== undefined) {
return `Azure OpenAI API error (${statusCode}): ${error.message}`;
}
return error.message;
}
try {
return JSON.stringify(error);
} catch {
return String(error);
}
}
// Azure OpenAI Responses-specific options
export interface AzureOpenAIResponsesOptions extends StreamOptions {
reasoningEffort?: "minimal" | "low" | "medium" | "high" | "xhigh";
@@ -119,7 +136,7 @@ export const streamAzureOpenAIResponses: StreamFunction<"azure-openai-responses"
delete (block as { partialJson?: string }).partialJson;
}
output.stopReason = options?.signal?.aborted ? "aborted" : "error";
output.errorMessage = error instanceof Error ? error.message : JSON.stringify(error);
output.errorMessage = formatAzureOpenAIError(error);
stream.push({ type: "error", reason: output.stopReason, error: output });
stream.end();
}
@@ -245,7 +262,7 @@ function buildParams(
model: deploymentName,
input: messages,
stream: true,
prompt_cache_key: options?.sessionId,
prompt_cache_key: clampOpenAIPromptCacheKey(options?.sessionId),
};
if (options?.maxTokens) {

View File

@@ -1,4 +1,4 @@
import type { Api, Model } from "../types.js";
import type { Api, Model } from "../types.ts";
/** Workers AI direct endpoint. */
export const CLOUDFLARE_WORKERS_AI_BASE_URL =

View File

@@ -1,4 +1,4 @@
import { registerApiProvider, unregisterApiProviders } from "../api-registry.js";
import { registerApiProvider, unregisterApiProviders } from "../api-registry.ts";
import type {
AssistantMessage,
AssistantMessageEventStream,
@@ -14,8 +14,8 @@ import type {
ToolCall,
ToolResultMessage,
Usage,
} from "../types.js";
import { createAssistantMessageEventStream } from "../utils/event-stream.js";
} from "../types.ts";
import { createAssistantMessageEventStream } from "../utils/event-stream.ts";
const DEFAULT_API = "faux";
const DEFAULT_PROVIDER = "faux";

View File

@@ -1,4 +1,4 @@
import type { Message } from "../types.js";
import type { Message } from "../types.ts";
// Copilot expects X-Initiator to indicate whether the request is user-initiated
// or agent-initiated (e.g. follow-up after assistant/tool messages).

View File

@@ -3,9 +3,9 @@
*/
import { type Content, FinishReason, FunctionCallingConfigMode, type Part } from "@google/genai";
import type { Context, ImageContent, Model, StopReason, TextContent, Tool } from "../types.js";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.js";
import { transformMessages } from "./transform-messages.js";
import type { Context, ImageContent, Model, StopReason, TextContent, Tool } from "../types.ts";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts";
import { transformMessages } from "./transform-messages.ts";
type GoogleApiType = "google-generative-ai" | "google-vertex";

View File

@@ -7,7 +7,7 @@ import {
type ThinkingConfig,
ThinkingLevel,
} from "@google/genai";
import { calculateCost, clampThinkingLevel } from "../models.js";
import { calculateCost, clampThinkingLevel } from "../models.ts";
import type {
Api,
AssistantMessage,
@@ -21,10 +21,10 @@ import type {
ThinkingBudgets,
ThinkingContent,
ToolCall,
} from "../types.js";
import { AssistantMessageEventStream } from "../utils/event-stream.js";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.js";
import type { GoogleThinkingLevel } from "./google-shared.js";
} from "../types.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts";
import type { GoogleThinkingLevel } from "./google-shared.ts";
import {
convertMessages,
convertTools,
@@ -32,8 +32,8 @@ import {
mapStopReason,
mapToolChoice,
retainThoughtSignature,
} from "./google-shared.js";
import { buildBaseOptions } from "./simple-options.js";
} from "./google-shared.ts";
import { buildBaseOptions } from "./simple-options.ts";
export interface GoogleVertexOptions extends StreamOptions {
toolChoice?: "auto" | "none" | "any";

View File

@@ -4,8 +4,8 @@ import {
GoogleGenAI,
type ThinkingConfig,
} from "@google/genai";
import { getEnvApiKey } from "../env-api-keys.js";
import { calculateCost, clampThinkingLevel } from "../models.js";
import { getEnvApiKey } from "../env-api-keys.ts";
import { calculateCost, clampThinkingLevel } from "../models.ts";
import type {
Api,
AssistantMessage,
@@ -19,10 +19,10 @@ import type {
ThinkingContent,
ThinkingLevel,
ToolCall,
} from "../types.js";
import { AssistantMessageEventStream } from "../utils/event-stream.js";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.js";
import type { GoogleThinkingLevel } from "./google-shared.js";
} from "../types.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts";
import type { GoogleThinkingLevel } from "./google-shared.ts";
import {
convertMessages,
convertTools,
@@ -30,8 +30,8 @@ import {
mapStopReason,
mapToolChoice,
retainThoughtSignature,
} from "./google-shared.js";
import { buildBaseOptions } from "./simple-options.js";
} from "./google-shared.ts";
import { buildBaseOptions } from "./simple-options.ts";
export interface GoogleOptions extends StreamOptions {
toolChoice?: "auto" | "none" | "any";

View File

@@ -6,7 +6,7 @@ import type {
ChatCompletionContentPartText,
ChatCompletionCreateParamsNonStreaming,
} from "openai/resources/chat/completions.js";
import { getEnvApiKey } from "../../env-api-keys.js";
import { getEnvApiKey } from "../../env-api-keys.ts";
import type {
AssistantImages,
ImageContent,
@@ -15,9 +15,9 @@ import type {
ImagesModel,
ImagesOptions,
TextContent,
} from "../../types.js";
import { headersToRecord } from "../../utils/headers.js";
import { sanitizeSurrogates } from "../../utils/sanitize-unicode.js";
} from "../../types.ts";
import { headersToRecord } from "../../utils/headers.ts";
import { sanitizeSurrogates } from "../../utils/sanitize-unicode.ts";
interface OpenRouterGeneratedImage {
image_url?: string | { url?: string };

View File

@@ -1,6 +1,6 @@
import { registerImagesApiProvider } from "../../images-api-registry.js";
import type { AssistantImages, ImagesContext, ImagesFunction, ImagesModel, ImagesOptions } from "../../types.js";
import type { generateImagesOpenRouter as generateImagesOpenRouterFunction } from "./openrouter.js";
import { registerImagesApiProvider } from "../../images-api-registry.ts";
import type { AssistantImages, ImagesContext, ImagesFunction, ImagesModel, ImagesOptions } from "../../types.ts";
import type { generateImagesOpenRouter as generateImagesOpenRouterFunction } from "./openrouter.ts";
interface OpenRouterImagesProviderModule {
generateImagesOpenRouter: typeof generateImagesOpenRouterFunction;
@@ -21,7 +21,7 @@ function createLazyLoadErrorImages(model: ImagesModel<"openrouter-images">, erro
}
function loadOpenRouterImagesProviderModule(): Promise<OpenRouterImagesProviderModule> {
openRouterImagesProviderModulePromise ||= import("./openrouter.js").then(
openRouterImagesProviderModulePromise ||= import("./openrouter.ts").then(
(module) => module as OpenRouterImagesProviderModule,
);
return openRouterImagesProviderModulePromise;

View File

@@ -6,8 +6,8 @@ import type {
ContentChunk,
FunctionTool,
} from "@mistralai/mistralai/models/components";
import { getEnvApiKey } from "../env-api-keys.js";
import { calculateCost, clampThinkingLevel } from "../models.js";
import { getEnvApiKey } from "../env-api-keys.ts";
import { calculateCost, clampThinkingLevel } from "../models.ts";
import type {
AssistantMessage,
Context,
@@ -21,13 +21,13 @@ import type {
ThinkingContent,
Tool,
ToolCall,
} from "../types.js";
import { AssistantMessageEventStream } from "../utils/event-stream.js";
import { shortHash } from "../utils/hash.js";
import { parseStreamingJson } from "../utils/json-parse.js";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.js";
import { buildBaseOptions } from "./simple-options.js";
import { transformMessages } from "./transform-messages.js";
} from "../types.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { shortHash } from "../utils/hash.ts";
import { parseStreamingJson } from "../utils/json-parse.ts";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts";
import { buildBaseOptions } from "./simple-options.ts";
import { transformMessages } from "./transform-messages.ts";
const MISTRAL_TOOL_CALL_ID_LENGTH = 9;
const MAX_MISTRAL_ERROR_BODY_CHARS = 4000;

View File

@@ -6,7 +6,7 @@ import type {
ResponseStreamEvent,
} from "openai/resources/responses/responses.js";
// NEVER convert to top-level runtime imports - breaks browser/Vite builds (web-ui)
// NEVER convert to top-level runtime imports - breaks browser/Vite builds
let _os: typeof NodeOs | null = null;
type DynamicImport = (specifier: string) => Promise<unknown>;
@@ -20,9 +20,9 @@ if (typeof process !== "undefined" && (process.versions?.node || process.version
});
}
import { getEnvApiKey } from "../env-api-keys.js";
import { clampThinkingLevel } from "../models.js";
import { registerSessionResourceCleanup } from "../session-resources.js";
import { getEnvApiKey } from "../env-api-keys.ts";
import { clampThinkingLevel } from "../models.ts";
import { registerSessionResourceCleanup } from "../session-resources.ts";
import type {
Api,
AssistantMessage,
@@ -32,16 +32,17 @@ import type {
StreamFunction,
StreamOptions,
Usage,
} from "../types.js";
} from "../types.ts";
import {
appendAssistantMessageDiagnostic,
createAssistantMessageDiagnostic,
formatThrownValue,
} from "../utils/diagnostics.js";
import { AssistantMessageEventStream } from "../utils/event-stream.js";
import { headersToRecord } from "../utils/headers.js";
import { convertResponsesMessages, convertResponsesTools, processResponsesStream } from "./openai-responses-shared.js";
import { buildBaseOptions } from "./simple-options.js";
} from "../utils/diagnostics.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { headersToRecord } from "../utils/headers.ts";
import { clampOpenAIPromptCacheKey } from "./openai-prompt-cache.ts";
import { convertResponsesMessages, convertResponsesTools, processResponsesStream } from "./openai-responses-shared.ts";
import { buildBaseOptions } from "./simple-options.ts";
// ============================================================================
// Configuration
@@ -254,7 +255,29 @@ export const streamOpenAICodexResponses: StreamFunction<"openai-codex-responses"
const errorText = await response.text();
if (attempt < MAX_RETRIES && isRetryableError(response.status, errorText)) {
const delayMs = BASE_DELAY_MS * 2 ** attempt;
let delayMs = BASE_DELAY_MS * 2 ** attempt;
const retryAfterMs = response.headers.get("retry-after-ms");
if (retryAfterMs !== null) {
const millis = Number(retryAfterMs);
if (Number.isFinite(millis)) {
delayMs = Math.max(0, millis);
}
} else {
const retryAfter = response.headers.get("retry-after");
if (retryAfter) {
const seconds = Number(retryAfter);
if (Number.isFinite(seconds)) {
delayMs = Math.max(0, seconds * 1000);
} else {
const date = Date.parse(retryAfter);
if (!Number.isNaN(date)) {
delayMs = Math.max(0, date - Date.now());
}
}
}
}
await sleep(delayMs, options?.signal);
continue;
}
@@ -356,7 +379,7 @@ function buildRequestBody(
input: messages,
text: { verbosity: options?.textVerbosity || "low" },
include: ["reasoning.encrypted_content"],
prompt_cache_key: options?.sessionId,
prompt_cache_key: clampOpenAIPromptCacheKey(options?.sessionId),
tool_choice: "auto",
parallel_tool_calls: true,
};
@@ -711,7 +734,35 @@ type WebSocketConstructor = new (
protocols?: string | string[] | { headers?: Record<string, string> },
) => WebSocketLike;
function getWebSocketConstructor(): WebSocketConstructor | null {
let _cachedWebsocket: WebSocketConstructor | null = null;
async function getWebSocketConstructor(): Promise<WebSocketConstructor | null> {
if (_cachedWebsocket) return _cachedWebsocket;
// bun doesn't respect http proxy envs, ref: https://github.com/oven-sh/bun/issues/15489
// TODO: remove this when bun supports proxy envs in websocket.
if (
process?.versions?.bun &&
(process.env.HTTP_PROXY || process.env.HTTPS_PROXY || process.env.http_proxy || process.env.https_proxy)
) {
const m = await dynamicImport("proxy-from-env");
const getProxyForUrl = (m as { getProxyForUrl: (url: string | object | URL) => string }).getProxyForUrl;
_cachedWebsocket = class extends WebSocket {
constructor(url: string | URL, options?: string | string[] | Record<string, unknown>) {
let _opts: Record<string, unknown> = {};
if (Array.isArray(options) || typeof options === "string") {
_opts = { protocols: options };
} else {
_opts = { ...options };
}
const proxy = getProxyForUrl(url.toString().replace(/^wss:/, "https:").replace(/^ws:/, "http:"));
super(url, { ..._opts, ...(proxy ? { proxy } : {}) } as any);
}
};
return _cachedWebsocket;
}
const ctor = (globalThis as { WebSocket?: unknown }).WebSocket;
if (typeof ctor !== "function") return null;
return ctor as unknown as WebSocketConstructor;
@@ -760,7 +811,7 @@ function scheduleSessionWebSocketExpiry(sessionId: string, entry: CachedWebSocke
}
async function connectWebSocket(url: string, headers: Headers, signal?: AbortSignal): Promise<WebSocketLike> {
const WebSocketCtor = getWebSocketConstructor();
const WebSocketCtor = await getWebSocketConstructor();
if (!WebSocketCtor) {
throw new Error("WebSocket transport is not available in this runtime");
}

View File

@@ -10,8 +10,8 @@ import type {
ChatCompletionSystemMessageParam,
ChatCompletionToolMessageParam,
} from "openai/resources/chat/completions.js";
import { getEnvApiKey } from "../env-api-keys.js";
import { calculateCost, clampThinkingLevel } from "../models.js";
import { getEnvApiKey } from "../env-api-keys.ts";
import { calculateCost, clampThinkingLevel } from "../models.ts";
import type {
AssistantMessage,
CacheRetention,
@@ -29,15 +29,16 @@ import type {
Tool,
ToolCall,
ToolResultMessage,
} from "../types.js";
import { AssistantMessageEventStream } from "../utils/event-stream.js";
import { headersToRecord } from "../utils/headers.js";
import { parseStreamingJson } from "../utils/json-parse.js";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.js";
import { isCloudflareProvider, resolveCloudflareBaseUrl } from "./cloudflare.js";
import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./github-copilot-headers.js";
import { buildBaseOptions } from "./simple-options.js";
import { transformMessages } from "./transform-messages.js";
} from "../types.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { headersToRecord } from "../utils/headers.ts";
import { parseStreamingJson } from "../utils/json-parse.ts";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts";
import { isCloudflareProvider, resolveCloudflareBaseUrl } from "./cloudflare.ts";
import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./github-copilot-headers.ts";
import { clampOpenAIPromptCacheKey } from "./openai-prompt-cache.ts";
import { buildBaseOptions } from "./simple-options.ts";
import { transformMessages } from "./transform-messages.ts";
/**
* Check if conversation messages contain tool calls or tool results.
@@ -165,6 +166,7 @@ export const streamOpenAICompletions: StreamFunction<"openai-completions", OpenA
let textBlock: TextContent | null = null;
let thinkingBlock: ThinkingContent | null = null;
let hasFinishReason = false;
const toolCallBlocksByIndex = new Map<number, StreamingToolCallBlock>();
const toolCallBlocksById = new Map<string, StreamingToolCallBlock>();
const blocks = output.content as StreamingBlock[];
@@ -288,6 +290,7 @@ export const streamOpenAICompletions: StreamFunction<"openai-completions", OpenA
if (finishReasonResult.errorMessage) {
output.errorMessage = finishReasonResult.errorMessage;
}
hasFinishReason = true;
}
if (choice.delta) {
@@ -324,7 +327,11 @@ export const streamOpenAICompletions: StreamFunction<"openai-completions", OpenA
if (foundReasoningField) {
const delta = deltaFields[foundReasoningField];
if (typeof delta === "string" && delta.length > 0) {
const block = ensureThinkingBlock(foundReasoningField);
const thinkingSignature =
model.provider === "opencode-go" && foundReasoningField === "reasoning"
? "reasoning_content"
: foundReasoningField;
const block = ensureThinkingBlock(thinkingSignature);
block.thinking += delta;
stream.push({
type: "thinking_delta",
@@ -390,6 +397,9 @@ export const streamOpenAICompletions: StreamFunction<"openai-completions", OpenA
if (output.stopReason === "error") {
throw new Error(output.errorMessage || "Provider returned an error stop reason");
}
if (!hasFinishReason) {
throw new Error("Stream ended without finish_reason");
}
stream.push({ type: "done", reason: output.stopReason, message: output });
stream.end();
@@ -507,7 +517,7 @@ function buildParams(
prompt_cache_key:
(model.baseUrl.includes("api.openai.com") && cacheRetention !== "none") ||
(cacheRetention === "long" && compat.supportsLongCacheRetention)
? options?.sessionId
? clampOpenAIPromptCacheKey(options?.sessionId)
: undefined,
prompt_cache_retention: cacheRetention === "long" && compat.supportsLongCacheRetention ? "24h" : undefined,
};
@@ -839,7 +849,10 @@ export function convertMessages(
}
// Use the signature from the first thinking block if available (for llama.cpp server + gpt-oss)
const signature = nonEmptyThinkingBlocks[0].thinkingSignature;
let signature = nonEmptyThinkingBlocks[0].thinkingSignature;
if (model.provider === "opencode-go" && signature === "reasoning") {
signature = "reasoning_content";
}
if (signature && signature.length > 0) {
(assistantMsg as any)[signature] = nonEmptyThinkingBlocks.map((block) => block.thinking).join("\n");
}
@@ -997,17 +1010,17 @@ function parseChunkUsage(
model: Model<"openai-completions">,
): AssistantMessage["usage"] {
const promptTokens = rawUsage.prompt_tokens || 0;
const reportedCachedTokens = rawUsage.prompt_tokens_details?.cached_tokens ?? rawUsage.prompt_cache_hit_tokens ?? 0;
const cacheReadTokens = rawUsage.prompt_tokens_details?.cached_tokens ?? rawUsage.prompt_cache_hit_tokens ?? 0;
const cacheWriteTokens = rawUsage.prompt_tokens_details?.cache_write_tokens || 0;
// Normalize to pi-ai semantics:
// - cacheRead: hits from cache created by previous requests only
// - cacheWrite: tokens written to cache in this request
// Some OpenAI-compatible providers (observed on OpenRouter) report cached_tokens
// as (previous hits + current writes). In that case, remove cacheWrite from cacheRead.
const cacheReadTokens =
cacheWriteTokens > 0 ? Math.max(0, reportedCachedTokens - cacheWriteTokens) : reportedCachedTokens;
// Follow documented OpenAI/OpenRouter semantics: cached_tokens is cache-read
// tokens (hits). OpenAI does not document or emit cache_write_tokens, but
// OpenRouter-compatible providers can include it as a separate write count.
// OpenRouter's own provider/tests affirm the separate mapping:
// https://github.com/OpenRouterTeam/ai-sdk-provider/pull/409
// Do not subtract writes from cached_tokens, otherwise spec-compliant
// providers are under-reported. DS4 mirrors this contract too:
// https://github.com/antirez/ds4/pull/29
const input = Math.max(0, promptTokens - cacheReadTokens - cacheWriteTokens);
// OpenAI completion_tokens already includes reasoning_tokens.
const outputTokens = rawUsage.completion_tokens || 0;

View File

@@ -0,0 +1,8 @@
export const OPENAI_PROMPT_CACHE_KEY_MAX_LENGTH = 64;
export function clampOpenAIPromptCacheKey(key: string | undefined): string | undefined {
if (key === undefined) return undefined;
const chars = Array.from(key);
if (chars.length <= OPENAI_PROMPT_CACHE_KEY_MAX_LENGTH) return key;
return chars.slice(0, OPENAI_PROMPT_CACHE_KEY_MAX_LENGTH).join("");
}

View File

@@ -12,7 +12,7 @@ import type {
ResponseReasoningItem,
ResponseStreamEvent,
} from "openai/resources/responses/responses.js";
import { calculateCost } from "../models.js";
import { calculateCost } from "../models.ts";
import type {
Api,
AssistantMessage,
@@ -26,12 +26,12 @@ import type {
Tool,
ToolCall,
Usage,
} from "../types.js";
import type { AssistantMessageEventStream } from "../utils/event-stream.js";
import { shortHash } from "../utils/hash.js";
import { parseStreamingJson } from "../utils/json-parse.js";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.js";
import { transformMessages } from "./transform-messages.js";
} from "../types.ts";
import type { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { shortHash } from "../utils/hash.ts";
import { parseStreamingJson } from "../utils/json-parse.ts";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.ts";
import { transformMessages } from "./transform-messages.ts";
// =============================================================================
// Utilities

View File

@@ -1,7 +1,7 @@
import OpenAI from "openai";
import type { ResponseCreateParamsStreaming } from "openai/resources/responses/responses.js";
import { getEnvApiKey } from "../env-api-keys.js";
import { clampThinkingLevel } from "../models.js";
import { getEnvApiKey } from "../env-api-keys.ts";
import { clampThinkingLevel } from "../models.ts";
import type {
Api,
AssistantMessage,
@@ -13,13 +13,14 @@ import type {
StreamFunction,
StreamOptions,
Usage,
} from "../types.js";
import { AssistantMessageEventStream } from "../utils/event-stream.js";
import { headersToRecord } from "../utils/headers.js";
import { isCloudflareProvider, resolveCloudflareBaseUrl } from "./cloudflare.js";
import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./github-copilot-headers.js";
import { convertResponsesMessages, convertResponsesTools, processResponsesStream } from "./openai-responses-shared.js";
import { buildBaseOptions } from "./simple-options.js";
} from "../types.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import { headersToRecord } from "../utils/headers.ts";
import { isCloudflareProvider, resolveCloudflareBaseUrl } from "./cloudflare.ts";
import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./github-copilot-headers.ts";
import { clampOpenAIPromptCacheKey } from "./openai-prompt-cache.ts";
import { convertResponsesMessages, convertResponsesTools, processResponsesStream } from "./openai-responses-shared.ts";
import { buildBaseOptions } from "./simple-options.ts";
const OPENAI_TOOL_CALL_PROVIDERS = new Set(["openai", "openai-codex", "opencode"]);
@@ -51,6 +52,22 @@ function getPromptCacheRetention(
return cacheRetention === "long" && compat.supportsLongCacheRetention ? "24h" : undefined;
}
function formatOpenAIResponsesError(error: unknown): string {
if (error instanceof Error) {
const status = (error as Error & { status?: unknown }).status;
const statusCode = typeof status === "number" ? status : undefined;
if (statusCode !== undefined) {
return `OpenAI API error (${statusCode}): ${error.message}`;
}
return error.message;
}
try {
return JSON.stringify(error);
} catch {
return String(error);
}
}
// OpenAI Responses-specific options
export interface OpenAIResponsesOptions extends StreamOptions {
reasoningEffort?: "minimal" | "low" | "medium" | "high" | "xhigh";
@@ -130,7 +147,7 @@ export const streamOpenAIResponses: StreamFunction<"openai-responses", OpenAIRes
delete (block as { partialJson?: string }).partialJson;
}
output.stopReason = options?.signal?.aborted ? "aborted" : "error";
output.errorMessage = error instanceof Error ? error.message : JSON.stringify(error);
output.errorMessage = formatOpenAIResponsesError(error);
stream.push({ type: "error", reason: output.stopReason, error: output });
stream.end();
}
@@ -224,7 +241,7 @@ function buildParams(model: Model<"openai-responses">, context: Context, options
model: model.id,
input: messages,
stream: true,
prompt_cache_key: cacheRetention === "none" ? undefined : options?.sessionId,
prompt_cache_key: cacheRetention === "none" ? undefined : clampOpenAIPromptCacheKey(options?.sessionId),
prompt_cache_retention: getPromptCacheRetention(compat, cacheRetention),
store: false,
};

View File

@@ -1,4 +1,4 @@
import { clearApiProviders, registerApiProvider } from "../api-registry.js";
import { clearApiProviders, registerApiProvider } from "../api-registry.ts";
import type {
Api,
AssistantMessage,
@@ -8,17 +8,17 @@ import type {
SimpleStreamOptions,
StreamFunction,
StreamOptions,
} from "../types.js";
import { AssistantMessageEventStream } from "../utils/event-stream.js";
import type { BedrockOptions } from "./amazon-bedrock.js";
import type { AnthropicOptions } from "./anthropic.js";
import type { AzureOpenAIResponsesOptions } from "./azure-openai-responses.js";
import type { GoogleOptions } from "./google.js";
import type { GoogleVertexOptions } from "./google-vertex.js";
import type { MistralOptions } from "./mistral.js";
import type { OpenAICodexResponsesOptions } from "./openai-codex-responses.js";
import type { OpenAICompletionsOptions } from "./openai-completions.js";
import type { OpenAIResponsesOptions } from "./openai-responses.js";
} from "../types.ts";
import { AssistantMessageEventStream } from "../utils/event-stream.ts";
import type { BedrockOptions } from "./amazon-bedrock.ts";
import type { AnthropicOptions } from "./anthropic.ts";
import type { AzureOpenAIResponsesOptions } from "./azure-openai-responses.ts";
import type { GoogleOptions } from "./google.ts";
import type { GoogleVertexOptions } from "./google-vertex.ts";
import type { MistralOptions } from "./mistral.ts";
import type { OpenAICodexResponsesOptions } from "./openai-codex-responses.ts";
import type { OpenAICompletionsOptions } from "./openai-completions.ts";
import type { OpenAIResponsesOptions } from "./openai-responses.ts";
interface LazyProviderModule<
TApi extends Api,
@@ -86,7 +86,10 @@ interface BedrockProviderModule {
) => AsyncIterable<AssistantMessageEvent>;
}
const importNodeOnlyProvider = (specifier: string): Promise<unknown> => import(specifier);
const importNodeOnlyProvider = (specifier: string): Promise<unknown> => {
const runtimeSpecifier = import.meta.url.endsWith(".js") ? specifier.replace(/\.ts$/, ".js") : specifier;
return import(runtimeSpecifier);
};
let anthropicProviderModulePromise:
| Promise<LazyProviderModule<"anthropic-messages", AnthropicOptions, SimpleStreamOptions>>
@@ -203,7 +206,7 @@ function createLazySimpleStream<
function loadAnthropicProviderModule(): Promise<
LazyProviderModule<"anthropic-messages", AnthropicOptions, SimpleStreamOptions>
> {
anthropicProviderModulePromise ||= import("./anthropic.js").then((module) => {
anthropicProviderModulePromise ||= import("./anthropic.ts").then((module) => {
const provider = module as AnthropicProviderModule;
return {
stream: provider.streamAnthropic,
@@ -216,7 +219,7 @@ function loadAnthropicProviderModule(): Promise<
function loadAzureOpenAIResponsesProviderModule(): Promise<
LazyProviderModule<"azure-openai-responses", AzureOpenAIResponsesOptions, SimpleStreamOptions>
> {
azureOpenAIResponsesProviderModulePromise ||= import("./azure-openai-responses.js").then((module) => {
azureOpenAIResponsesProviderModulePromise ||= import("./azure-openai-responses.ts").then((module) => {
const provider = module as AzureOpenAIResponsesProviderModule;
return {
stream: provider.streamAzureOpenAIResponses,
@@ -229,7 +232,7 @@ function loadAzureOpenAIResponsesProviderModule(): Promise<
function loadGoogleProviderModule(): Promise<
LazyProviderModule<"google-generative-ai", GoogleOptions, SimpleStreamOptions>
> {
googleProviderModulePromise ||= import("./google.js").then((module) => {
googleProviderModulePromise ||= import("./google.ts").then((module) => {
const provider = module as GoogleProviderModule;
return {
stream: provider.streamGoogle,
@@ -242,7 +245,7 @@ function loadGoogleProviderModule(): Promise<
function loadGoogleVertexProviderModule(): Promise<
LazyProviderModule<"google-vertex", GoogleVertexOptions, SimpleStreamOptions>
> {
googleVertexProviderModulePromise ||= import("./google-vertex.js").then((module) => {
googleVertexProviderModulePromise ||= import("./google-vertex.ts").then((module) => {
const provider = module as GoogleVertexProviderModule;
return {
stream: provider.streamGoogleVertex,
@@ -255,7 +258,7 @@ function loadGoogleVertexProviderModule(): Promise<
function loadMistralProviderModule(): Promise<
LazyProviderModule<"mistral-conversations", MistralOptions, SimpleStreamOptions>
> {
mistralProviderModulePromise ||= import("./mistral.js").then((module) => {
mistralProviderModulePromise ||= import("./mistral.ts").then((module) => {
const provider = module as MistralProviderModule;
return {
stream: provider.streamMistral,
@@ -268,7 +271,7 @@ function loadMistralProviderModule(): Promise<
function loadOpenAICodexResponsesProviderModule(): Promise<
LazyProviderModule<"openai-codex-responses", OpenAICodexResponsesOptions, SimpleStreamOptions>
> {
openAICodexResponsesProviderModulePromise ||= import("./openai-codex-responses.js").then((module) => {
openAICodexResponsesProviderModulePromise ||= import("./openai-codex-responses.ts").then((module) => {
const provider = module as OpenAICodexResponsesProviderModule;
return {
stream: provider.streamOpenAICodexResponses,
@@ -281,7 +284,7 @@ function loadOpenAICodexResponsesProviderModule(): Promise<
function loadOpenAICompletionsProviderModule(): Promise<
LazyProviderModule<"openai-completions", OpenAICompletionsOptions, SimpleStreamOptions>
> {
openAICompletionsProviderModulePromise ||= import("./openai-completions.js").then((module) => {
openAICompletionsProviderModulePromise ||= import("./openai-completions.ts").then((module) => {
const provider = module as OpenAICompletionsProviderModule;
return {
stream: provider.streamOpenAICompletions,
@@ -294,7 +297,7 @@ function loadOpenAICompletionsProviderModule(): Promise<
function loadOpenAIResponsesProviderModule(): Promise<
LazyProviderModule<"openai-responses", OpenAIResponsesOptions, SimpleStreamOptions>
> {
openAIResponsesProviderModulePromise ||= import("./openai-responses.js").then((module) => {
openAIResponsesProviderModulePromise ||= import("./openai-responses.ts").then((module) => {
const provider = module as OpenAIResponsesProviderModule;
return {
stream: provider.streamOpenAIResponses,
@@ -310,7 +313,7 @@ function loadBedrockProviderModule(): Promise<
if (bedrockProviderModuleOverride) {
return Promise.resolve(bedrockProviderModuleOverride);
}
bedrockProviderModulePromise ||= importNodeOnlyProvider("./amazon-bedrock.js").then((module) => {
bedrockProviderModulePromise ||= importNodeOnlyProvider("./amazon-bedrock.ts").then((module) => {
const provider = module as BedrockProviderModule;
return {
stream: provider.streamBedrock,

View File

@@ -1,9 +1,9 @@
import type { Api, Model, SimpleStreamOptions, StreamOptions, ThinkingBudgets, ThinkingLevel } from "../types.js";
import type { Api, Model, SimpleStreamOptions, StreamOptions, ThinkingBudgets, ThinkingLevel } from "../types.ts";
export function buildBaseOptions(model: Model<Api>, options?: SimpleStreamOptions, apiKey?: string): StreamOptions {
export function buildBaseOptions(_model: Model<Api>, options?: SimpleStreamOptions, apiKey?: string): StreamOptions {
return {
temperature: options?.temperature,
maxTokens: options?.maxTokens ?? (model.maxTokens > 0 ? Math.min(model.maxTokens, 32000) : undefined),
maxTokens: options?.maxTokens,
signal: options?.signal,
apiKey: apiKey || options?.apiKey,
transport: options?.transport,
@@ -24,7 +24,8 @@ export function clampReasoning(effort: ThinkingLevel | undefined): Exclude<Think
}
export function adjustMaxTokensForThinking(
baseMaxTokens: number,
// Undefined means no explicit caller cap. Use the model cap and fit thinking inside it.
baseMaxTokens: number | undefined,
modelMaxTokens: number,
reasoningLevel: ThinkingLevel,
customBudgets?: ThinkingBudgets,
@@ -40,7 +41,8 @@ export function adjustMaxTokensForThinking(
const minOutputTokens = 1024;
const level = clampReasoning(reasoningLevel)!;
let thinkingBudget = budgets[level]!;
const maxTokens = Math.min(baseMaxTokens + thinkingBudget, modelMaxTokens);
const maxTokens =
baseMaxTokens === undefined ? modelMaxTokens : Math.min(baseMaxTokens + thinkingBudget, modelMaxTokens);
if (maxTokens <= thinkingBudget) {
thinkingBudget = Math.max(0, maxTokens - minOutputTokens);

View File

@@ -7,7 +7,7 @@ import type {
TextContent,
ToolCall,
ToolResultMessage,
} from "../types.js";
} from "../types.ts";
const NON_VISION_USER_IMAGE_PLACEHOLDER = "(image omitted: model does not support images)";
const NON_VISION_TOOL_IMAGE_PLACEHOLDER = "(tool image omitted: model does not support images)";

View File

@@ -1,6 +1,6 @@
import "./providers/register-builtins.js";
import "./providers/register-builtins.ts";
import { getApiProvider } from "./api-registry.js";
import { getApiProvider } from "./api-registry.ts";
import type {
Api,
AssistantMessage,
@@ -10,9 +10,9 @@ import type {
ProviderStreamOptions,
SimpleStreamOptions,
StreamOptions,
} from "./types.js";
} from "./types.ts";
export { getEnvApiKey } from "./env-api-keys.js";
export { getEnvApiKey } from "./env-api-keys.ts";
function resolveApiProvider(api: Api) {
const provider = getApiProvider(api);

View File

@@ -1,7 +1,7 @@
import type { AssistantMessageDiagnostic } from "./utils/diagnostics.js";
import type { AssistantMessageEventStream } from "./utils/event-stream.js";
import type { AssistantMessageDiagnostic } from "./utils/diagnostics.ts";
import type { AssistantMessageEventStream } from "./utils/event-stream.ts";
export type { AssistantMessageEventStream } from "./utils/event-stream.js";
export type { AssistantMessageEventStream } from "./utils/event-stream.ts";
export type KnownApi =
| "openai-completions"
@@ -419,6 +419,22 @@ export interface AnthropicMessagesCompat {
supportsEagerToolInputStreaming?: boolean;
/** Whether the provider supports Anthropic long cache retention (`cache_control.ttl: "1h"`). Default: true. */
supportsLongCacheRetention?: boolean;
/**
* Whether to send the `x-session-affinity` header from `options.sessionId`
* when caching is enabled. Required for providers like Fireworks that use
* session affinity for prompt cache routing (requests to the same replica
* maximize cache hits).
* Default: false.
*/
sendSessionAffinityHeaders?: boolean;
/**
* Whether the provider supports Anthropic-style `cache_control` markers on
* tool definitions. When false, `cache_control` is omitted from tool params.
* Some Anthropic-compatible providers (e.g., Fireworks) do not support this
* field on tools and may reject or ignore it.
* Default: true.
*/
supportsCacheControlOnTools?: boolean;
}
/**

View File

@@ -1,4 +1,4 @@
import type { AssistantMessage, AssistantMessageEvent } from "../types.js";
import type { AssistantMessage, AssistantMessageEvent } from "../types.ts";
// Generic event stream class for async iteration
export class EventStream<T, R = T> implements AsyncIterable<T> {
@@ -7,11 +7,12 @@ export class EventStream<T, R = T> implements AsyncIterable<T> {
private done = false;
private finalResultPromise: Promise<R>;
private resolveFinalResult!: (result: R) => void;
private isComplete: (event: T) => boolean;
private extractResult: (event: T) => R;
constructor(
private isComplete: (event: T) => boolean,
private extractResult: (event: T) => R,
) {
constructor(isComplete: (event: T) => boolean, extractResult: (event: T) => R) {
this.isComplete = isComplete;
this.extractResult = extractResult;
this.finalResultPromise = new Promise((resolve) => {
this.resolveFinalResult = resolve;
});

View File

@@ -0,0 +1,123 @@
import type { Agent as HttpAgent } from "node:http";
import type { Agent as HttpsAgent } from "node:https";
import { HttpProxyAgent } from "http-proxy-agent";
import { HttpsProxyAgent } from "https-proxy-agent";
const DEFAULT_PROXY_PORTS: Record<string, number> = {
ftp: 21,
gopher: 70,
http: 80,
https: 443,
ws: 80,
wss: 443,
};
export interface NodeHttpProxyAgents {
httpAgent: HttpAgent;
httpsAgent: HttpsAgent;
}
export const UNSUPPORTED_PROXY_PROTOCOL_MESSAGE =
"Unsupported proxy protocol. SOCKS and PAC proxy URLs are not supported; use an HTTP or HTTPS proxy URL.";
function getProxyEnv(key: string): string {
return process.env[key.toLowerCase()] || process.env[key.toUpperCase()] || "";
}
function parseProxyTargetUrl(targetUrl: string | URL): URL | undefined {
if (targetUrl instanceof URL) {
return targetUrl;
}
try {
return new URL(targetUrl);
} catch {
return undefined;
}
}
function shouldProxyHostname(hostname: string, port: number): boolean {
const noProxy = getProxyEnv("no_proxy").toLowerCase();
if (!noProxy) {
return true;
}
if (noProxy === "*") {
return false;
}
return noProxy.split(/[,\s]/).every((proxy) => {
if (!proxy) {
return true;
}
const parsedProxy = proxy.match(/^(.+):(\d+)$/);
let proxyHostname = parsedProxy ? parsedProxy[1] : proxy;
const proxyPort = parsedProxy ? Number.parseInt(parsedProxy[2]!, 10) : 0;
if (proxyPort && proxyPort !== port) {
return true;
}
if (!/^[.*]/.test(proxyHostname)) {
return hostname !== proxyHostname;
}
if (proxyHostname.startsWith("*")) {
proxyHostname = proxyHostname.slice(1);
}
return !hostname.endsWith(proxyHostname);
});
}
function getProxyForUrl(targetUrl: string | URL): string {
const parsedUrl = parseProxyTargetUrl(targetUrl);
if (!parsedUrl?.protocol || !parsedUrl.host) {
return "";
}
const protocol = parsedUrl.protocol.split(":", 1)[0]!;
const hostname = parsedUrl.host.replace(/:\d*$/, "");
const port = Number.parseInt(parsedUrl.port, 10) || DEFAULT_PROXY_PORTS[protocol] || 0;
if (!shouldProxyHostname(hostname, port)) {
return "";
}
let proxy = getProxyEnv(`${protocol}_proxy`) || getProxyEnv("all_proxy");
if (proxy && !proxy.includes("://")) {
proxy = `${protocol}://${proxy}`;
}
return proxy;
}
export function resolveHttpProxyUrlForTarget(targetUrl: string | URL): URL | undefined {
const proxy = getProxyForUrl(targetUrl);
if (!proxy) {
return undefined;
}
let proxyUrl: URL;
try {
proxyUrl = new URL(proxy);
} catch (error) {
throw new Error(
`Invalid proxy URL ${JSON.stringify(proxy)}: ${error instanceof Error ? error.message : String(error)}`,
);
}
if (proxyUrl.protocol !== "http:" && proxyUrl.protocol !== "https:") {
throw new Error(`${UNSUPPORTED_PROXY_PROTOCOL_MESSAGE} Got ${proxyUrl.protocol}`);
}
return proxyUrl;
}
export function createHttpProxyAgentsForTarget(targetUrl: string | URL): NodeHttpProxyAgents | undefined {
const proxyUrl = resolveHttpProxyUrlForTarget(targetUrl);
if (!proxyUrl) {
return undefined;
}
return {
httpAgent: new HttpProxyAgent(proxyUrl),
httpsAgent: new HttpsProxyAgent(proxyUrl) as unknown as HttpsAgent,
};
}

View File

@@ -6,9 +6,9 @@
*/
import type { Server } from "node:http";
import { oauthErrorHtml, oauthSuccessHtml } from "./oauth-page.js";
import { generatePKCE } from "./pkce.js";
import type { OAuthCredentials, OAuthLoginCallbacks, OAuthPrompt, OAuthProviderInterface } from "./types.js";
import { oauthErrorHtml, oauthSuccessHtml } from "./oauth-page.ts";
import { generatePKCE } from "./pkce.ts";
import type { OAuthCredentials, OAuthLoginCallbacks, OAuthPrompt, OAuthProviderInterface } from "./types.ts";
type CallbackServerInfo = {
server: Server;

View File

@@ -2,9 +2,9 @@
* GitHub Copilot OAuth flow
*/
import { getModels } from "../../models.js";
import type { Api, Model } from "../../types.js";
import type { OAuthCredentials, OAuthLoginCallbacks, OAuthProviderInterface } from "./types.js";
import { getModels } from "../../models.ts";
import type { Api, Model } from "../../types.ts";
import type { OAuthCredentials, OAuthLoginCallbacks, OAuthProviderInterface } from "./types.ts";
type CopilotCredentials = OAuthCredentials & {
enterpriseUrl?: string;

View File

@@ -8,7 +8,7 @@
*/
// Anthropic
export { anthropicOAuthProvider, loginAnthropic, refreshAnthropicToken } from "./anthropic.js";
export { anthropicOAuthProvider, loginAnthropic, refreshAnthropicToken } from "./anthropic.ts";
// GitHub Copilot
export {
getGitHubCopilotBaseUrl,
@@ -16,20 +16,20 @@ export {
loginGitHubCopilot,
normalizeDomain,
refreshGitHubCopilotToken,
} from "./github-copilot.js";
} from "./github-copilot.ts";
// OpenAI Codex (ChatGPT OAuth)
export { loginOpenAICodex, openaiCodexOAuthProvider, refreshOpenAICodexToken } from "./openai-codex.js";
export { loginOpenAICodex, openaiCodexOAuthProvider, refreshOpenAICodexToken } from "./openai-codex.ts";
export * from "./types.js";
export * from "./types.ts";
// ============================================================================
// Provider Registry
// ============================================================================
import { anthropicOAuthProvider } from "./anthropic.js";
import { githubCopilotOAuthProvider } from "./github-copilot.js";
import { openaiCodexOAuthProvider } from "./openai-codex.js";
import type { OAuthCredentials, OAuthProviderId, OAuthProviderInfo, OAuthProviderInterface } from "./types.js";
import { anthropicOAuthProvider } from "./anthropic.ts";
import { githubCopilotOAuthProvider } from "./github-copilot.ts";
import { openaiCodexOAuthProvider } from "./openai-codex.ts";
import type { OAuthCredentials, OAuthProviderId, OAuthProviderInfo, OAuthProviderInterface } from "./types.ts";
const BUILT_IN_OAUTH_PROVIDERS: OAuthProviderInterface[] = [
anthropicOAuthProvider,

View File

@@ -5,7 +5,7 @@
* It is only intended for CLI use, not browser environments.
*/
// NEVER convert to top-level imports - breaks browser/Vite builds (web-ui)
// NEVER convert to top-level imports - breaks browser/Vite builds
let _randomBytes: typeof import("node:crypto").randomBytes | null = null;
let _http: typeof import("node:http") | null = null;
if (typeof process !== "undefined" && (process.versions?.node || process.versions?.bun)) {
@@ -17,9 +17,9 @@ if (typeof process !== "undefined" && (process.versions?.node || process.version
});
}
import { oauthErrorHtml, oauthSuccessHtml } from "./oauth-page.js";
import { generatePKCE } from "./pkce.js";
import type { OAuthCredentials, OAuthLoginCallbacks, OAuthPrompt, OAuthProviderInterface } from "./types.js";
import { oauthErrorHtml, oauthSuccessHtml } from "./oauth-page.ts";
import { generatePKCE } from "./pkce.ts";
import type { OAuthCredentials, OAuthLoginCallbacks, OAuthPrompt, OAuthProviderInterface } from "./types.ts";
const CALLBACK_HOST = process.env.PI_OAUTH_CALLBACK_HOST || "127.0.0.1";
const CLIENT_ID = "app_EMoamEEZ73f0CkXaXp7hrann";

View File

@@ -1,4 +1,4 @@
import type { Api, Model } from "../../types.js";
import type { Api, Model } from "../../types.ts";
export type OAuthCredentials = {
refresh: string;

View File

@@ -1,4 +1,4 @@
import type { AssistantMessage } from "../types.js";
import type { AssistantMessage } from "../types.ts";
/**
* Regex patterns to detect context overflow errors from different providers.
@@ -11,6 +11,7 @@ import type { AssistantMessage } from "../types.js";
* - Anthropic: "prompt is too long: 213462 tokens > 200000 maximum"
* - Anthropic: "413 {\"error\":{\"type\":\"request_too_large\",\"message\":\"Request exceeds the maximum size\"}}"
* - OpenAI: "Your input exceeds the context window of this model"
* - OpenAI/LiteLLM: "Requested token count exceeds the model's maximum context length of 131072 tokens"
* - Google: "The input token count (1196265) exceeds the maximum number of tokens allowed (1048575)"
* - xAI: "This model's maximum prompt length is 131072 but the request contains 537812 tokens"
* - Groq: "Please reduce the length of the messages or completion"
@@ -34,6 +35,7 @@ const OVERFLOW_PATTERNS = [
/request_too_large/i, // Anthropic request byte-size overflow (HTTP 413)
/input is too long for requested model/i, // Amazon Bedrock
/exceeds the context window/i, // OpenAI (Completions & Responses API)
/exceeds (?:the )?(?:model'?s )?maximum context length of [\d,]+ tokens?/i, // OpenAI-compatible proxies (LiteLLM)
/input token count.*exceeds the maximum/i, // Google (Gemini)
/maximum prompt length is \d+/i, // xAI (Grok)
/reduce the length of the messages/i, // Groq
@@ -81,7 +83,7 @@ const NON_OVERFLOW_PATTERNS = [
*
* **Reliable detection (returns error with detectable message):**
* - Anthropic: "prompt is too long: X tokens > Y maximum" or "request_too_large"
* - OpenAI (Completions & Responses): "exceeds the context window"
* - OpenAI (Completions & Responses): "exceeds the context window" or "exceeds the model's maximum context length of X tokens"
* - Google Gemini: "input token count exceeds the maximum"
* - xAI (Grok): "maximum prompt length is X but request contains Y"
* - Groq: "reduce the length of the messages"

View File

@@ -1,7 +1,7 @@
import { Compile } from "typebox/compile";
import type { TLocalizedValidationError } from "typebox/error";
import { Value } from "typebox/value";
import type { Tool, ToolCall } from "../types.js";
import type { Tool, ToolCall } from "../types.ts";
const validatorCache = new WeakMap<object, ReturnType<typeof Compile>>();
const TYPEBOX_KIND = Symbol.for("TypeBox.Kind");

View File

@@ -1,13 +1,13 @@
import { describe, expect, it } from "vitest";
import { getModel } from "../src/models.js";
import { complete, stream } from "../src/stream.js";
import type { Api, Context, Model, StreamOptions } from "../src/types.js";
import { getModel } from "../src/models.ts";
import { complete, stream } from "../src/stream.ts";
import type { Api, Context, Model, StreamOptions } from "../src/types.ts";
type StreamOptionsWithExtras = StreamOptions & Record<string, unknown>;
import { hasAzureOpenAICredentials, resolveAzureDeploymentName } from "./azure-utils.js";
import { hasBedrockCredentials } from "./bedrock-utils.js";
import { resolveApiKey } from "./oauth.js";
import { hasAzureOpenAICredentials, resolveAzureDeploymentName } from "./azure-utils.ts";
import { hasBedrockCredentials } from "./bedrock-utils.ts";
import { resolveApiKey } from "./oauth.ts";
// Resolve OAuth tokens at module level (async, runs before tests)
const [openaiCodexToken] = await Promise.all([resolveApiKey("openai-codex")]);
@@ -276,12 +276,12 @@ describe("AI Providers Abort Tests", () => {
describe("OpenAI Codex Provider Abort", () => {
it.skipIf(!openaiCodexToken)("should abort mid-stream", { retry: 3 }, async () => {
const llm = getModel("openai-codex", "gpt-5.2-codex");
const llm = getModel("openai-codex", "gpt-5.5");
await testAbortSignal(llm, { apiKey: openaiCodexToken });
});
it.skipIf(!openaiCodexToken)("should handle immediate abort", { retry: 3 }, async () => {
const llm = getModel("openai-codex", "gpt-5.2-codex");
const llm = getModel("openai-codex", "gpt-5.5");
await testImmediateAbort(llm, { apiKey: openaiCodexToken });
});
});

View File

@@ -2,8 +2,8 @@ import { createServer, type IncomingMessage, type ServerResponse } from "node:ht
import type { AddressInfo } from "node:net";
import { Type } from "typebox";
import { describe, expect, it } from "vitest";
import { streamAnthropic } from "../src/providers/anthropic.js";
import type { Context, Model, Tool } from "../src/types.js";
import { streamAnthropic } from "../src/providers/anthropic.ts";
import type { Context, Model, Tool } from "../src/types.ts";
interface CapturedRequest {
headers: IncomingMessage["headers"];

View File

@@ -1,10 +1,10 @@
import { Type } from "typebox";
import { describe, expect, it } from "vitest";
import { getEnvApiKey } from "../src/env-api-keys.js";
import { getModels, getProviders } from "../src/models.js";
import { complete } from "../src/stream.js";
import type { Api, KnownProvider, Model, ProviderStreamOptions, Tool } from "../src/types.js";
import { resolveApiKey } from "./oauth.js";
import { getEnvApiKey } from "../src/env-api-keys.ts";
import { getModels, getProviders } from "../src/models.ts";
import { complete } from "../src/stream.ts";
import type { Api, KnownProvider, Model, ProviderStreamOptions, Tool } from "../src/types.ts";
import { resolveApiKey } from "./oauth.ts";
const githubCopilotToken = await resolveApiKey("github-copilot");

View File

@@ -1,9 +1,9 @@
import { describe, expect, it } from "vitest";
import { getEnvApiKey } from "../src/env-api-keys.js";
import { getModels, getProviders } from "../src/models.js";
import { complete } from "../src/stream.js";
import type { Api, KnownProvider, Model, ProviderStreamOptions } from "../src/types.js";
import { resolveApiKey } from "./oauth.js";
import { getEnvApiKey } from "../src/env-api-keys.ts";
import { getModels, getProviders } from "../src/models.ts";
import { complete } from "../src/stream.ts";
import type { Api, KnownProvider, Model, ProviderStreamOptions } from "../src/types.ts";
import { resolveApiKey } from "./oauth.ts";
const githubCopilotToken = await resolveApiKey("github-copilot");

View File

@@ -1,5 +1,5 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import { loginAnthropic, refreshAnthropicToken } from "../src/utils/oauth/anthropic.js";
import { loginAnthropic, refreshAnthropicToken } from "../src/utils/oauth/anthropic.ts";
function jsonResponse(body: unknown, status: number = 200): Response {
return new Response(JSON.stringify(body), {

View File

@@ -1,7 +1,7 @@
import { describe, expect, it } from "vitest";
import { getModel } from "../src/models.js";
import { streamSimple } from "../src/stream.js";
import type { Context } from "../src/types.js";
import { getModel } from "../src/models.ts";
import { streamSimple } from "../src/stream.ts";
import type { Context } from "../src/types.ts";
interface AnthropicThinkingPayload {
thinking?: { type: string };

View File

@@ -1,9 +1,9 @@
import type Anthropic from "@anthropic-ai/sdk";
import { Type } from "typebox";
import { describe, expect, it } from "vitest";
import { getModel } from "../src/models.js";
import { streamAnthropic } from "../src/providers/anthropic.js";
import type { Context, ToolCall } from "../src/types.js";
import { getModel } from "../src/models.ts";
import { streamAnthropic } from "../src/providers/anthropic.ts";
import type { Context, ToolCall } from "../src/types.ts";
function createSseResponse(events: Array<{ event: string; data: string }>): Response {
const body = events.map(({ event, data }) => `event: ${event}\ndata: ${data}\n`).join("\n");

View File

@@ -1,13 +1,20 @@
import { describe, expect, it } from "vitest";
import { getModel } from "../src/models.js";
import { streamSimple } from "../src/stream.js";
import type { Context, Model, SimpleStreamOptions } from "../src/types.js";
import { getModel } from "../src/models.ts";
import { streamSimple } from "../src/stream.ts";
import type { Context, Model, SimpleStreamOptions } from "../src/types.ts";
interface AnthropicThinkingPayload {
thinking?: { type: string; budget_tokens?: number; display?: string };
output_config?: { effort?: string };
}
class PayloadCaptured extends Error {
constructor() {
super("payload captured");
this.name = "PayloadCaptured";
}
}
function makePayloadCaptureContext(): Context {
return {
messages: [{ role: "user", content: "Hello", timestamp: Date.now() }],
@@ -29,7 +36,7 @@ async function capturePayload(
apiKey: "fake-key",
onPayload: (payload) => {
capturedPayload = payload as AnthropicThinkingPayload;
return payload;
throw new PayloadCaptured();
},
});

View File

@@ -1,9 +1,9 @@
import { Type } from "typebox";
import { describe, expect, it } from "vitest";
import { getModel } from "../src/models.js";
import { stream } from "../src/stream.js";
import type { Context, Tool } from "../src/types.js";
import { resolveApiKey } from "./oauth.js";
import { getModel } from "../src/models.ts";
import { stream } from "../src/stream.ts";
import type { Context, Tool } from "../src/types.ts";
import { resolveApiKey } from "./oauth.ts";
const oauthToken = await resolveApiKey("anthropic");

View File

@@ -1,7 +1,7 @@
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { getModel } from "../src/models.js";
import { streamAzureOpenAIResponses } from "../src/providers/azure-openai-responses.js";
import type { Context } from "../src/types.js";
import { getModel } from "../src/models.ts";
import { streamAzureOpenAIResponses } from "../src/providers/azure-openai-responses.ts";
import type { Context } from "../src/types.ts";
interface CapturedAzureClientOptions {
apiKey: string;
@@ -11,14 +11,20 @@ interface CapturedAzureClientOptions {
baseURL: string;
}
interface CapturedAzureResponsesPayload {
prompt_cache_key?: string;
}
const azureMock = vi.hoisted(() => ({
constructorCalls: [] as CapturedAzureClientOptions[],
lastParams: undefined as CapturedAzureResponsesPayload | undefined,
}));
vi.mock("openai", () => {
class AzureOpenAI {
responses = {
create: () => {
create: (params: CapturedAzureResponsesPayload) => {
azureMock.lastParams = params;
throw new Error("mock create");
},
};
@@ -42,6 +48,7 @@ const originalAzureOpenAIApiKey = process.env.AZURE_OPENAI_API_KEY;
beforeEach(() => {
azureMock.constructorCalls.length = 0;
azureMock.lastParams = undefined;
delete process.env.AZURE_OPENAI_BASE_URL;
delete process.env.AZURE_OPENAI_RESOURCE_NAME;
delete process.env.AZURE_OPENAI_API_VERSION;
@@ -126,6 +133,17 @@ describe("azure-openai-responses base URL normalization", () => {
expect(result.errorMessage).toContain("Invalid Azure OpenAI base URL");
});
it("clamps prompt_cache_key to OpenAI's 64-character limit", async () => {
const model = getModel("azure-openai-responses", "gpt-4o-mini");
await streamAzureOpenAIResponses(model, context, {
apiKey: "test-api-key",
azureBaseUrl: "https://my-resource.openai.azure.com",
sessionId: "x".repeat(67),
}).result();
expect(azureMock.lastParams?.prompt_cache_key).toBe("x".repeat(64));
});
it("builds correct default URL from AZURE_OPENAI_RESOURCE_NAME", async () => {
process.env.AZURE_OPENAI_RESOURCE_NAME = "my-resource";
const model = getModel("azure-openai-responses", "gpt-4o-mini");

View File

@@ -0,0 +1,159 @@
import { describe, expect, it, vi } from "vitest";
const bedrockMock = vi.hoisted(() => ({
constructorCalls: [] as Array<Record<string, unknown>>,
}));
vi.mock("@aws-sdk/client-bedrock-runtime", () => {
class BedrockRuntimeServiceException extends Error {}
class BedrockRuntimeClient {
constructor(config: Record<string, unknown>) {
bedrockMock.constructorCalls.push(config);
}
send(): Promise<never> {
return Promise.reject(new Error("mock send"));
}
}
class ConverseStreamCommand {
readonly input: unknown;
constructor(input: unknown) {
this.input = input;
}
}
return {
BedrockRuntimeClient,
BedrockRuntimeServiceException,
ConverseStreamCommand,
StopReason: {
END_TURN: "end_turn",
STOP_SEQUENCE: "stop_sequence",
MAX_TOKENS: "max_tokens",
MODEL_CONTEXT_WINDOW_EXCEEDED: "model_context_window_exceeded",
TOOL_USE: "tool_use",
},
CachePointType: { DEFAULT: "default" },
CacheTTL: { ONE_HOUR: "ONE_HOUR" },
ConversationRole: { ASSISTANT: "assistant", USER: "user" },
ImageFormat: { JPEG: "jpeg", PNG: "png", GIF: "gif", WEBP: "webp" },
ToolResultStatus: { ERROR: "error", SUCCESS: "success" },
};
});
import { getModel } from "../src/models.ts";
import { streamBedrock } from "../src/providers/amazon-bedrock.ts";
import type { Context, Message } from "../src/types.ts";
const baseModel = getModel("amazon-bedrock", "us.anthropic.claude-sonnet-4-5-20250929-v1:0");
async function capturePayload(context: Context): Promise<unknown> {
let capturedPayload: unknown;
const s = streamBedrock(baseModel, context, {
cacheRetention: "none",
signal: AbortSignal.abort(),
onPayload: (payload) => {
capturedPayload = payload;
return payload;
},
});
for await (const event of s) {
if (event.type === "error") break;
}
return capturedPayload;
}
describe("bedrock convertMessages skips unknown content types", () => {
it("skips unknown user content blocks instead of throwing", async () => {
const messages: Message[] = [
{
role: "user",
content: [
{ type: "text", text: "hello" },
{ type: "unknown", data: "foo" },
] as any,
timestamp: Date.now(),
},
];
const payload = await capturePayload({ messages });
expect(payload).toBeDefined();
const p = payload as { messages: Array<{ role: string; content: unknown[] }> };
expect(p.messages).toHaveLength(1);
expect(p.messages[0].content).toHaveLength(1);
expect(p.messages[0].content[0]).toEqual({ text: "hello" });
});
it("skips unknown assistant content blocks instead of throwing", async () => {
const messages: Message[] = [
{
role: "assistant",
content: [
{ type: "text", text: "hello" },
{ type: "unknown", data: "foo" },
] as any,
api: "bedrock-converse-stream",
provider: "amazon-bedrock",
model: baseModel.id,
usage: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
totalTokens: 0,
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
},
stopReason: "stop",
timestamp: Date.now(),
},
];
const payload = await capturePayload({ messages });
expect(payload).toBeDefined();
const p = payload as { messages: Array<{ role: string; content: unknown[] }> };
expect(p.messages).toHaveLength(1);
expect(p.messages[0].content).toHaveLength(1);
expect(p.messages[0].content[0]).toEqual({ text: "hello" });
});
it("skips user messages with only unknown content blocks", async () => {
const messages: Message[] = [
{
role: "user",
content: [{ type: "unknown", data: "foo" }] as any,
timestamp: Date.now(),
},
];
const payload = await capturePayload({ messages });
expect(payload).toBeDefined();
const p = payload as { messages: Array<{ role: string; content: unknown[] }> };
expect(p.messages).toHaveLength(0);
});
it("skips assistant messages with only unknown content blocks", async () => {
const messages: Message[] = [
{
role: "assistant",
content: [{ type: "unknown", data: "foo" }] as any,
api: "bedrock-converse-stream",
provider: "amazon-bedrock",
model: baseModel.id,
usage: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
totalTokens: 0,
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
},
stopReason: "stop",
timestamp: Date.now(),
},
];
const payload = await capturePayload({ messages });
expect(payload).toBeDefined();
const p = payload as { messages: Array<{ role: string; content: unknown[] }> };
expect(p.messages).toHaveLength(0);
});
});

View File

@@ -44,9 +44,9 @@ vi.mock("@aws-sdk/client-bedrock-runtime", () => {
};
});
import { getModel } from "../src/models.js";
import { streamBedrock } from "../src/providers/amazon-bedrock.js";
import type { Context, Model } from "../src/types.js";
import { getModel } from "../src/models.ts";
import { streamBedrock } from "../src/providers/amazon-bedrock.ts";
import type { Context, Model } from "../src/types.ts";
const context: Context = {
messages: [{ role: "user", content: "hello", timestamp: Date.now() }],

View File

@@ -17,10 +17,10 @@
*/
import { describe, expect, it } from "vitest";
import { getModels } from "../src/models.js";
import { complete } from "../src/stream.js";
import type { Context } from "../src/types.js";
import { hasBedrockCredentials } from "./bedrock-utils.js";
import { getModels } from "../src/models.ts";
import { complete } from "../src/stream.ts";
import type { Context } from "../src/types.ts";
import { hasBedrockCredentials } from "./bedrock-utils.ts";
describe("Amazon Bedrock Models", () => {
const models = getModels("amazon-bedrock");

View File

@@ -1,7 +1,8 @@
import { describe, expect, it } from "vitest";
import { getModel } from "../src/models.js";
import { type BedrockOptions, streamBedrock } from "../src/providers/amazon-bedrock.js";
import type { Context, Model } from "../src/types.js";
import { getModel } from "../src/models.ts";
import { type BedrockOptions, streamBedrock } from "../src/providers/amazon-bedrock.ts";
import type { Context, Model } from "../src/types.ts";
import { hasBedrockCredentials } from "./bedrock-utils.ts";
interface BedrockThinkingPayload {
additionalModelRequestFields?: {
@@ -11,6 +12,13 @@ interface BedrockThinkingPayload {
};
}
class PayloadCaptured extends Error {
constructor() {
super("payload captured");
this.name = "PayloadCaptured";
}
}
function makeContext(): Context {
return {
messages: [{ role: "user", content: "Hello", timestamp: Date.now() }],
@@ -25,10 +33,9 @@ async function capturePayload(
const s = streamBedrock(model, makeContext(), {
...options,
reasoning: options?.reasoning ?? "high",
signal: AbortSignal.abort(),
onPayload: (payload) => {
capturedPayload = payload as BedrockThinkingPayload;
return payload;
throw new PayloadCaptured();
},
});
@@ -106,6 +113,39 @@ describe("Bedrock thinking payload", () => {
});
});
describe.skipIf(!hasBedrockCredentials())("Bedrock Claude max tokens E2E", () => {
it(
"uses the model maxTokens cap instead of Bedrock's 4096-token default for adaptive Claude models",
{ retry: 2, timeout: 180000 },
async () => {
const baseModel = getModel("amazon-bedrock", "global.anthropic.claude-sonnet-4-6");
const model: Model<"bedrock-converse-stream"> = {
...baseModel,
maxTokens: 6000,
};
const response = await streamBedrock(
model,
{
systemPrompt: "You are a deterministic text generator. Follow the requested output format exactly.",
messages: [
{
role: "user",
content:
"Output exactly 5200 repetitions of the token alpha, separated by single spaces. Do not number them. Do not use markdown. Do not add any other text.",
timestamp: Date.now(),
},
],
},
{ reasoning: "low" },
).result();
expect(response.stopReason, response.errorMessage).not.toBe("error");
expect(response.usage.output).toBeGreaterThan(4096);
},
);
});
describe("Application inference profile support", () => {
it("uses adaptive thinking when model.name contains the model name but ARN does not", async () => {
const baseModel = getModel("amazon-bedrock", "global.anthropic.claude-opus-4-6-v1");
@@ -137,10 +177,9 @@ describe("Application inference profile support", () => {
messages: [{ role: "user", content: "Hello", timestamp: Date.now() }],
},
{
signal: AbortSignal.abort(),
onPayload: (payload) => {
capturedPayload = payload;
return payload;
throw new PayloadCaptured();
},
},
);

View File

@@ -1,7 +1,24 @@
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import { getModel } from "../src/models.js";
import { stream } from "../src/stream.js";
import type { Context, Model } from "../src/types.js";
import { getModel } from "../src/models.ts";
import { streamAnthropic } from "../src/providers/anthropic.ts";
import { streamOpenAICompletions } from "../src/providers/openai-completions.ts";
import { streamOpenAIResponses } from "../src/providers/openai-responses.ts";
import { stream } from "../src/stream.ts";
import type { Context, Model } from "../src/types.ts";
class PayloadCaptured extends Error {
constructor() {
super("payload captured");
this.name = "PayloadCaptured";
}
}
function stopAfterPayload<TPayload>(capture: (payload: TPayload) => void): (payload: unknown) => never {
return (payload: unknown): never => {
capture(payload as TPayload);
throw new PayloadCaptured();
};
}
describe("Cache Retention (PI_CACHE_RETENTION)", () => {
const originalEnv = process.env.PI_CACHE_RETENTION;
@@ -31,9 +48,9 @@ describe("Cache Retention (PI_CACHE_RETENTION)", () => {
let capturedPayload: any = null;
const s = stream(model, context, {
onPayload: (payload) => {
onPayload: stopAfterPayload((payload) => {
capturedPayload = payload;
},
}),
});
// Consume the stream to trigger the request
@@ -54,9 +71,9 @@ describe("Cache Retention (PI_CACHE_RETENTION)", () => {
let capturedPayload: any = null;
const s = stream(model, context, {
onPayload: (payload) => {
onPayload: stopAfterPayload((payload) => {
capturedPayload = payload;
},
}),
});
// Consume the stream to trigger the request
@@ -88,14 +105,13 @@ describe("Cache Retention (PI_CACHE_RETENTION)", () => {
// Since we can't easily test this without mocking, we'll skip the actual API call
// and just verify the helper logic works correctly
const { streamAnthropic } = await import("../src/providers/anthropic.js");
try {
const s = streamAnthropic(proxyModel, context, {
apiKey: "fake-key",
onPayload: (payload) => {
onPayload: stopAfterPayload((payload) => {
capturedPayload = payload;
},
}),
});
// This will fail since we're using a fake key and fake proxy, but the payload should be captured
@@ -119,15 +135,13 @@ describe("Cache Retention (PI_CACHE_RETENTION)", () => {
};
let capturedPayload: any = null;
const { streamAnthropic } = await import("../src/providers/anthropic.js");
try {
const s = streamAnthropic(proxyModel, context, {
apiKey: "fake-key",
cacheRetention: "long",
onPayload: (payload) => {
onPayload: stopAfterPayload((payload) => {
capturedPayload = payload;
},
}),
});
for await (const event of s) {
@@ -145,15 +159,13 @@ describe("Cache Retention (PI_CACHE_RETENTION)", () => {
const baseModel = getModel("anthropic", "claude-haiku-4-5");
let capturedPayload: any = null;
const { streamAnthropic } = await import("../src/providers/anthropic.js");
try {
const s = streamAnthropic(baseModel, context, {
apiKey: "fake-key",
cacheRetention: "none",
onPayload: (payload) => {
onPayload: stopAfterPayload((payload) => {
capturedPayload = payload;
},
}),
});
for await (const event of s) {
@@ -171,14 +183,12 @@ describe("Cache Retention (PI_CACHE_RETENTION)", () => {
const baseModel = getModel("anthropic", "claude-haiku-4-5");
let capturedPayload: any = null;
const { streamAnthropic } = await import("../src/providers/anthropic.js");
try {
const s = streamAnthropic(baseModel, context, {
apiKey: "fake-key",
onPayload: (payload) => {
onPayload: stopAfterPayload((payload) => {
capturedPayload = payload;
},
}),
});
for await (const event of s) {
@@ -199,15 +209,13 @@ describe("Cache Retention (PI_CACHE_RETENTION)", () => {
const baseModel = getModel("anthropic", "claude-haiku-4-5");
let capturedPayload: any = null;
const { streamAnthropic } = await import("../src/providers/anthropic.js");
try {
const s = streamAnthropic(baseModel, context, {
apiKey: "fake-key",
cacheRetention: "long",
onPayload: (payload) => {
onPayload: stopAfterPayload((payload) => {
capturedPayload = payload;
},
}),
});
for await (const event of s) {
@@ -230,9 +238,9 @@ describe("Cache Retention (PI_CACHE_RETENTION)", () => {
let capturedPayload: any = null;
const s = stream(model, context, {
onPayload: (payload) => {
onPayload: stopAfterPayload((payload) => {
capturedPayload = payload;
},
}),
});
// Consume the stream to trigger the request
@@ -253,9 +261,9 @@ describe("Cache Retention (PI_CACHE_RETENTION)", () => {
let capturedPayload: any = null;
const s = stream(model, context, {
onPayload: (payload) => {
onPayload: stopAfterPayload((payload) => {
capturedPayload = payload;
},
}),
});
// Consume the stream to trigger the request
@@ -280,14 +288,12 @@ describe("Cache Retention (PI_CACHE_RETENTION)", () => {
let capturedPayload: any = null;
const { streamOpenAIResponses } = await import("../src/providers/openai-responses.js");
try {
const s = streamOpenAIResponses(proxyModel, context, {
apiKey: "fake-key",
onPayload: (payload) => {
onPayload: stopAfterPayload((payload) => {
capturedPayload = payload;
},
}),
});
// This will fail since we're using a fake key and fake proxy, but the payload should be captured
@@ -309,16 +315,14 @@ describe("Cache Retention (PI_CACHE_RETENTION)", () => {
};
let capturedPayload: any = null;
const { streamOpenAIResponses } = await import("../src/providers/openai-responses.js");
try {
const s = streamOpenAIResponses(model, context, {
apiKey: "fake-key",
cacheRetention: "long",
sessionId: "session-compat-false",
onPayload: (payload) => {
onPayload: stopAfterPayload((payload) => {
capturedPayload = payload;
},
}),
});
for await (const event of s) {
@@ -336,16 +340,14 @@ describe("Cache Retention (PI_CACHE_RETENTION)", () => {
const model = getModel("openai", "gpt-4o-mini");
let capturedPayload: any = null;
const { streamOpenAIResponses } = await import("../src/providers/openai-responses.js");
try {
const s = streamOpenAIResponses(model, context, {
apiKey: "fake-key",
cacheRetention: "none",
sessionId: "session-1",
onPayload: (payload) => {
onPayload: stopAfterPayload((payload) => {
capturedPayload = payload;
},
}),
});
for await (const event of s) {
@@ -364,16 +366,14 @@ describe("Cache Retention (PI_CACHE_RETENTION)", () => {
const model = getModel("openai", "gpt-4o-mini");
let capturedPayload: any = null;
const { streamOpenAIResponses } = await import("../src/providers/openai-responses.js");
try {
const s = streamOpenAIResponses(model, context, {
apiKey: "fake-key",
cacheRetention: "long",
sessionId: "session-2",
onPayload: (payload) => {
onPayload: stopAfterPayload((payload) => {
capturedPayload = payload;
},
}),
});
for await (const event of s) {
@@ -408,16 +408,15 @@ describe("Cache Retention (PI_CACHE_RETENTION)", () => {
it("should set prompt_cache_retention for non-api.openai.com baseUrl by default", async () => {
let capturedPayload: any = null;
const { streamOpenAICompletions } = await import("../src/providers/openai-completions.js");
try {
const s = streamOpenAICompletions(createCompletionsModel(), context, {
apiKey: "fake-key",
cacheRetention: "long",
sessionId: "session-completions",
onPayload: (payload) => {
onPayload: stopAfterPayload((payload) => {
capturedPayload = payload;
},
}),
});
for await (const event of s) {
@@ -434,16 +433,15 @@ describe("Cache Retention (PI_CACHE_RETENTION)", () => {
it("should omit prompt_cache_retention when supportsLongCacheRetention is false", async () => {
let capturedPayload: any = null;
const { streamOpenAICompletions } = await import("../src/providers/openai-completions.js");
try {
const s = streamOpenAICompletions(createCompletionsModel({ supportsLongCacheRetention: false }), context, {
apiKey: "fake-key",
cacheRetention: "long",
sessionId: "session-completions-false",
onPayload: (payload) => {
onPayload: stopAfterPayload((payload) => {
capturedPayload = payload;
},
}),
});
for await (const event of s) {

View File

@@ -1,4 +1,4 @@
#!/usr/bin/env tsx
#!/usr/bin/env node
/**
* Live probe for OpenAI Codex Responses websocket-cached mode.
*
@@ -9,15 +9,15 @@
import { tmpdir } from "node:os";
import { join, resolve } from "node:path";
import { Type } from "typebox";
import { AuthStorage } from "../../coding-agent/src/core/auth-storage.js";
import { getModel } from "../src/models.js";
import { AuthStorage } from "../../coding-agent/src/core/auth-storage.ts";
import { getModel } from "../src/models.ts";
import {
closeOpenAICodexWebSocketSessions,
getOpenAICodexWebSocketDebugStats,
resetOpenAICodexWebSocketDebugStats,
streamOpenAICodexResponses,
} from "../src/providers/openai-codex-responses.js";
import type { AssistantMessage, Context, Message, Model, Tool, ToolResultMessage, Transport } from "../src/types.js";
} from "../src/providers/openai-codex-responses.ts";
import type { AssistantMessage, Context, Message, Model, Tool, ToolResultMessage, Transport } from "../src/types.ts";
type ThinkingLevel = "minimal" | "low" | "medium" | "high" | "xhigh";
@@ -85,7 +85,7 @@ function required(value: string | undefined, flag: string): string {
}
function printHelp(): void {
console.log(`Usage: npx tsx test/codex-websocket-cached-probe.ts [options]
console.log(`Usage: node test/codex-websocket-cached-probe.ts [options]
Options:
--turns <n> Number of user turns. Default: ${DEFAULT_TURNS}

View File

@@ -14,13 +14,13 @@
import type { ChildProcess } from "child_process";
import { execSync, spawn } from "child_process";
import { afterAll, beforeAll, describe, expect, it } from "vitest";
import { getModel } from "../src/models.js";
import { complete } from "../src/stream.js";
import type { AssistantMessage, Context, Model, Usage } from "../src/types.js";
import { isContextOverflow } from "../src/utils/overflow.js";
import { hasAzureOpenAICredentials } from "./azure-utils.js";
import { hasBedrockCredentials } from "./bedrock-utils.js";
import { resolveApiKey } from "./oauth.js";
import { getModel } from "../src/models.ts";
import { complete } from "../src/stream.ts";
import type { AssistantMessage, Context, Model, Usage } from "../src/types.ts";
import { isContextOverflow } from "../src/utils/overflow.ts";
import { hasAzureOpenAICredentials } from "./azure-utils.ts";
import { hasBedrockCredentials } from "./bedrock-utils.ts";
import { resolveApiKey } from "./oauth.ts";
// Resolve OAuth tokens at module level (async, runs before tests)
const oauthTokens = await Promise.all([resolveApiKey("github-copilot"), resolveApiKey("openai-codex")]);
@@ -143,7 +143,7 @@ describe("Context overflow error handling", () => {
it.skipIf(!githubCopilotToken)(
"claude-sonnet-4 - should detect overflow via isContextOverflow",
async () => {
const model = getModel("github-copilot", "claude-sonnet-4");
const model = getModel("github-copilot", "claude-sonnet-4.6");
const result = await testContextOverflow(model, githubCopilotToken!);
logResult(result);
@@ -228,9 +228,9 @@ describe("Context overflow error handling", () => {
describe("OpenAI Codex (OAuth)", () => {
it.skipIf(!openaiCodexToken)(
"gpt-5.2-codex - should detect overflow via isContextOverflow",
"gpt-5.5 - should detect overflow via isContextOverflow",
async () => {
const model = getModel("openai-codex", "gpt-5.2-codex");
const model = getModel("openai-codex", "gpt-5.5");
const result = await testContextOverflow(model, openaiCodexToken!);
logResult(result);

View File

@@ -25,12 +25,12 @@
import { writeFileSync } from "fs";
import { Type } from "typebox";
import { beforeAll, describe, expect, it } from "vitest";
import { getModel } from "../src/models.js";
import { completeSimple, getEnvApiKey } from "../src/stream.js";
import type { Api, AssistantMessage, Message, Model, Tool, ToolResultMessage } from "../src/types.js";
import { hasAzureOpenAICredentials } from "./azure-utils.js";
import { hasCloudflareAiGatewayCredentials, hasCloudflareWorkersAICredentials } from "./cloudflare-utils.js";
import { resolveApiKey } from "./oauth.js";
import { getModel } from "../src/models.ts";
import { completeSimple, getEnvApiKey } from "../src/stream.ts";
import type { Api, AssistantMessage, Message, Model, Tool, ToolResultMessage } from "../src/types.ts";
import { hasAzureOpenAICredentials } from "./azure-utils.ts";
import { hasCloudflareAiGatewayCredentials, hasCloudflareWorkersAICredentials } from "./cloudflare-utils.ts";
import { resolveApiKey } from "./oauth.ts";
// Simple tool for testing
const testToolSchema = Type.Object({
@@ -67,7 +67,7 @@ const PROVIDER_MODEL_PAIRS: ProviderModelPair[] = [
{ provider: "openai", model: "gpt-5-mini", label: "openai-responses-gpt-5-mini" },
{ provider: "azure-openai-responses", model: "gpt-4o-mini", label: "azure-openai-responses-gpt-4o-mini" },
// OpenAI Codex
{ provider: "openai-codex", model: "gpt-5.2-codex", label: "openai-codex-gpt-5.2-codex" },
{ provider: "openai-codex", model: "gpt-5.5", label: "openai-codex-gpt-5.5" },
// GitHub Copilot
{ provider: "github-copilot", model: "claude-sonnet-4.5", label: "copilot-claude-sonnet-4.5" },
{ provider: "github-copilot", model: "gpt-5.1-codex", label: "copilot-gpt-5.1-codex" },

View File

@@ -1,14 +1,14 @@
import { describe, expect, it } from "vitest";
import { getModel } from "../src/models.js";
import { complete } from "../src/stream.js";
import type { Api, AssistantMessage, Context, Model, StreamOptions, UserMessage } from "../src/types.js";
import { getModel } from "../src/models.ts";
import { complete } from "../src/stream.ts";
import type { Api, AssistantMessage, Context, Model, StreamOptions, UserMessage } from "../src/types.ts";
type StreamOptionsWithExtras = StreamOptions & Record<string, unknown>;
import { hasAzureOpenAICredentials, resolveAzureDeploymentName } from "./azure-utils.js";
import { hasBedrockCredentials } from "./bedrock-utils.js";
import { hasCloudflareAiGatewayCredentials, hasCloudflareWorkersAICredentials } from "./cloudflare-utils.js";
import { resolveApiKey } from "./oauth.js";
import { hasAzureOpenAICredentials, resolveAzureDeploymentName } from "./azure-utils.ts";
import { hasBedrockCredentials } from "./bedrock-utils.ts";
import { hasCloudflareAiGatewayCredentials, hasCloudflareWorkersAICredentials } from "./cloudflare-utils.ts";
import { resolveApiKey } from "./oauth.ts";
// Resolve OAuth tokens at module level (async, runs before tests)
const oauthTokens = await Promise.all([
@@ -669,7 +669,7 @@ describe("AI Providers Empty Message Tests", () => {
"claude-sonnet-4 - should handle empty content array",
{ retry: 3, timeout: 30000 },
async () => {
const llm = getModel("github-copilot", "claude-sonnet-4");
const llm = getModel("github-copilot", "claude-sonnet-4.6");
await testEmptyMessage(llm, { apiKey: githubCopilotToken });
},
);
@@ -678,7 +678,7 @@ describe("AI Providers Empty Message Tests", () => {
"claude-sonnet-4 - should handle empty string content",
{ retry: 3, timeout: 30000 },
async () => {
const llm = getModel("github-copilot", "claude-sonnet-4");
const llm = getModel("github-copilot", "claude-sonnet-4.6");
await testEmptyStringMessage(llm, { apiKey: githubCopilotToken });
},
);
@@ -687,7 +687,7 @@ describe("AI Providers Empty Message Tests", () => {
"claude-sonnet-4 - should handle whitespace-only content",
{ retry: 3, timeout: 30000 },
async () => {
const llm = getModel("github-copilot", "claude-sonnet-4");
const llm = getModel("github-copilot", "claude-sonnet-4.6");
await testWhitespaceOnlyMessage(llm, { apiKey: githubCopilotToken });
},
);
@@ -696,7 +696,7 @@ describe("AI Providers Empty Message Tests", () => {
"claude-sonnet-4 - should handle empty assistant message in conversation",
{ retry: 3, timeout: 30000 },
async () => {
const llm = getModel("github-copilot", "claude-sonnet-4");
const llm = getModel("github-copilot", "claude-sonnet-4.6");
await testEmptyAssistantMessage(llm, { apiKey: githubCopilotToken });
},
);
@@ -704,37 +704,37 @@ describe("AI Providers Empty Message Tests", () => {
describe("OpenAI Codex Provider Empty Messages", () => {
it.skipIf(!openaiCodexToken)(
"gpt-5.2-codex - should handle empty content array",
"gpt-5.5 - should handle empty content array",
{ retry: 3, timeout: 30000 },
async () => {
const llm = getModel("openai-codex", "gpt-5.2-codex");
const llm = getModel("openai-codex", "gpt-5.5");
await testEmptyMessage(llm, { apiKey: openaiCodexToken });
},
);
it.skipIf(!openaiCodexToken)(
"gpt-5.2-codex - should handle empty string content",
"gpt-5.5 - should handle empty string content",
{ retry: 3, timeout: 30000 },
async () => {
const llm = getModel("openai-codex", "gpt-5.2-codex");
const llm = getModel("openai-codex", "gpt-5.5");
await testEmptyStringMessage(llm, { apiKey: openaiCodexToken });
},
);
it.skipIf(!openaiCodexToken)(
"gpt-5.2-codex - should handle whitespace-only content",
"gpt-5.5 - should handle whitespace-only content",
{ retry: 3, timeout: 30000 },
async () => {
const llm = getModel("openai-codex", "gpt-5.2-codex");
const llm = getModel("openai-codex", "gpt-5.5");
await testWhitespaceOnlyMessage(llm, { apiKey: openaiCodexToken });
},
);
it.skipIf(!openaiCodexToken)(
"gpt-5.2-codex - should handle empty assistant message in conversation",
"gpt-5.5 - should handle empty assistant message in conversation",
{ retry: 3, timeout: 30000 },
async () => {
const llm = getModel("openai-codex", "gpt-5.2-codex");
const llm = getModel("openai-codex", "gpt-5.5");
await testEmptyAssistantMessage(llm, { apiKey: openaiCodexToken });
},
);

View File

@@ -0,0 +1,46 @@
import { afterEach, describe, expect, it } from "vitest";
import { findEnvKeys, getEnvApiKey } from "../src/env-api-keys.ts";
const originalCopilotGitHubToken = process.env.COPILOT_GITHUB_TOKEN;
const originalGhToken = process.env.GH_TOKEN;
const originalGitHubToken = process.env.GITHUB_TOKEN;
afterEach(() => {
if (originalCopilotGitHubToken === undefined) {
delete process.env.COPILOT_GITHUB_TOKEN;
} else {
process.env.COPILOT_GITHUB_TOKEN = originalCopilotGitHubToken;
}
if (originalGhToken === undefined) {
delete process.env.GH_TOKEN;
} else {
process.env.GH_TOKEN = originalGhToken;
}
if (originalGitHubToken === undefined) {
delete process.env.GITHUB_TOKEN;
} else {
process.env.GITHUB_TOKEN = originalGitHubToken;
}
});
describe("environment API keys", () => {
it("does not treat generic GitHub tokens as GitHub Copilot credentials", () => {
delete process.env.COPILOT_GITHUB_TOKEN;
process.env.GH_TOKEN = "gh-token";
process.env.GITHUB_TOKEN = "github-token";
expect(findEnvKeys("github-copilot")).toBeUndefined();
expect(getEnvApiKey("github-copilot")).toBeUndefined();
});
it("resolves GitHub Copilot credentials from COPILOT_GITHUB_TOKEN", () => {
process.env.COPILOT_GITHUB_TOKEN = "copilot-token";
process.env.GH_TOKEN = "gh-token";
process.env.GITHUB_TOKEN = "github-token";
expect(findEnvKeys("github-copilot")).toEqual(["COPILOT_GITHUB_TOKEN"]);
expect(getEnvApiKey("github-copilot")).toBe("copilot-token");
});
});

View File

@@ -8,8 +8,8 @@ import {
registerFauxProvider,
stream,
Type,
} from "../src/index.js";
import type { AssistantMessageEvent, Context } from "../src/types.js";
} from "../src/index.ts";
import type { AssistantMessageEvent, Context } from "../src/types.ts";
async function collectEvents(streamResult: ReturnType<typeof stream>): Promise<AssistantMessageEvent[]> {
const events: AssistantMessageEvent[] = [];

View File

@@ -1,6 +1,11 @@
import { createServer, type IncomingMessage, type ServerResponse } from "node:http";
import type { AddressInfo } from "node:net";
import { Type } from "typebox";
import { afterEach, describe, expect, it } from "vitest";
import { findEnvKeys, getEnvApiKey } from "../src/env-api-keys.js";
import { getModel } from "../src/models.js";
import { findEnvKeys, getEnvApiKey } from "../src/env-api-keys.ts";
import { getModel } from "../src/models.ts";
import { streamAnthropic } from "../src/providers/anthropic.ts";
import type { Context, Model, Tool } from "../src/types.ts";
const originalFireworksApiKey = process.env.FIREWORKS_API_KEY;
@@ -47,4 +52,197 @@ describe("Fireworks models", () => {
expect(findEnvKeys("fireworks")).toEqual(["FIREWORKS_API_KEY"]);
expect(getEnvApiKey("fireworks")).toBe("test-fireworks-key");
});
it("sets Fireworks-specific compat for session affinity and unsupported tool fields", () => {
const model = getModel("fireworks", "accounts/fireworks/models/kimi-k2p6");
expect(model.compat).toBeDefined();
expect(model.compat?.sendSessionAffinityHeaders).toBe(true);
expect(model.compat?.supportsEagerToolInputStreaming).toBe(false);
expect(model.compat?.supportsCacheControlOnTools).toBe(false);
expect(model.compat?.supportsLongCacheRetention).toBe(false);
});
});
// --- Integration tests for Fireworks Anthropic session affinity and tool compat ---
interface CapturedRequest {
headers: IncomingMessage["headers"];
body: Record<string, unknown>;
}
const tool: Tool = {
name: "lookup",
description: "Look up a value",
parameters: Type.Object({ value: Type.String() }),
};
function createFireworksModel(compat?: Model<"anthropic-messages">["compat"]): Model<"anthropic-messages"> {
return {
id: "accounts/fireworks/models/kimi-k2p6",
name: "Kimi K2.6",
api: "anthropic-messages",
provider: "fireworks",
baseUrl: "http://127.0.0.1:0", // overridden by captureAnthropicRequest
reasoning: true,
input: ["text", "image"],
cost: { input: 0.95, output: 4, cacheRead: 0.16, cacheWrite: 0 },
contextWindow: 262000,
maxTokens: 262000,
compat,
};
}
function createAnthropicModel(): Model<"anthropic-messages"> {
return {
id: "claude-opus-4-7",
name: "Claude Opus 4.7",
api: "anthropic-messages",
provider: "anthropic",
baseUrl: "http://127.0.0.1:0", // overridden by captureAnthropicRequest
reasoning: true,
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 200000,
maxTokens: 32000,
};
}
function createContext(tools: Tool[] = [tool]): Context {
return {
messages: [{ role: "user", content: "Use the tool", timestamp: Date.now() }],
...(tools.length > 0 ? { tools } : {}),
};
}
async function readRequestBody(request: IncomingMessage): Promise<Record<string, unknown>> {
const chunks: Buffer[] = [];
for await (const chunk of request) {
chunks.push(Buffer.isBuffer(chunk) ? chunk : Buffer.from(chunk));
}
return JSON.parse(Buffer.concat(chunks).toString("utf8")) as Record<string, unknown>;
}
function writeEmptySseResponse(response: ServerResponse): void {
response.writeHead(200, { "content-type": "text/event-stream" });
response.end();
}
async function captureAnthropicRequest(
model: Model<"anthropic-messages">,
context: Context,
options?: { sessionId?: string; cacheRetention?: string },
): Promise<CapturedRequest> {
let capturedRequest: CapturedRequest | undefined;
const server = createServer(async (request, response) => {
capturedRequest = {
headers: request.headers,
body: await readRequestBody(request),
};
writeEmptySseResponse(response);
});
await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
const address = server.address() as AddressInfo;
try {
// Override the model's baseUrl to point to the local test server
const localModel = { ...model, baseUrl: `http://127.0.0.1:${address.port}` };
const stream = streamAnthropic(localModel, context, {
apiKey: "test-key",
cacheRetention: (options?.cacheRetention as "none" | "short" | "long") ?? "short",
sessionId: options?.sessionId,
});
for await (const event of stream) {
if (event.type === "done" || event.type === "error") break;
}
} finally {
await new Promise<void>((resolve, reject) => {
server.close((error) => (error ? reject(error) : resolve()));
});
}
if (!capturedRequest) {
throw new Error("Anthropic request was not captured");
}
return capturedRequest;
}
function getTools(body: Record<string, unknown>): Record<string, unknown>[] {
const tools = body.tools;
if (!Array.isArray(tools)) {
throw new Error("Expected tools in request body");
}
return tools as Record<string, unknown>[];
}
describe("Fireworks Anthropic session affinity and tool compat", () => {
it("sends x-session-affinity header for Fireworks models", async () => {
const model = createFireworksModel();
// Need a real port, capture will assign one
const request = await captureAnthropicRequest(model, createContext(), {
sessionId: "fireworks-session-1",
});
expect(request.headers["x-session-affinity"]).toBe("fireworks-session-1");
});
it("omits x-session-affinity header for native Anthropic models", async () => {
const model = createAnthropicModel();
const request = await captureAnthropicRequest(model, createContext(), {
sessionId: "anthropic-session-1",
});
expect(request.headers["x-session-affinity"]).toBeUndefined();
});
it("omits x-session-affinity header when cacheRetention is none", async () => {
const model = createFireworksModel();
const request = await captureAnthropicRequest(model, createContext(), {
sessionId: "fireworks-session-2",
cacheRetention: "none",
});
expect(request.headers["x-session-affinity"]).toBeUndefined();
});
it("omits cache_control on tools for Fireworks models", async () => {
const model = createFireworksModel();
const request = await captureAnthropicRequest(model, createContext());
const tools = getTools(request.body);
const lastTool = tools[tools.length - 1];
expect(lastTool.cache_control).toBeUndefined();
});
it("omits eager_input_streaming on tools for Fireworks models", async () => {
const model = createFireworksModel();
const request = await captureAnthropicRequest(model, createContext());
const tools = getTools(request.body);
for (const t of tools) {
expect(t.eager_input_streaming).toBeUndefined();
}
});
it("sends cache_control on tools for native Anthropic models", async () => {
const model = createAnthropicModel();
const request = await captureAnthropicRequest(model, createContext());
const tools = getTools(request.body);
const lastTool = tools[tools.length - 1];
expect(lastTool.cache_control).toBeDefined();
expect((lastTool.cache_control as { type: string }).type).toBe("ephemeral");
});
it("sends eager_input_streaming on tools for native Anthropic models", async () => {
const model = createAnthropicModel();
const request = await captureAnthropicRequest(model, createContext());
const tools = getTools(request.body);
expect(tools[0].eager_input_streaming).toBe(true);
});
});

View File

@@ -1,6 +1,7 @@
import { describe, expect, it, vi } from "vitest";
import { getModel } from "../src/models.js";
import type { Context } from "../src/types.js";
import { getModel } from "../src/models.ts";
import { streamAnthropic } from "../src/providers/anthropic.ts";
import type { Context } from "../src/types.ts";
const mockState = vi.hoisted(() => ({
constructorOpts: undefined as Record<string, unknown> | undefined,
@@ -54,10 +55,9 @@ describe("Copilot Claude via Anthropic Messages", () => {
};
it("uses Bearer auth, Copilot headers, and valid Anthropic Messages payload", async () => {
const model = getModel("github-copilot", "claude-sonnet-4");
const model = getModel("github-copilot", "claude-sonnet-4.6");
expect(model.api).toBe("anthropic-messages");
const { streamAnthropic } = await import("../src/providers/anthropic.js");
const s = streamAnthropic(model, context, { apiKey: "tid_copilot_session_test_token" });
for await (const event of s) {
if (event.type === "error") break;
@@ -85,15 +85,14 @@ describe("Copilot Claude via Anthropic Messages", () => {
// Payload is valid Anthropic Messages format
const params = mockState.createParams!;
expect(params.model).toBe("claude-sonnet-4");
expect(params.model).toBe("claude-sonnet-4.6");
expect(params.stream).toBe(true);
expect(params.max_tokens).toBeGreaterThan(0);
expect(params.max_tokens).toBe(model.maxTokens);
expect(Array.isArray(params.messages)).toBe(true);
});
it("includes interleaved-thinking beta when reasoning is enabled", async () => {
const model = getModel("github-copilot", "claude-sonnet-4");
const { streamAnthropic } = await import("../src/providers/anthropic.js");
it("omits interleaved-thinking beta for adaptive-thinking models", async () => {
const model = getModel("github-copilot", "claude-sonnet-4.6");
const s = streamAnthropic(model, context, {
apiKey: "tid_copilot_session_test_token",
interleavedThinking: true,
@@ -103,6 +102,6 @@ describe("Copilot Claude via Anthropic Messages", () => {
}
const headers = mockState.constructorOpts!.defaultHeaders as Record<string, string>;
expect(headers["anthropic-beta"]).toContain("interleaved-thinking-2025-05-14");
expect(headers["anthropic-beta"] ?? "").not.toContain("interleaved-thinking-2025-05-14");
});
});

View File

@@ -1,5 +1,5 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import { loginGitHubCopilot } from "../src/utils/oauth/github-copilot.js";
import { loginGitHubCopilot } from "../src/utils/oauth/github-copilot.ts";
function jsonResponse(body: unknown, status: number = 200): Response {
return new Response(JSON.stringify(body), {

View File

@@ -1,6 +1,6 @@
import { describe, expect, it } from "vitest";
import { convertTools } from "../src/providers/google-shared.js";
import type { Tool } from "../src/types.js";
import { convertTools } from "../src/providers/google-shared.ts";
import type { Tool } from "../src/types.ts";
function makeTool(parameters: Record<string, unknown>): Tool {
return {

View File

@@ -1,6 +1,6 @@
import { describe, expect, it } from "vitest";
import { convertMessages } from "../src/providers/google-shared.js";
import type { Context, Model } from "../src/types.js";
import { convertMessages } from "../src/providers/google-shared.ts";
import type { Context, Model } from "../src/types.ts";
function makeGemini3Model<TApi extends "google-generative-ai" | "google-vertex">(
api: TApi,

View File

@@ -1,6 +1,6 @@
import { describe, expect, it } from "vitest";
import { convertMessages } from "../src/providers/google-shared.js";
import type { Context, Model } from "../src/types.js";
import { convertMessages } from "../src/providers/google-shared.ts";
import type { Context, Model } from "../src/types.ts";
function makeModel<TApi extends "google-generative-ai">(
api: TApi,

View File

@@ -1,7 +1,7 @@
import { describe, expect, it } from "vitest";
import { getModel } from "../src/models.js";
import { streamSimple } from "../src/stream.js";
import type { Api, Context, Model, SimpleStreamOptions } from "../src/types.js";
import { getModel } from "../src/models.ts";
import { streamSimple } from "../src/stream.ts";
import type { Api, Context, Model, SimpleStreamOptions } from "../src/types.ts";
type SimpleOptionsWithExtras = SimpleStreamOptions & Record<string, unknown>;

View File

@@ -1,5 +1,5 @@
import { describe, expect, it } from "vitest";
import { isThinkingPart, retainThoughtSignature } from "../src/providers/google-shared.js";
import { isThinkingPart, retainThoughtSignature } from "../src/providers/google-shared.ts";
describe("Google thinking detection (thoughtSignature)", () => {
it("treats part.thought === true as thinking", () => {

View File

@@ -45,9 +45,9 @@ vi.mock("@google/genai", () => {
};
});
import { getModel } from "../src/models.js";
import { streamGoogleVertex } from "../src/providers/google-vertex.js";
import type { Context, Model } from "../src/types.js";
import { getModel } from "../src/models.ts";
import { streamGoogleVertex } from "../src/providers/google-vertex.ts";
import type { Context, Model } from "../src/types.ts";
const model = getModel("google-vertex", "gemini-3-flash-preview");
const context: Context = {

View File

@@ -2,15 +2,15 @@ import { readFileSync } from "node:fs";
import { join } from "node:path";
import { Type } from "typebox";
import { describe, expect, it } from "vitest";
import type { Api, Context, Model, Tool, ToolResultMessage } from "../src/index.js";
import { complete, getModel } from "../src/index.js";
import type { StreamOptions } from "../src/types.js";
import type { Api, Context, Model, Tool, ToolResultMessage } from "../src/index.ts";
import { complete, getModel } from "../src/index.ts";
import type { StreamOptions } from "../src/types.ts";
type StreamOptionsWithExtras = StreamOptions & Record<string, unknown>;
import { hasAzureOpenAICredentials, resolveAzureDeploymentName } from "./azure-utils.js";
import { hasBedrockCredentials } from "./bedrock-utils.js";
import { resolveApiKey } from "./oauth.js";
import { hasAzureOpenAICredentials, resolveAzureDeploymentName } from "./azure-utils.ts";
import { hasBedrockCredentials } from "./bedrock-utils.ts";
import { resolveApiKey } from "./oauth.ts";
// Resolve OAuth tokens at module level (async, runs before tests)
const oauthTokens = await Promise.all([
@@ -464,7 +464,7 @@ describe("Tool Results with Images", () => {
"claude-sonnet-4 - should handle tool result with only image",
{ retry: 3, timeout: 30000 },
async () => {
const llm = getModel("github-copilot", "claude-sonnet-4");
const llm = getModel("github-copilot", "claude-sonnet-4.6");
await handleToolWithImageResult(llm, { apiKey: githubCopilotToken });
},
);
@@ -473,7 +473,7 @@ describe("Tool Results with Images", () => {
"claude-sonnet-4 - should handle tool result with text and image",
{ retry: 3, timeout: 30000 },
async () => {
const llm = getModel("github-copilot", "claude-sonnet-4");
const llm = getModel("github-copilot", "claude-sonnet-4.6");
await handleToolWithTextAndImageResult(llm, { apiKey: githubCopilotToken });
},
);
@@ -481,19 +481,19 @@ describe("Tool Results with Images", () => {
describe("OpenAI Codex Provider", () => {
it.skipIf(!openaiCodexToken)(
"gpt-5.2-codex - should handle tool result with only image",
"gpt-5.5 - should handle tool result with only image",
{ retry: 3, timeout: 30000 },
async () => {
const llm = getModel("openai-codex", "gpt-5.2-codex");
const llm = getModel("openai-codex", "gpt-5.5");
await handleToolWithImageResult(llm, { apiKey: openaiCodexToken });
},
);
it.skipIf(!openaiCodexToken)(
"gpt-5.2-codex - should handle tool result with text and image",
"gpt-5.5 - should handle tool result with text and image",
{ retry: 3, timeout: 30000 },
async () => {
const llm = getModel("openai-codex", "gpt-5.2-codex");
const llm = getModel("openai-codex", "gpt-5.5");
await handleToolWithTextAndImageResult(llm, { apiKey: openaiCodexToken });
},
);

View File

@@ -2,9 +2,9 @@ import { readFileSync } from "node:fs";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
import { describe, expect, it } from "vitest";
import { getImageModel } from "../src/image-models.js";
import { generateImages } from "../src/images.js";
import type { ImageContent, ImagesContext, ImagesModel, ProviderImagesOptions } from "../src/types.js";
import { getImageModel } from "../src/image-models.ts";
import { generateImages } from "../src/images.ts";
import type { ImageContent, ImagesContext, ImagesModel, ProviderImagesOptions } from "../src/types.ts";
const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);

View File

@@ -1,11 +1,11 @@
import { Type } from "typebox";
import { describe, expect, it } from "vitest";
import { getEnvApiKey } from "../src/env-api-keys.js";
import { getModel } from "../src/models.js";
import { completeSimple } from "../src/stream.js";
import type { Api, Context, Model, StopReason, Tool, ToolCall, ToolResultMessage } from "../src/types.js";
import { StringEnum } from "../src/utils/typebox-helpers.js";
import { hasBedrockCredentials } from "./bedrock-utils.js";
import { getEnvApiKey } from "../src/env-api-keys.ts";
import { getModel } from "../src/models.ts";
import { completeSimple } from "../src/stream.ts";
import type { Api, Context, Model, StopReason, Tool, ToolCall, ToolResultMessage } from "../src/types.ts";
import { StringEnum } from "../src/utils/typebox-helpers.ts";
import { hasBedrockCredentials } from "./bedrock-utils.ts";
const calculatorSchema = Type.Object({
a: Type.Number({ description: "First number" }),

View File

@@ -1,11 +1,8 @@
import { spawnSync } from "node:child_process";
import { createRequire } from "node:module";
import { dirname, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import { describe, expect, it } from "vitest";
const require = createRequire(import.meta.url);
const tsxLoader = require.resolve("tsx/esm");
const packageRoot = resolve(dirname(fileURLToPath(import.meta.url)), "..");
const aiEntryUrl = new URL("../src/index.ts", import.meta.url).href;
@@ -42,7 +39,7 @@ function runProbe(action: string): ProbeResult {
console.log(JSON.stringify({ loadedSpecifiers: [...new Set(loaded)] }));
`;
const result = spawnSync(process.execPath, ["--import", tsxLoader, "--input-type=module", "--eval", script], {
const result = spawnSync(process.execPath, ["--input-type=module", "--eval", script], {
cwd: packageRoot,
encoding: "utf8",
});

View File

@@ -1,7 +1,7 @@
import { describe, expect, it } from "vitest";
import { getModel } from "../src/models.js";
import { streamSimple } from "../src/stream.js";
import type { Context, Model, SimpleStreamOptions } from "../src/types.js";
import { getModel } from "../src/models.ts";
import { streamSimple } from "../src/stream.ts";
import type { Context, Model, SimpleStreamOptions } from "../src/types.ts";
interface MistralPayload {
promptMode?: "reasoning";

View File

@@ -1,8 +1,8 @@
import { Type } from "typebox";
import { describe, expect, it } from "vitest";
import { getModel } from "../src/models.js";
import { complete } from "../src/stream.js";
import type { Context, Model } from "../src/types.js";
import { getModel } from "../src/models.ts";
import { complete } from "../src/stream.ts";
import type { Context, Model } from "../src/types.ts";
interface MistralToolPayload {
tools?: Array<{

View File

@@ -0,0 +1,65 @@
import { afterEach, describe, expect, it } from "vitest";
import { resolveHttpProxyUrlForTarget, UNSUPPORTED_PROXY_PROTOCOL_MESSAGE } from "../src/utils/node-http-proxy.ts";
const PROXY_ENV_KEYS = [
"HTTP_PROXY",
"HTTPS_PROXY",
"NO_PROXY",
"ALL_PROXY",
"http_proxy",
"https_proxy",
"no_proxy",
"all_proxy",
"npm_config_http_proxy",
"npm_config_https_proxy",
"npm_config_proxy",
"npm_config_no_proxy",
] as const;
const originalEnv = new Map<string, string | undefined>();
for (const key of PROXY_ENV_KEYS) {
originalEnv.set(key, process.env[key]);
}
function resetProxyEnv(): void {
for (const key of PROXY_ENV_KEYS) {
delete process.env[key];
}
}
afterEach(() => {
resetProxyEnv();
for (const [key, value] of originalEnv) {
if (value !== undefined) {
process.env[key] = value;
}
}
});
describe("node HTTP proxy resolution", () => {
it("respects NO_PROXY exclusions", () => {
resetProxyEnv();
process.env.HTTPS_PROXY = "http://proxy.example:8080";
process.env.NO_PROXY = "bedrock-runtime.us-east-1.amazonaws.com";
expect(resolveHttpProxyUrlForTarget("https://bedrock-runtime.us-east-1.amazonaws.com")).toBeUndefined();
});
it("resolves HTTP and HTTPS proxy URLs", () => {
resetProxyEnv();
process.env.HTTPS_PROXY = "http://proxy.example:8080";
expect(resolveHttpProxyUrlForTarget("https://bedrock-runtime.us-east-1.amazonaws.com")?.toString()).toBe(
"http://proxy.example:8080/",
);
});
it("rejects SOCKS and PAC proxy URLs explicitly", () => {
resetProxyEnv();
process.env.HTTPS_PROXY = "socks5://proxy.example:1080";
expect(() => resolveHttpProxyUrlForTarget("https://bedrock-runtime.us-east-1.amazonaws.com")).toThrow(
UNSUPPORTED_PROXY_PROTOCOL_MESSAGE,
);
});
});

View File

@@ -8,8 +8,8 @@
import { chmodSync, existsSync, mkdirSync, readFileSync, writeFileSync } from "fs";
import { homedir } from "os";
import { dirname, join } from "path";
import { getOAuthApiKey } from "../src/utils/oauth/index.js";
import type { OAuthCredentials, OAuthProvider } from "../src/utils/oauth/types.js";
import { getOAuthApiKey } from "../src/utils/oauth/index.ts";
import type { OAuthCredentials, OAuthProvider } from "../src/utils/oauth/types.ts";
const AUTH_PATH = join(homedir(), ".pi", "agent", "auth.json");
@@ -74,7 +74,12 @@ export async function resolveApiKey(provider: string): Promise<string | undefine
}
}
const result = await getOAuthApiKey(provider as OAuthProvider, oauthCredentials);
let result: { newCredentials: OAuthCredentials; apiKey: string } | null = null;
try {
result = await getOAuthApiKey(provider as OAuthProvider, oauthCredentials);
} catch (e) {
console.log(JSON.stringify(e));
}
if (!result) return undefined;
// Save refreshed credentials back to auth.json

View File

@@ -1,14 +1,14 @@
import { describe, expect, it } from "vitest";
import { getModel } from "../src/models.js";
import { complete } from "../src/stream.js";
import type { Context } from "../src/types.js";
import { resolveApiKey } from "./oauth.js";
import { getModel } from "../src/models.ts";
import { complete } from "../src/stream.ts";
import type { Context } from "../src/types.ts";
import { resolveApiKey } from "./oauth.ts";
const codexToken = await resolveApiKey("openai-codex");
describe("openai-codex cache affinity e2e", () => {
it.skipIf(!codexToken)("handles SSE requests with aligned cache-affinity identifiers", async () => {
const model = getModel("openai-codex", "gpt-5.3-codex");
const model = getModel("openai-codex", "gpt-5.5");
const sessionId = "0195d6e4-4cf9-7f44-a2d8-f8f7f49ee9d3";
const context: Context = {
systemPrompt: "You are a helpful assistant. Reply exactly as requested.",

View File

@@ -1,5 +1,5 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import { refreshOpenAICodexToken } from "../src/utils/oauth/openai-codex.js";
import { refreshOpenAICodexToken } from "../src/utils/oauth/openai-codex.ts";
describe("OpenAI Codex OAuth", () => {
afterEach(() => {

View File

@@ -7,22 +7,20 @@ import {
resetOpenAICodexWebSocketDebugStats,
streamOpenAICodexResponses,
streamSimpleOpenAICodexResponses,
} from "../src/providers/openai-codex-responses.js";
import type { Context, Model } from "../src/types.js";
} from "../src/providers/openai-codex-responses.ts";
import type { Context, Model } from "../src/types.ts";
const originalFetch = global.fetch;
const originalWebSocket = globalThis.WebSocket;
const originalAgentDir = process.env.PI_CODING_AGENT_DIR;
afterEach(() => {
global.fetch = originalFetch;
globalThis.WebSocket = originalWebSocket;
vi.unstubAllGlobals();
if (originalAgentDir === undefined) {
delete process.env.PI_CODING_AGENT_DIR;
} else {
process.env.PI_CODING_AGENT_DIR = originalAgentDir;
}
resetOpenAICodexWebSocketDebugStats();
vi.useRealTimers();
vi.restoreAllMocks();
});
@@ -155,7 +153,7 @@ describe("openai-codex streaming", () => {
return new Response("not found", { status: 404 });
});
global.fetch = fetchMock as typeof fetch;
vi.stubGlobal("fetch", fetchMock);
const model: Model<"openai-codex-responses"> = {
id: "gpt-5.1-codex",
@@ -175,7 +173,7 @@ describe("openai-codex streaming", () => {
messages: [{ role: "user", content: "Say hello", timestamp: Date.now() }],
};
const streamResult = streamOpenAICodexResponses(model, context, { apiKey: token });
const streamResult = streamOpenAICodexResponses(model, context, { apiKey: token, transport: "sse" });
let sawTextDelta = false;
let sawDone = false;
@@ -206,7 +204,7 @@ describe("openai-codex streaming", () => {
},
});
global.fetch = vi.fn(async (input: string | URL) => {
const fetchMock = vi.fn(async (input: string | URL) => {
const url = typeof input === "string" ? input : input.toString();
if (url === "https://api.github.com/repos/openai/codex/releases/latest") {
return new Response(JSON.stringify({ tag_name: "rust-v0.0.0" }), { status: 200 });
@@ -221,7 +219,8 @@ describe("openai-codex streaming", () => {
});
}
return new Response("not found", { status: 404 });
}) as typeof fetch;
});
vi.stubGlobal("fetch", fetchMock);
const model: Model<"openai-codex-responses"> = {
id: "gpt-5.1-codex",
@@ -265,7 +264,7 @@ describe("openai-codex streaming", () => {
},
});
global.fetch = vi.fn(async (input: string | URL) => {
const fetchMock = vi.fn(async (input: string | URL) => {
const url = typeof input === "string" ? input : input.toString();
if (url === "https://api.github.com/repos/openai/codex/releases/latest") {
return new Response(JSON.stringify({ tag_name: "rust-v0.0.0" }), { status: 200 });
@@ -280,7 +279,8 @@ describe("openai-codex streaming", () => {
});
}
return new Response("not found", { status: 404 });
}) as typeof fetch;
});
vi.stubGlobal("fetch", fetchMock);
const model: Model<"openai-codex-responses"> = {
id: "gpt-5.1-codex",
@@ -387,7 +387,7 @@ describe("openai-codex streaming", () => {
return new Response("not found", { status: 404 });
});
global.fetch = fetchMock as typeof fetch;
vi.stubGlobal("fetch", fetchMock);
const model: Model<"openai-codex-responses"> = {
id: "gpt-5.1-codex",
@@ -407,10 +407,60 @@ describe("openai-codex streaming", () => {
messages: [{ role: "user", content: "Say hello", timestamp: Date.now() }],
};
const streamResult = streamOpenAICodexResponses(model, context, { apiKey: token, sessionId });
const streamResult = streamOpenAICodexResponses(model, context, { apiKey: token, sessionId, transport: "sse" });
await streamResult.result();
});
it("clamps prompt_cache_key to OpenAI's 64-character limit", async () => {
const token = mockToken();
const sessionId = "x".repeat(67);
let capturedPayload: { prompt_cache_key?: string } | undefined;
const encoder = new TextEncoder();
vi.stubGlobal(
"fetch",
vi.fn(
async () =>
new Response(
new ReadableStream<Uint8Array>({
start(controller) {
controller.enqueue(encoder.encode(buildSSEPayload({ status: "completed" })));
controller.close();
},
}),
{ status: 200, headers: { "content-type": "text/event-stream" } },
),
),
);
const model: Model<"openai-codex-responses"> = {
id: "gpt-5.1-codex",
name: "GPT-5.1 Codex",
api: "openai-codex-responses",
provider: "openai-codex",
baseUrl: "https://chatgpt.com/backend-api",
reasoning: true,
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 400000,
maxTokens: 128000,
};
const context: Context = {
systemPrompt: "You are a helpful assistant.",
messages: [{ role: "user", content: "Say hello", timestamp: Date.now() }],
};
await streamOpenAICodexResponses(model, context, {
apiKey: token,
transport: "sse",
sessionId,
onPayload: (payload) => {
capturedPayload = payload as { prompt_cache_key?: string };
},
}).result();
expect(capturedPayload?.prompt_cache_key).toBe("x".repeat(64));
});
it("preserves gpt-5.5 xhigh reasoning effort from simple options", async () => {
const tempDir = mkdtempSync(join(tmpdir(), "pi-codex-stream-"));
process.env.PI_CODING_AGENT_DIR = tempDir;
@@ -425,7 +475,7 @@ describe("openai-codex streaming", () => {
});
let requestedReasoning: unknown;
global.fetch = vi.fn(async (input: string | URL, init?: RequestInit) => {
const fetchMock = vi.fn(async (input: string | URL, init?: RequestInit) => {
const url = typeof input === "string" ? input : input.toString();
if (url === "https://api.github.com/repos/openai/codex/releases/latest") {
return new Response(JSON.stringify({ tag_name: "rust-v0.0.0" }), { status: 200 });
@@ -442,7 +492,8 @@ describe("openai-codex streaming", () => {
});
}
return new Response("not found", { status: 404 });
}) as typeof fetch;
});
vi.stubGlobal("fetch", fetchMock);
const model: Model<"openai-codex-responses"> = {
id: "gpt-5.5",
@@ -462,7 +513,11 @@ describe("openai-codex streaming", () => {
messages: [{ role: "user", content: "Say hello", timestamp: Date.now() }],
};
await streamSimpleOpenAICodexResponses(model, context, { apiKey: token, reasoning: "xhigh" }).result();
await streamSimpleOpenAICodexResponses(model, context, {
apiKey: token,
reasoning: "xhigh",
transport: "sse",
}).result();
expect(requestedReasoning).toEqual({ effort: "xhigh", summary: "auto" });
});
@@ -508,6 +563,7 @@ describe("openai-codex streaming", () => {
})}`,
].join("\n\n")}\n\n`;
let requestedReasoning: unknown;
const encoder = new TextEncoder();
const stream = new ReadableStream<Uint8Array>({
start(controller) {
@@ -526,7 +582,7 @@ describe("openai-codex streaming", () => {
}
if (url === "https://chatgpt.com/backend-api/codex/responses") {
const body = typeof init?.body === "string" ? (JSON.parse(init.body) as Record<string, unknown>) : null;
expect(body?.reasoning).toEqual({ effort: "low", summary: "auto" });
requestedReasoning = body?.reasoning;
return new Response(stream, {
status: 200,
@@ -536,7 +592,7 @@ describe("openai-codex streaming", () => {
return new Response("not found", { status: 404 });
});
global.fetch = fetchMock as typeof fetch;
vi.stubGlobal("fetch", fetchMock);
const model: Model<"openai-codex-responses"> = {
id: modelId,
@@ -545,6 +601,7 @@ describe("openai-codex streaming", () => {
provider: "openai-codex",
baseUrl: "https://chatgpt.com/backend-api",
reasoning: true,
thinkingLevelMap: { minimal: "low" },
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 400000,
@@ -559,8 +616,10 @@ describe("openai-codex streaming", () => {
const streamResult = streamOpenAICodexResponses(model, context, {
apiKey: token,
reasoningEffort: "minimal",
transport: "sse",
});
await streamResult.result();
expect(requestedReasoning).toEqual({ effort: "low", summary: "auto" });
});
it.each([
@@ -614,7 +673,7 @@ describe("openai-codex streaming", () => {
},
});
global.fetch = vi.fn(async (input: string | URL) => {
const fetchMock = vi.fn(async (input: string | URL) => {
const url = typeof input === "string" ? input : input.toString();
if (url === "https://api.github.com/repos/openai/codex/releases/latest") {
return new Response(JSON.stringify({ tag_name: "rust-v0.0.0" }), { status: 200 });
@@ -629,7 +688,8 @@ describe("openai-codex streaming", () => {
});
}
return new Response("not found", { status: 404 });
}) as typeof fetch;
});
vi.stubGlobal("fetch", fetchMock);
const model: Model<"openai-codex-responses"> = {
id: modelId,
@@ -649,7 +709,11 @@ describe("openai-codex streaming", () => {
messages: [{ role: "user", content: "Say hello", timestamp: Date.now() }],
};
const result = await streamOpenAICodexResponses(model, context, { apiKey: token, serviceTier }).result();
const result = await streamOpenAICodexResponses(model, context, {
apiKey: token,
serviceTier,
transport: "sse",
}).result();
expect(result.usage.cost.input).toBe(1 * multiplier);
expect(result.usage.cost.output).toBe(2 * multiplier);
@@ -728,7 +792,7 @@ describe("openai-codex streaming", () => {
return new Response("not found", { status: 404 });
});
global.fetch = fetchMock as typeof fetch;
vi.stubGlobal("fetch", fetchMock);
const model: Model<"openai-codex-responses"> = {
id: "gpt-5.1-codex",
@@ -749,14 +813,15 @@ describe("openai-codex streaming", () => {
};
// No sessionId provided
const streamResult = streamOpenAICodexResponses(model, context, { apiKey: token });
const streamResult = streamOpenAICodexResponses(model, context, { apiKey: token, transport: "sse" });
await streamResult.result();
});
it("forwards auto transport from streamSimple options and uses cached websocket context", async () => {
const token = mockToken();
const sentBodies: unknown[] = [];
global.fetch = vi.fn(async () => new Response("unexpected fetch", { status: 500 })) as typeof fetch;
const fetchMock = vi.fn(async () => new Response("unexpected fetch", { status: 500 }));
vi.stubGlobal("fetch", fetchMock);
class MockWebSocket {
private listeners = new Map<string, Set<(event: unknown) => void>>();
@@ -826,7 +891,7 @@ describe("openai-codex streaming", () => {
}
}
globalThis.WebSocket = MockWebSocket as unknown as typeof WebSocket;
vi.stubGlobal("WebSocket", MockWebSocket);
const model: Model<"openai-codex-responses"> = {
id: "gpt-5.1-codex",
@@ -949,7 +1014,7 @@ describe("openai-codex streaming", () => {
}
}
globalThis.WebSocket = MockWebSocket as unknown as typeof WebSocket;
vi.stubGlobal("WebSocket", MockWebSocket);
const model: Model<"openai-codex-responses"> = {
id: "gpt-5.1-codex",
@@ -1005,4 +1070,142 @@ describe("openai-codex streaming", () => {
lastPreviousResponseId: "resp_1",
});
});
it.each([
["retry-after-ms", () => ({ "content-type": "application/json", "retry-after-ms": "1500" }), 1500],
["retry-after seconds", () => ({ "content-type": "application/json", "retry-after": "60" }), 60_000],
[
"retry-after HTTP date",
() => ({ "content-type": "application/json", "retry-after": new Date(Date.now() + 45_000).toUTCString() }),
45_000,
],
] as const)("uses %s for SSE retries", async (_name, makeHeaders, expectedDelay) => {
vi.useFakeTimers();
vi.setSystemTime(new Date("2026-05-13T00:00:00Z"));
const setTimeoutSpy = vi.spyOn(globalThis, "setTimeout");
const token = mockToken();
const encoder = new TextEncoder();
const sse = buildSSEPayload({ status: "completed" });
let codexRequests = 0;
const fetchMock = vi.fn(async (input: string | URL) => {
const url = typeof input === "string" ? input : input.toString();
if (url !== "https://chatgpt.com/backend-api/codex/responses") {
throw new Error(`Unexpected URL: ${url}`);
}
codexRequests++;
if (codexRequests === 1) {
return new Response(JSON.stringify({ error: { code: "rate_limit_exceeded", message: "rate limited" } }), {
status: 429,
headers: makeHeaders(),
});
}
return new Response(
new ReadableStream<Uint8Array>({
start(controller) {
controller.enqueue(encoder.encode(sse));
controller.close();
},
}),
{ status: 200, headers: { "content-type": "text/event-stream" } },
);
});
vi.stubGlobal("fetch", fetchMock);
const model: Model<"openai-codex-responses"> = {
id: "gpt-5.1-codex",
name: "GPT-5.1 Codex",
api: "openai-codex-responses",
provider: "openai-codex",
baseUrl: "https://chatgpt.com/backend-api",
reasoning: true,
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 400000,
maxTokens: 128000,
};
const context: Context = {
systemPrompt: "You are a helpful assistant.",
messages: [{ role: "user", content: "Say hello", timestamp: Date.now() }],
};
const resultPromise = streamOpenAICodexResponses(model, context, { apiKey: token, transport: "sse" }).result();
await vi.advanceTimersByTimeAsync(0);
expect(setTimeoutSpy).toHaveBeenCalledWith(expect.any(Function), expectedDelay);
await vi.advanceTimersToNextTimerAsync();
const result = await resultPromise;
expect(result.content.find((content) => content.type === "text")?.text).toBe("Hello");
expect(codexRequests).toBe(2);
});
it("uses exponential backoff across repeated SSE retries without retry headers", async () => {
vi.useFakeTimers();
vi.setSystemTime(new Date("2026-05-13T00:00:00Z"));
const setTimeoutSpy = vi.spyOn(globalThis, "setTimeout");
const token = mockToken();
const encoder = new TextEncoder();
const sse = buildSSEPayload({ status: "completed" });
let codexRequests = 0;
const fetchMock = vi.fn(async (input: string | URL) => {
const url = typeof input === "string" ? input : input.toString();
if (url !== "https://chatgpt.com/backend-api/codex/responses") {
throw new Error(`Unexpected URL: ${url}`);
}
codexRequests++;
if (codexRequests <= 3) {
return new Response(JSON.stringify({ error: { code: "rate_limit_exceeded", message: "rate limited" } }), {
status: 429,
headers: { "content-type": "application/json" },
});
}
return new Response(
new ReadableStream<Uint8Array>({
start(controller) {
controller.enqueue(encoder.encode(sse));
controller.close();
},
}),
{ status: 200, headers: { "content-type": "text/event-stream" } },
);
});
vi.stubGlobal("fetch", fetchMock);
const model: Model<"openai-codex-responses"> = {
id: "gpt-5.1-codex",
name: "GPT-5.1 Codex",
api: "openai-codex-responses",
provider: "openai-codex",
baseUrl: "https://chatgpt.com/backend-api",
reasoning: true,
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 400000,
maxTokens: 128000,
};
const context: Context = {
systemPrompt: "You are a helpful assistant.",
messages: [{ role: "user", content: "Say hello", timestamp: Date.now() }],
};
const resultPromise = streamOpenAICodexResponses(model, context, { apiKey: token, transport: "sse" }).result();
await vi.advanceTimersByTimeAsync(0);
expect(setTimeoutSpy).toHaveBeenNthCalledWith(1, expect.any(Function), 1000);
await vi.advanceTimersToNextTimerAsync();
expect(setTimeoutSpy).toHaveBeenNthCalledWith(2, expect.any(Function), 2000);
await vi.advanceTimersToNextTimerAsync();
expect(setTimeoutSpy).toHaveBeenNthCalledWith(3, expect.any(Function), 4000);
await vi.advanceTimersToNextTimerAsync();
const result = await resultPromise;
expect(result.content.find((content) => content.type === "text")?.text).toBe("Hello");
expect(codexRequests).toBe(4);
});
});

View File

@@ -1,8 +1,8 @@
import { Type } from "typebox";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { getModel } from "../src/models.js";
import { streamOpenAICompletions } from "../src/providers/openai-completions.js";
import type { Model } from "../src/types.js";
import { getModel } from "../src/models.ts";
import { streamOpenAICompletions } from "../src/providers/openai-completions.ts";
import type { Model } from "../src/types.ts";
interface CacheControl {
type: "ephemeral";

View File

@@ -1,6 +1,6 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import { getModel } from "../src/models.js";
import { streamSimple } from "../src/stream.js";
import { getModel } from "../src/models.ts";
import { streamSimple } from "../src/stream.ts";
// Empty tools arrays must NOT be serialized as `tools: []` — some OpenAI-compatible
// backends (e.g. DashScope / Aliyun Qwen via compatible-mode) reject the request with
@@ -93,6 +93,40 @@ describe("openai-completions empty tools handling", () => {
expect("tools" in (params as object)).toBe(false);
});
it("does not send default max token fields", async () => {
const { compat: _compat, ...baseModel } = getModel("openai", "gpt-4o-mini")!;
const model = { ...baseModel, api: "openai-completions" } as const;
await streamSimple(
model,
{
messages: [{ role: "user", content: "hi", timestamp: Date.now() }],
},
{ apiKey: "test" },
).result();
const params = mockState.lastParams as { max_tokens?: number; max_completion_tokens?: number };
expect(params.max_tokens).toBeUndefined();
expect(params.max_completion_tokens).toBeUndefined();
});
it("sends explicit maxTokens", async () => {
const { compat: _compat, ...baseModel } = getModel("openai", "gpt-4o-mini")!;
const model = { ...baseModel, api: "openai-completions" } as const;
await streamSimple(
model,
{
messages: [{ role: "user", content: "hi", timestamp: Date.now() }],
},
{ apiKey: "test", maxTokens: 1234 },
).result();
const params = mockState.lastParams as { max_tokens?: number; max_completion_tokens?: number };
expect(params.max_tokens).toBeUndefined();
expect(params.max_completion_tokens).toBe(1234);
});
it("uses conservative OpenAI-compatible fields for Cloudflare AI Gateway /compat models", async () => {
process.env.CLOUDFLARE_ACCOUNT_ID = "account-id";
process.env.CLOUDFLARE_GATEWAY_ID = "gateway-id";
@@ -104,7 +138,7 @@ describe("openai-completions empty tools handling", () => {
systemPrompt: "You are helpful.",
messages: [{ role: "user", content: "hi", timestamp: Date.now() }],
},
{ apiKey: "test", reasoning: "high" },
{ apiKey: "test", maxTokens: 1234, reasoning: "high" },
).result();
const params = mockState.lastParams as {
@@ -115,7 +149,7 @@ describe("openai-completions empty tools handling", () => {
store?: boolean;
};
expect(params.messages[0].role).toBe("system");
expect(params.max_tokens).toBeDefined();
expect(params.max_tokens).toBe(1234);
expect(params.max_completion_tokens).toBeUndefined();
expect(params.reasoning_effort).toBeUndefined();
expect(params.store).toBeUndefined();

View File

@@ -1,7 +1,7 @@
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { getModel } from "../src/models.js";
import { streamOpenAICompletions } from "../src/providers/openai-completions.js";
import type { Model } from "../src/types.js";
import { getModel } from "../src/models.ts";
import { streamOpenAICompletions } from "../src/providers/openai-completions.ts";
import type { Model } from "../src/types.ts";
interface FakeOpenAIClientOptions {
apiKey: string;
@@ -125,6 +125,13 @@ describe("openai-completions prompt caching", () => {
expect(payload?.prompt_cache_retention).toBe("24h");
});
it("clamps prompt_cache_key to OpenAI's 64-character limit", async () => {
const sessionId = "x".repeat(67);
const { payload } = await captureRequest({ sessionId });
expect(payload?.prompt_cache_key).toBe("x".repeat(64));
});
it("omits prompt cache fields when cacheRetention is none", async () => {
const { payload } = await captureRequest({ cacheRetention: "none", sessionId: "session-789" });

View File

@@ -1,6 +1,6 @@
import { beforeEach, describe, expect, it, vi } from "vitest";
import { complete } from "../src/stream.js";
import type { Model } from "../src/types.js";
import { complete } from "../src/stream.ts";
import type { Model } from "../src/types.ts";
// Router/virtual ids (e.g. OpenRouter `auto`) keep `model` pinned to the
// requested id and surface the routed concrete id on `responseModel`.

View File

@@ -2,7 +2,7 @@ import { once } from "node:events";
import http from "node:http";
import type { AddressInfo } from "node:net";
import { afterEach, describe, expect, it } from "vitest";
import { convertMessages, streamOpenAICompletions } from "../src/providers/openai-completions.js";
import { convertMessages, streamOpenAICompletions } from "../src/providers/openai-completions.ts";
import type {
AssistantMessage,
AssistantMessageEvent,
@@ -10,7 +10,7 @@ import type {
Model,
OpenAICompletionsCompat,
Usage,
} from "../src/types.js";
} from "../src/types.ts";
const emptyUsage: Usage = {
input: 0,

View File

@@ -1,8 +1,9 @@
import { Type } from "typebox";
import { beforeEach, describe, expect, it, vi } from "vitest";
import { getModel } from "../src/models.js";
import { streamSimple } from "../src/stream.js";
import type { Tool } from "../src/types.js";
import { getModel } from "../src/models.ts";
import { convertMessages } from "../src/providers/openai-completions.ts";
import { streamSimple } from "../src/stream.ts";
import type { AssistantMessage, Model, Tool, ToolResultMessage } from "../src/types.ts";
const mockState = vi.hoisted(() => ({
lastParams: undefined as unknown,
@@ -441,6 +442,38 @@ describe("openai-completions tool_choice", () => {
expect(response.content).toEqual([{ type: "text", text: "OK" }]);
});
it("errors when a stream ends after only null finish_reason chunks", async () => {
mockState.chunks = [
{
id: "chatcmpl-truncated",
choices: [{ delta: { content: "partial answer" }, finish_reason: null }],
},
{
id: "chatcmpl-truncated",
choices: [{ delta: { content: "partial answer" }, finish_reason: null }],
},
];
const { compat: _compat, ...baseModel } = getModel("openai", "gpt-4o-mini")!;
const model = { ...baseModel, api: "openai-completions" } as const;
const response = await streamSimple(
model,
{
messages: [
{
role: "user",
content: "Reply with a longer sentence",
timestamp: Date.now(),
},
],
},
{ apiKey: "test" },
).result();
expect(response.stopReason).toBe("error");
expect(response.errorMessage).toBe("Stream ended without finish_reason");
});
it("coalesces tool call deltas by stable index when provider mutates ids mid-stream", async () => {
mockState.chunks = [
{
@@ -784,6 +817,184 @@ describe("openai-completions tool_choice", () => {
expect(writeCall).not.toHaveProperty("partialArgs");
});
it("stores Xiaomi MiMo reasoning replay compat in built-in metadata", () => {
const providers = ["xiaomi", "xiaomi-token-plan-cn", "xiaomi-token-plan-ams", "xiaomi-token-plan-sgp"] as const;
for (const provider of providers) {
const model = getModel(provider, "mimo-v2.5-pro")!;
expect(model.compat?.requiresReasoningContentOnAssistantMessages).toBe(true);
expect(model.compat?.thinkingFormat).toBe("deepseek");
expect(model.compat?.maxTokensField).toBeUndefined();
expect(model.compat?.supportsDeveloperRole).toBeUndefined();
}
});
it("replays Xiaomi MiMo assistant tool calls with empty reasoning_content when thinking is missing", async () => {
const model = getModel("xiaomi", "mimo-v2.5-pro")!;
const assistantMessage: AssistantMessage = {
role: "assistant",
api: "openai-completions",
provider: "xiaomi",
model: "mimo-v2.5-pro",
content: [{ type: "toolCall", id: "call_1", name: "read", arguments: { path: "README.md" } }],
usage: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
totalTokens: 0,
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
},
stopReason: "toolUse",
timestamp: Date.now(),
};
const toolResult: ToolResultMessage = {
role: "toolResult",
toolCallId: "call_1",
toolName: "read",
content: [{ type: "text", text: "contents" }],
isError: false,
timestamp: Date.now(),
};
let payload: unknown;
await streamSimple(
model,
{
messages: [
{ role: "user", content: "Read README.md", timestamp: Date.now() },
assistantMessage,
toolResult,
],
},
{
apiKey: "test",
reasoning: "high",
onPayload: (params: unknown) => {
payload = params;
},
},
).result();
const params = (payload ?? mockState.lastParams) as {
messages?: Array<Record<string, unknown>>;
thinking?: { type?: string };
reasoning_effort?: string;
};
const replayedAssistant = params.messages?.find((message) => message.role === "assistant");
expect(replayedAssistant).toMatchObject({ role: "assistant", reasoning_content: "" });
expect(params.thinking).toEqual({ type: "enabled" });
expect(params.reasoning_effort).toBe("high");
});
it("normalizes OpenCode Go reasoning deltas to reasoning_content for replay", async () => {
mockState.chunks = [
{
id: "chatcmpl-opencode-go-reasoning",
choices: [{ delta: { reasoning: "think" }, finish_reason: "stop" }],
},
];
const { compat: _compat, ...baseModel } = getModel("opencode-go", "kimi-k2.6")!;
const model = { ...baseModel, api: "openai-completions" } as const;
const response = await streamSimple(
model,
{
messages: [{ role: "user", content: "Use reasoning.", timestamp: Date.now() }],
},
{ apiKey: "test" },
).result();
expect(response.content).toEqual([
{
type: "thinking",
thinking: "think",
thinkingSignature: "reasoning_content",
},
]);
});
it("keeps non-OpenCode Go reasoning deltas on the original reasoning field", async () => {
mockState.chunks = [
{
id: "chatcmpl-reasoning",
choices: [{ delta: { reasoning: "think" }, finish_reason: "stop" }],
},
];
const { compat: _compat, ...baseModel } = getModel("openai", "gpt-4o-mini")!;
const model = { ...baseModel, api: "openai-completions" } as const;
const response = await streamSimple(
model,
{
messages: [{ role: "user", content: "Use reasoning.", timestamp: Date.now() }],
},
{ apiKey: "test" },
).result();
expect(response.content).toEqual([
{
type: "thinking",
thinking: "think",
thinkingSignature: "reasoning",
},
]);
});
it("replays OpenCode Go reasoning thinking blocks as reasoning_content", () => {
const { compat: _compat, ...baseModel } = getModel("opencode-go", "kimi-k2.6")!;
const model = { ...baseModel, api: "openai-completions" } as Model<"openai-completions">;
const messages = convertMessages(
model,
{
messages: [
{
role: "assistant",
api: "openai-completions",
provider: "opencode-go",
model: "kimi-k2.6",
content: [
{ type: "thinking", thinking: "think", thinkingSignature: "reasoning" },
{ type: "toolCall", id: "call_1", name: "read", arguments: { path: "README.md" } },
],
usage: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
totalTokens: 0,
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
},
stopReason: "stop",
timestamp: Date.now(),
},
],
},
{
...model.compat,
supportsStore: false,
supportsDeveloperRole: false,
supportsReasoningEffort: true,
supportsUsageInStreaming: true,
maxTokensField: "max_completion_tokens",
requiresToolResultName: false,
requiresAssistantAfterToolResult: false,
requiresThinkingAsText: false,
requiresReasoningContentOnAssistantMessages: false,
thinkingFormat: "openai",
openRouterRouting: {},
vercelGatewayRouting: {},
zaiToolStream: false,
supportsStrictMode: true,
sendSessionAffinityHeaders: false,
supportsLongCacheRetention: true,
},
);
expect(messages[0]).toMatchObject({ role: "assistant", reasoning_content: "think" });
expect(messages[0]).not.toHaveProperty("reasoning");
});
it("does not double-count reasoning tokens in completion usage", async () => {
mockState.chunks = [
{
@@ -819,7 +1030,7 @@ describe("openai-completions tool_choice", () => {
expect(response.usage.totalTokens).toBe(43);
});
it("preserves prompt_tokens_details.cache_write_tokens from chunk usage", async () => {
it("preserves prompt_tokens_details cache read/write fields from chunk usage", async () => {
mockState.chunks = [
{
id: "chatcmpl-cache-write",
@@ -853,13 +1064,14 @@ describe("openai-completions tool_choice", () => {
{ apiKey: "test" },
).result();
expect(response.usage.input).toBe(50);
expect(response.usage.cacheRead).toBe(20);
// cached_tokens is documented as cache reads; cache_write_tokens is separate.
expect(response.usage.input).toBe(20);
expect(response.usage.cacheRead).toBe(50);
expect(response.usage.cacheWrite).toBe(30);
expect(response.usage.totalTokens).toBe(105);
});
it("preserves prompt_tokens_details.cache_write_tokens from choice usage fallback", async () => {
it("preserves prompt_tokens_details cache read/write fields from choice usage fallback", async () => {
mockState.chunks = [
{
id: "chatcmpl-cache-write-choice",
@@ -898,8 +1110,9 @@ describe("openai-completions tool_choice", () => {
{ apiKey: "test" },
).result();
expect(response.usage.input).toBe(50);
expect(response.usage.cacheRead).toBe(20);
// cached_tokens is documented as cache reads; cache_write_tokens is separate.
expect(response.usage.input).toBe(20);
expect(response.usage.cacheRead).toBe(50);
expect(response.usage.cacheWrite).toBe(30);
expect(response.usage.totalTokens).toBe(105);
});

View File

@@ -1,6 +1,6 @@
import { describe, expect, it } from "vitest";
import { getModel } from "../src/models.js";
import { convertMessages } from "../src/providers/openai-completions.js";
import { getModel } from "../src/models.ts";
import { convertMessages } from "../src/providers/openai-completions.ts";
import type {
AssistantMessage,
Context,
@@ -8,7 +8,7 @@ import type {
OpenAICompletionsCompat,
ToolResultMessage,
Usage,
} from "../src/types.js";
} from "../src/types.ts";
const emptyUsage: Usage = {
input: 0,

View File

@@ -1,7 +1,7 @@
import { describe, expect, it } from "vitest";
import { getModel } from "../src/models.js";
import { complete } from "../src/stream.js";
import type { Context } from "../src/types.js";
import { getModel } from "../src/models.ts";
import { complete } from "../src/stream.ts";
import type { Context } from "../src/types.ts";
describe.skipIf(!process.env.OPENAI_API_KEY)("openai responses cache affinity e2e", () => {
it("handles direct OpenAI Responses requests with aligned cache-affinity identifiers", { retry: 2 }, async () => {

View File

@@ -1,7 +1,7 @@
import { afterEach, describe, expect, it, vi } from "vitest";
import { getModel } from "../src/models.js";
import { streamOpenAIResponses } from "../src/providers/openai-responses.js";
import type { Model } from "../src/types.js";
import { getModel } from "../src/models.ts";
import { streamOpenAIResponses } from "../src/providers/openai-responses.ts";
import type { Model } from "../src/types.ts";
type CapturedHeaders = Headers | string[][] | Record<string, string | readonly string[]> | undefined;
@@ -171,6 +171,38 @@ describe("openai-responses provider defaults", () => {
expect(captured).toEqual({ sessionId: "session-123", clientRequestId: "session-123" });
});
it("clamps prompt_cache_key to OpenAI's 64-character limit", async () => {
const sessionId = "x".repeat(67);
let capturedPayload: { prompt_cache_key?: string } | undefined;
vi.spyOn(globalThis, "fetch").mockResolvedValue(
new Response("data: [DONE]\n\n", {
status: 200,
headers: { "content-type": "text/event-stream" },
}),
);
const stream = streamOpenAIResponses(
getModel("openai", "gpt-5.4"),
{
systemPrompt: "sys",
messages: [{ role: "user", content: "hi", timestamp: Date.now() }],
},
{
apiKey: "test-key",
sessionId,
onPayload: (payload) => {
capturedPayload = payload as { prompt_cache_key?: string };
},
},
);
for await (const event of stream) {
if (event.type === "done" || event.type === "error") break;
}
expect(capturedPayload?.prompt_cache_key).toBe("x".repeat(64));
});
it("sets cache-affinity headers for proxy OpenAI Responses requests with a sessionId", async () => {
const proxyModel: Model<"openai-responses"> = {
...getModel("openai", "gpt-5.4"),

View File

@@ -1,8 +1,8 @@
import { describe, expect, it } from "vitest";
import { getModel } from "../src/models.js";
import { convertResponsesMessages } from "../src/providers/openai-responses-shared.js";
import type { AssistantMessage, Context, ToolResultMessage, Usage } from "../src/types.js";
import { shortHash } from "../src/utils/hash.js";
import { getModel } from "../src/models.ts";
import { convertResponsesMessages } from "../src/providers/openai-responses-shared.ts";
import type { AssistantMessage, Context, ToolResultMessage, Usage } from "../src/types.ts";
import { shortHash } from "../src/utils/hash.ts";
const COPILOT_RAW_TOOL_CALL_ID =
"call_4VnzVawQXPB9MgYib7CiQFEY|I9b95oN1wD/cHXKTw3PpRkL6KkCtzTJhUxMouMWYwHeTo2j3htzfSk7YPx2vifiIM4g3A8XXyOj8q4Bt6SLUG7gqY1E3ELkrkVQNHglRfUmWj84lqxJY+Puieb3VKyX0FB+83TUzn91cDMF/4gzt990IzqVrc+nIb9RRscRD070Du16q1glydVjWR0SBJsE6TbY/esOjFpqplogQqrajm1eI++f3eLi73R6q7hVusY0QbeFySVxABCjhN0lXB04caBe1rzHjYzul6MAXj7uq+0r17VLq+yrtyYhN12wkmFqHeqTyEei6EFPbMy24Nc+IbJlkP0OCg02W+gOnyBFcbi2ctvJFSOhSjt1CqBdqCnnhwUqXjbWiT0wh3DmLScRgTHmGkaI+oAcQQjfic65nxj+TnEkReA==";
@@ -18,7 +18,7 @@ const usage: Usage = {
describe("OpenAI Responses foreign tool call ID normalization", () => {
it("hashes foreign Copilot tool item IDs into a bounded Codex-safe fc_<hash> shape", () => {
const model = getModel("openai-codex", "gpt-5.3-codex");
const model = getModel("openai-codex", "gpt-5.5");
const assistant: AssistantMessage = {
role: "assistant",
content: [
@@ -31,7 +31,7 @@ describe("OpenAI Responses foreign tool call ID normalization", () => {
],
api: "openai-responses",
provider: "github-copilot",
model: "gpt-5.3-codex",
model: "gpt-5.5",
usage,
stopReason: "toolUse",
timestamp: Date.now() - 2000,

Some files were not shown because too many files have changed in this diff Show More