refactor(agent): finalize harness resource config
This commit is contained in:
@@ -4,6 +4,20 @@
|
|||||||
|
|
||||||
This document describes the current direction and implemented behavior. Some extension/session-facade details are planned and called out explicitly.
|
This document describes the current direction and implemented behavior. Some extension/session-facade details are planned and called out explicitly.
|
||||||
|
|
||||||
|
## Ultimate lifecycle goal
|
||||||
|
|
||||||
|
Harness listeners and hooks should be able to close over the `AgentHarness` instance and call public harness APIs from any event where those APIs are documented as allowed. Those calls must not corrupt in-flight turn snapshots, reorder persisted transcript entries, lose pending writes, deadlock settlement, or leave the harness in the wrong phase.
|
||||||
|
|
||||||
|
The intended rule is:
|
||||||
|
|
||||||
|
- structural operations remain rejected while busy
|
||||||
|
- queue operations are accepted at documented turn-safe points
|
||||||
|
- runtime config setters update future snapshots without mutating the current provider request
|
||||||
|
- session writes made while busy are durably queued and flushed in deterministic order
|
||||||
|
- getters return latest harness config, not in-flight snapshots
|
||||||
|
|
||||||
|
A final lifecycle hardening pass should prove these guarantees with a broad listener/hook reentrancy test suite.
|
||||||
|
|
||||||
## State model
|
## State model
|
||||||
|
|
||||||
The harness separates state into four categories.
|
The harness separates state into four categories.
|
||||||
@@ -23,7 +37,7 @@ Getters return harness config. They do not return the snapshot used by an in-fli
|
|||||||
|
|
||||||
Setters update harness config immediately, including while a turn is in flight. Changes affect the next turn snapshot, not the currently running provider request.
|
Setters update harness config immediately, including while a turn is in flight. Changes affect the next turn snapshot, not the currently running provider request.
|
||||||
|
|
||||||
`setResources()` accepts concrete resources and emits `resources_update` with shallow-copied resources. Applications own loading/reloading resources from disk or other sources and should call `setResources()` with new values.
|
`setResources()` accepts concrete resources and emits `resources_update` on every call with shallow-copied current and previous resources. Applications own loading/reloading resources from disk or other sources and should call `setResources()` with new values.
|
||||||
|
|
||||||
`getResources()` returns shallow-copied current resources. It is a live config read, not the last turn snapshot.
|
`getResources()` returns shallow-copied current resources. It is a live config read, not the last turn snapshot.
|
||||||
|
|
||||||
@@ -173,3 +187,18 @@ They are allowed only while idle and are not queued. They operate on persisted s
|
|||||||
Branch summary generation is part of the tree navigation operation.
|
Branch summary generation is part of the tree navigation operation.
|
||||||
|
|
||||||
Auto-compaction and retry decision points are not implemented in `AgentHarness` yet.
|
Auto-compaction and retry decision points are not implemented in `AgentHarness` yet.
|
||||||
|
|
||||||
|
## Final lifecycle hardening todo
|
||||||
|
|
||||||
|
Before treating `AgentHarness` as migration-ready, add a broad test suite that exercises listeners and hooks closing over the harness and calling public APIs during every relevant event:
|
||||||
|
|
||||||
|
- runtime config setters from low-level lifecycle events and harness events
|
||||||
|
- resource/tool/model/thinking updates during active turns and save points
|
||||||
|
- session writes from listeners and hooks, including writes from `settled`
|
||||||
|
- queue operations from turn events, tool events, and provider hooks
|
||||||
|
- rejected structural operations while busy
|
||||||
|
- abort from listeners/hooks
|
||||||
|
- getter behavior during active operations
|
||||||
|
- deterministic ordering of agent-emitted messages and pending listener writes
|
||||||
|
- no deadlocks when async listeners call harness APIs and await them
|
||||||
|
- phase cleanup through success, provider error, hook error, abort, compaction, and tree navigation
|
||||||
|
|||||||
@@ -17,7 +17,9 @@ import type {
|
|||||||
ExecutionEnv,
|
ExecutionEnv,
|
||||||
NavigateTreeResult,
|
NavigateTreeResult,
|
||||||
PendingSessionWrite,
|
PendingSessionWrite,
|
||||||
|
PromptTemplate,
|
||||||
Session,
|
Session,
|
||||||
|
Skill,
|
||||||
} from "./types.js";
|
} from "./types.js";
|
||||||
|
|
||||||
function createUserMessage(text: string, images?: ImageContent[]): UserMessage {
|
function createUserMessage(text: string, images?: ImageContent[]): UserMessage {
|
||||||
@@ -26,7 +28,11 @@ function createUserMessage(text: string, images?: ImageContent[]): UserMessage {
|
|||||||
return { role: "user", content, timestamp: Date.now() };
|
return { role: "user", content, timestamp: Date.now() };
|
||||||
}
|
}
|
||||||
|
|
||||||
export class AgentHarness {
|
export class AgentHarness<
|
||||||
|
TSkill extends Skill = Skill,
|
||||||
|
TPromptTemplate extends PromptTemplate = PromptTemplate,
|
||||||
|
TTool extends AgentTool = AgentTool,
|
||||||
|
> {
|
||||||
readonly agent: Agent;
|
readonly agent: Agent;
|
||||||
readonly env: ExecutionEnv;
|
readonly env: ExecutionEnv;
|
||||||
private session: Session;
|
private session: Session;
|
||||||
@@ -38,14 +44,16 @@ export class AgentHarness {
|
|||||||
private steerQueue: UserMessage[] = [];
|
private steerQueue: UserMessage[] = [];
|
||||||
private followUpQueue: UserMessage[] = [];
|
private followUpQueue: UserMessage[] = [];
|
||||||
private pendingSessionWrites: PendingSessionWrite[] = [];
|
private pendingSessionWrites: PendingSessionWrite[] = [];
|
||||||
private resources: AgentHarnessResources;
|
private resources: AgentHarnessResources<TSkill, TPromptTemplate>;
|
||||||
private systemPrompt: AgentHarnessOptions["systemPrompt"];
|
private systemPrompt: AgentHarnessOptions<TSkill, TPromptTemplate, TTool>["systemPrompt"];
|
||||||
private getApiKeyAndHeaders?: AgentHarnessOptions["getApiKeyAndHeaders"];
|
private getApiKeyAndHeaders?: AgentHarnessOptions["getApiKeyAndHeaders"];
|
||||||
private tools = new Map<string, AgentTool>();
|
private tools = new Map<string, TTool>();
|
||||||
private listeners = new Set<(event: AgentHarnessEvent, signal?: AbortSignal) => Promise<void> | void>();
|
private listeners = new Set<
|
||||||
|
(event: AgentHarnessEvent<TSkill, TPromptTemplate>, signal?: AbortSignal) => Promise<void> | void
|
||||||
|
>();
|
||||||
private hooks = new Map<keyof AgentHarnessEventResultMap, Set<(event: any) => Promise<any> | any>>();
|
private hooks = new Map<keyof AgentHarnessEventResultMap, Set<(event: any) => Promise<any> | any>>();
|
||||||
|
|
||||||
constructor(options: AgentHarnessOptions) {
|
constructor(options: AgentHarnessOptions<TSkill, TPromptTemplate, TTool>) {
|
||||||
this.agent = new Agent({
|
this.agent = new Agent({
|
||||||
initialState: {
|
initialState: {
|
||||||
model: options.model,
|
model: options.model,
|
||||||
@@ -60,12 +68,12 @@ export class AgentHarness {
|
|||||||
this.resources = options.resources ?? {};
|
this.resources = options.resources ?? {};
|
||||||
this.systemPrompt = options.systemPrompt;
|
this.systemPrompt = options.systemPrompt;
|
||||||
this.getApiKeyAndHeaders = options.getApiKeyAndHeaders;
|
this.getApiKeyAndHeaders = options.getApiKeyAndHeaders;
|
||||||
for (const tool of this.agent.state.tools) {
|
for (const tool of options.tools ?? []) {
|
||||||
this.tools.set(tool.name, tool);
|
this.tools.set(tool.name, tool);
|
||||||
}
|
}
|
||||||
this.model = options.model;
|
this.model = options.model;
|
||||||
this.thinkingLevel = options.thinkingLevel ?? this.agent.state.thinkingLevel;
|
this.thinkingLevel = options.thinkingLevel ?? this.agent.state.thinkingLevel;
|
||||||
this.activeToolNames = options.activeToolNames ?? this.agent.state.tools.map((tool) => tool.name);
|
this.activeToolNames = options.activeToolNames ?? (options.tools ?? []).map((tool) => tool.name);
|
||||||
this.agent.state.model = this.model;
|
this.agent.state.model = this.model;
|
||||||
this.agent.state.thinkingLevel = this.thinkingLevel;
|
this.agent.state.thinkingLevel = this.thinkingLevel;
|
||||||
this.agent.getApiKey = async (provider) => {
|
this.agent.getApiKey = async (provider) => {
|
||||||
@@ -127,13 +135,13 @@ export class AgentHarness {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
private async emitOwn(event: AgentHarnessOwnEvent, signal?: AbortSignal): Promise<void> {
|
private async emitOwn(event: AgentHarnessOwnEvent<TSkill, TPromptTemplate>, signal?: AbortSignal): Promise<void> {
|
||||||
for (const listener of this.listeners) {
|
for (const listener of this.listeners) {
|
||||||
await listener(event, signal);
|
await listener(event, signal);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private async emitAny(event: AgentHarnessEvent, signal?: AbortSignal): Promise<void> {
|
private async emitAny(event: AgentHarnessEvent<TSkill, TPromptTemplate>, signal?: AbortSignal): Promise<void> {
|
||||||
for (const listener of this.listeners) {
|
for (const listener of this.listeners) {
|
||||||
await listener(event, signal);
|
await listener(event, signal);
|
||||||
}
|
}
|
||||||
@@ -163,13 +171,13 @@ export class AgentHarness {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
private async createTurnState(): Promise<AgentHarnessTurnState> {
|
private async createTurnState(): Promise<AgentHarnessTurnState<TSkill, TPromptTemplate, TTool>> {
|
||||||
const context = await this.session.buildContext();
|
const context = await this.session.buildContext();
|
||||||
const resources = this.getResources();
|
const resources = this.getResources();
|
||||||
const tools = [...this.tools.values()];
|
const tools = [...this.tools.values()];
|
||||||
const activeTools = this.activeToolNames
|
const activeTools = this.activeToolNames
|
||||||
.map((name) => this.tools.get(name))
|
.map((name) => this.tools.get(name))
|
||||||
.filter((tool): tool is AgentTool => tool !== undefined);
|
.filter((tool): tool is TTool => tool !== undefined);
|
||||||
let systemPrompt = "You are a helpful assistant.";
|
let systemPrompt = "You are a helpful assistant.";
|
||||||
if (typeof this.systemPrompt === "string") {
|
if (typeof this.systemPrompt === "string") {
|
||||||
systemPrompt = this.systemPrompt;
|
systemPrompt = this.systemPrompt;
|
||||||
@@ -194,7 +202,7 @@ export class AgentHarness {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
private applyTurnState(turnState: AgentHarnessTurnState): void {
|
private applyTurnState(turnState: AgentHarnessTurnState<TSkill, TPromptTemplate, TTool>): void {
|
||||||
this.agent.state.messages = turnState.messages;
|
this.agent.state.messages = turnState.messages;
|
||||||
this.agent.state.systemPrompt = turnState.systemPrompt;
|
this.agent.state.systemPrompt = turnState.systemPrompt;
|
||||||
this.agent.state.model = turnState.model;
|
this.agent.state.model = turnState.model;
|
||||||
@@ -263,7 +271,7 @@ export class AgentHarness {
|
|||||||
}
|
}
|
||||||
|
|
||||||
private async executeTurn(
|
private async executeTurn(
|
||||||
turnState: AgentHarnessTurnState,
|
turnState: AgentHarnessTurnState<TSkill, TPromptTemplate, TTool>,
|
||||||
text: string,
|
text: string,
|
||||||
options?: { images?: ImageContent[] },
|
options?: { images?: ImageContent[] },
|
||||||
): Promise<AssistantMessage> {
|
): Promise<AssistantMessage> {
|
||||||
@@ -578,22 +586,23 @@ export class AgentHarness {
|
|||||||
this.agent.followUpMode = mode;
|
this.agent.followUpMode = mode;
|
||||||
}
|
}
|
||||||
|
|
||||||
getResources(): AgentHarnessResources {
|
getResources(): AgentHarnessResources<TSkill, TPromptTemplate> {
|
||||||
return {
|
return {
|
||||||
skills: this.resources.skills?.slice(),
|
skills: this.resources.skills?.slice(),
|
||||||
promptTemplates: this.resources.promptTemplates?.slice(),
|
promptTemplates: this.resources.promptTemplates?.slice(),
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
async setResources(resources: AgentHarnessResources): Promise<void> {
|
async setResources(resources: AgentHarnessResources<TSkill, TPromptTemplate>): Promise<void> {
|
||||||
|
const previousResources = this.getResources();
|
||||||
this.resources = {
|
this.resources = {
|
||||||
skills: resources.skills?.slice(),
|
skills: resources.skills?.slice(),
|
||||||
promptTemplates: resources.promptTemplates?.slice(),
|
promptTemplates: resources.promptTemplates?.slice(),
|
||||||
};
|
};
|
||||||
await this.emitOwn({ type: "resources_update", resources: this.getResources() });
|
await this.emitOwn({ type: "resources_update", resources: this.getResources(), previousResources });
|
||||||
}
|
}
|
||||||
|
|
||||||
async setTools(tools: AgentTool[], activeToolNames?: string[]): Promise<void> {
|
async setTools(tools: TTool[], activeToolNames?: string[]): Promise<void> {
|
||||||
this.tools = new Map(tools.map((tool) => [tool.name, tool]));
|
this.tools = new Map(tools.map((tool) => [tool.name, tool]));
|
||||||
if (activeToolNames) {
|
if (activeToolNames) {
|
||||||
this.validateToolNames(activeToolNames);
|
this.validateToolNames(activeToolNames);
|
||||||
@@ -623,7 +632,9 @@ export class AgentHarness {
|
|||||||
await this.agent.waitForIdle();
|
await this.agent.waitForIdle();
|
||||||
}
|
}
|
||||||
|
|
||||||
subscribe(listener: (event: AgentHarnessEvent, signal?: AbortSignal) => Promise<void> | void): () => void {
|
subscribe(
|
||||||
|
listener: (event: AgentHarnessEvent<TSkill, TPromptTemplate>, signal?: AbortSignal) => Promise<void> | void,
|
||||||
|
): () => void {
|
||||||
this.listeners.add(listener);
|
this.listeners.add(listener);
|
||||||
return () => this.listeners.delete(listener);
|
return () => this.listeners.delete(listener);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,13 +0,0 @@
|
|||||||
import { AgentHarness } from "./agent-harness.js";
|
|
||||||
import { Session } from "./session/session.js";
|
|
||||||
import type { AgentHarnessOptions, SessionMetadata, SessionStorage } from "./types.js";
|
|
||||||
|
|
||||||
export function createSession<TMetadata extends SessionMetadata>(
|
|
||||||
storage: SessionStorage<TMetadata>,
|
|
||||||
): Session<TMetadata> {
|
|
||||||
return new Session(storage);
|
|
||||||
}
|
|
||||||
|
|
||||||
export function createAgentHarness(options: AgentHarnessOptions): AgentHarness {
|
|
||||||
return new AgentHarness(options);
|
|
||||||
}
|
|
||||||
@@ -52,19 +52,26 @@ export async function loadPromptTemplates(
|
|||||||
* Source values are preserved exactly and attached to every loaded prompt template and diagnostic. The agent package does
|
* Source values are preserved exactly and attached to every loaded prompt template and diagnostic. The agent package does
|
||||||
* not interpret source values; applications define their own provenance shape.
|
* not interpret source values; applications define their own provenance shape.
|
||||||
*/
|
*/
|
||||||
export async function loadSourcedPromptTemplates<TSource>(
|
export async function loadSourcedPromptTemplates<TSource, TPromptTemplate extends PromptTemplate = PromptTemplate>(
|
||||||
env: ExecutionEnv,
|
env: ExecutionEnv,
|
||||||
inputs: Array<{ path: string; source: TSource }>,
|
inputs: Array<{ path: string; source: TSource }>,
|
||||||
|
mapPromptTemplate?: (promptTemplate: PromptTemplate, source: TSource) => TPromptTemplate,
|
||||||
): Promise<{
|
): Promise<{
|
||||||
promptTemplates: Array<{ promptTemplate: PromptTemplate; source: TSource }>;
|
promptTemplates: Array<{ promptTemplate: TPromptTemplate; source: TSource }>;
|
||||||
diagnostics: Array<PromptTemplateDiagnostic & { source: TSource }>;
|
diagnostics: Array<PromptTemplateDiagnostic & { source: TSource }>;
|
||||||
}> {
|
}> {
|
||||||
const promptTemplates: Array<{ promptTemplate: PromptTemplate; source: TSource }> = [];
|
const promptTemplates: Array<{ promptTemplate: TPromptTemplate; source: TSource }> = [];
|
||||||
const diagnostics: Array<PromptTemplateDiagnostic & { source: TSource }> = [];
|
const diagnostics: Array<PromptTemplateDiagnostic & { source: TSource }> = [];
|
||||||
for (const input of inputs) {
|
for (const input of inputs) {
|
||||||
const result = await loadPromptTemplates(env, input.path);
|
const result = await loadPromptTemplates(env, input.path);
|
||||||
for (const promptTemplate of result.promptTemplates)
|
for (const promptTemplate of result.promptTemplates) {
|
||||||
promptTemplates.push({ promptTemplate, source: input.source });
|
promptTemplates.push({
|
||||||
|
promptTemplate: mapPromptTemplate
|
||||||
|
? mapPromptTemplate(promptTemplate, input.source)
|
||||||
|
: (promptTemplate as TPromptTemplate),
|
||||||
|
source: input.source,
|
||||||
|
});
|
||||||
|
}
|
||||||
for (const diagnostic of result.diagnostics) diagnostics.push({ ...diagnostic, source: input.source });
|
for (const diagnostic of result.diagnostics) diagnostics.push({ ...diagnostic, source: input.source });
|
||||||
}
|
}
|
||||||
return { promptTemplates, diagnostics };
|
return { promptTemplates, diagnostics };
|
||||||
|
|||||||
@@ -59,18 +59,21 @@ export async function loadSkills(
|
|||||||
* Source values are preserved exactly and attached to every loaded skill and diagnostic. The agent package does not
|
* Source values are preserved exactly and attached to every loaded skill and diagnostic. The agent package does not
|
||||||
* interpret source values; applications define their own provenance shape.
|
* interpret source values; applications define their own provenance shape.
|
||||||
*/
|
*/
|
||||||
export async function loadSourcedSkills<TSource>(
|
export async function loadSourcedSkills<TSource, TSkill extends Skill = Skill>(
|
||||||
env: ExecutionEnv,
|
env: ExecutionEnv,
|
||||||
inputs: Array<{ path: string; source: TSource }>,
|
inputs: Array<{ path: string; source: TSource }>,
|
||||||
|
mapSkill?: (skill: Skill, source: TSource) => TSkill,
|
||||||
): Promise<{
|
): Promise<{
|
||||||
skills: Array<{ skill: Skill; source: TSource }>;
|
skills: Array<{ skill: TSkill; source: TSource }>;
|
||||||
diagnostics: Array<SkillDiagnostic & { source: TSource }>;
|
diagnostics: Array<SkillDiagnostic & { source: TSource }>;
|
||||||
}> {
|
}> {
|
||||||
const skills: Array<{ skill: Skill; source: TSource }> = [];
|
const skills: Array<{ skill: TSkill; source: TSource }> = [];
|
||||||
const diagnostics: Array<SkillDiagnostic & { source: TSource }> = [];
|
const diagnostics: Array<SkillDiagnostic & { source: TSource }> = [];
|
||||||
for (const input of inputs) {
|
for (const input of inputs) {
|
||||||
const result = await loadSkills(env, input.path);
|
const result = await loadSkills(env, input.path);
|
||||||
for (const skill of result.skills) skills.push({ skill, source: input.source });
|
for (const skill of result.skills) {
|
||||||
|
skills.push({ skill: mapSkill ? mapSkill(skill, input.source) : (skill as TSkill), source: input.source });
|
||||||
|
}
|
||||||
for (const diagnostic of result.diagnostics) diagnostics.push({ ...diagnostic, source: input.source });
|
for (const diagnostic of result.diagnostics) diagnostics.push({ ...diagnostic, source: input.source });
|
||||||
}
|
}
|
||||||
return { skills, diagnostics };
|
return { skills, diagnostics };
|
||||||
|
|||||||
@@ -33,11 +33,14 @@ export interface PromptTemplate {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/** Resources made available to explicit invocation methods and system-prompt callbacks. */
|
/** Resources made available to explicit invocation methods and system-prompt callbacks. */
|
||||||
export interface AgentHarnessResources {
|
export interface AgentHarnessResources<
|
||||||
|
TSkill extends Skill = Skill,
|
||||||
|
TPromptTemplate extends PromptTemplate = PromptTemplate,
|
||||||
|
> {
|
||||||
/** Prompt templates available for explicit invocation. */
|
/** Prompt templates available for explicit invocation. */
|
||||||
promptTemplates?: PromptTemplate[];
|
promptTemplates?: TPromptTemplate[];
|
||||||
/** Skills available to the model and explicit skill invocation. */
|
/** Skills available to the model and explicit skill invocation. */
|
||||||
skills?: Skill[];
|
skills?: TSkill[];
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Kind of filesystem object as addressed by an {@link ExecutionEnv}. Symlinks are not followed automatically. */
|
/** Kind of filesystem object as addressed by an {@link ExecutionEnv}. Symlinks are not followed automatically. */
|
||||||
@@ -295,14 +298,18 @@ export type PendingSessionWrite = SessionTreeEntry extends infer TEntry
|
|||||||
: never
|
: never
|
||||||
: never;
|
: never;
|
||||||
|
|
||||||
export interface AgentHarnessTurnState {
|
export interface AgentHarnessTurnState<
|
||||||
|
TSkill extends Skill = Skill,
|
||||||
|
TPromptTemplate extends PromptTemplate = PromptTemplate,
|
||||||
|
TTool extends AgentTool = AgentTool,
|
||||||
|
> {
|
||||||
messages: AgentMessage[];
|
messages: AgentMessage[];
|
||||||
resources: AgentHarnessResources;
|
resources: AgentHarnessResources<TSkill, TPromptTemplate>;
|
||||||
systemPrompt: string;
|
systemPrompt: string;
|
||||||
model: Model<any>;
|
model: Model<any>;
|
||||||
thinkingLevel: ThinkingLevel;
|
thinkingLevel: ThinkingLevel;
|
||||||
tools: AgentTool[];
|
tools: TTool[];
|
||||||
activeTools: AgentTool[];
|
activeTools: TTool[];
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface QueueUpdateEvent {
|
export interface QueueUpdateEvent {
|
||||||
@@ -328,12 +335,15 @@ export interface SettledEvent {
|
|||||||
nextTurnCount: number;
|
nextTurnCount: number;
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface BeforeAgentStartEvent {
|
export interface BeforeAgentStartEvent<
|
||||||
|
TSkill extends Skill = Skill,
|
||||||
|
TPromptTemplate extends PromptTemplate = PromptTemplate,
|
||||||
|
> {
|
||||||
type: "before_agent_start";
|
type: "before_agent_start";
|
||||||
prompt: string;
|
prompt: string;
|
||||||
images?: ImageContent[];
|
images?: ImageContent[];
|
||||||
systemPrompt: string;
|
systemPrompt: string;
|
||||||
resources: AgentHarnessResources;
|
resources: AgentHarnessResources<TSkill, TPromptTemplate>;
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface ContextEvent {
|
export interface ContextEvent {
|
||||||
@@ -410,17 +420,24 @@ export interface ThinkingLevelSelectEvent {
|
|||||||
previousLevel: ThinkingLevel;
|
previousLevel: ThinkingLevel;
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface ResourcesUpdateEvent {
|
export interface ResourcesUpdateEvent<
|
||||||
|
TSkill extends Skill = Skill,
|
||||||
|
TPromptTemplate extends PromptTemplate = PromptTemplate,
|
||||||
|
> {
|
||||||
type: "resources_update";
|
type: "resources_update";
|
||||||
resources: AgentHarnessResources;
|
resources: AgentHarnessResources<TSkill, TPromptTemplate>;
|
||||||
|
previousResources: AgentHarnessResources<TSkill, TPromptTemplate>;
|
||||||
}
|
}
|
||||||
|
|
||||||
export type AgentHarnessOwnEvent =
|
export type AgentHarnessOwnEvent<
|
||||||
|
TSkill extends Skill = Skill,
|
||||||
|
TPromptTemplate extends PromptTemplate = PromptTemplate,
|
||||||
|
> =
|
||||||
| QueueUpdateEvent
|
| QueueUpdateEvent
|
||||||
| SavePointEvent
|
| SavePointEvent
|
||||||
| AbortEvent
|
| AbortEvent
|
||||||
| SettledEvent
|
| SettledEvent
|
||||||
| BeforeAgentStartEvent
|
| BeforeAgentStartEvent<TSkill, TPromptTemplate>
|
||||||
| ContextEvent
|
| ContextEvent
|
||||||
| BeforeProviderRequestEvent
|
| BeforeProviderRequestEvent
|
||||||
| AfterProviderResponseEvent
|
| AfterProviderResponseEvent
|
||||||
@@ -432,9 +449,11 @@ export type AgentHarnessOwnEvent =
|
|||||||
| SessionTreeEvent
|
| SessionTreeEvent
|
||||||
| ModelSelectEvent
|
| ModelSelectEvent
|
||||||
| ThinkingLevelSelectEvent
|
| ThinkingLevelSelectEvent
|
||||||
| ResourcesUpdateEvent;
|
| ResourcesUpdateEvent<TSkill, TPromptTemplate>;
|
||||||
|
|
||||||
export type AgentHarnessEvent = AgentEvent | AgentHarnessOwnEvent;
|
export type AgentHarnessEvent<TSkill extends Skill = Skill, TPromptTemplate extends PromptTemplate = PromptTemplate> =
|
||||||
|
| AgentEvent
|
||||||
|
| AgentHarnessOwnEvent<TSkill, TPromptTemplate>;
|
||||||
|
|
||||||
export interface BeforeAgentStartResult {
|
export interface BeforeAgentStartResult {
|
||||||
messages?: AgentMessage[];
|
messages?: AgentMessage[];
|
||||||
@@ -568,15 +587,19 @@ export interface BranchSummaryResult {
|
|||||||
error?: string;
|
error?: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface AgentHarnessOptions {
|
export interface AgentHarnessOptions<
|
||||||
|
TSkill extends Skill = Skill,
|
||||||
|
TPromptTemplate extends PromptTemplate = PromptTemplate,
|
||||||
|
TTool extends AgentTool = AgentTool,
|
||||||
|
> {
|
||||||
env: ExecutionEnv;
|
env: ExecutionEnv;
|
||||||
session: Session;
|
session: Session;
|
||||||
tools?: AgentTool[];
|
tools?: TTool[];
|
||||||
/**
|
/**
|
||||||
* Concrete resources available to explicit invocation methods and system-prompt callbacks.
|
* Concrete resources available to explicit invocation methods and system-prompt callbacks.
|
||||||
* Applications own loading/reloading resources and should call `setResources()` with new values.
|
* Applications own loading/reloading resources and should call `setResources()` with new values.
|
||||||
*/
|
*/
|
||||||
resources?: AgentHarnessResources;
|
resources?: AgentHarnessResources<TSkill, TPromptTemplate>;
|
||||||
systemPrompt?:
|
systemPrompt?:
|
||||||
| string
|
| string
|
||||||
| ((context: {
|
| ((context: {
|
||||||
@@ -584,8 +607,8 @@ export interface AgentHarnessOptions {
|
|||||||
session: Session;
|
session: Session;
|
||||||
model: Model<any>;
|
model: Model<any>;
|
||||||
thinkingLevel: ThinkingLevel;
|
thinkingLevel: ThinkingLevel;
|
||||||
activeTools: AgentTool[];
|
activeTools: TTool[];
|
||||||
resources: AgentHarnessResources;
|
resources: AgentHarnessResources<TSkill, TPromptTemplate>;
|
||||||
}) => string | Promise<string>);
|
}) => string | Promise<string>);
|
||||||
getApiKeyAndHeaders?: (
|
getApiKeyAndHeaders?: (
|
||||||
model: Model<any>,
|
model: Model<any>,
|
||||||
|
|||||||
@@ -23,7 +23,6 @@ export {
|
|||||||
shouldCompact,
|
shouldCompact,
|
||||||
} from "./harness/compaction/compaction.js";
|
} from "./harness/compaction/compaction.js";
|
||||||
export * from "./harness/execution-env.js";
|
export * from "./harness/execution-env.js";
|
||||||
export * from "./harness/factory.js";
|
|
||||||
export * from "./harness/messages.js";
|
export * from "./harness/messages.js";
|
||||||
export * from "./harness/prompt-templates.js";
|
export * from "./harness/prompt-templates.js";
|
||||||
export * from "./harness/session/repo/jsonl.js";
|
export * from "./harness/session/repo/jsonl.js";
|
||||||
|
|||||||
82
packages/agent/test/harness/agent-harness.test.ts
Normal file
82
packages/agent/test/harness/agent-harness.test.ts
Normal file
@@ -0,0 +1,82 @@
|
|||||||
|
import { getModel } from "@earendil-works/pi-ai";
|
||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
import { AgentHarness } from "../../src/harness/agent-harness.js";
|
||||||
|
import { NodeExecutionEnv } from "../../src/harness/execution-env.js";
|
||||||
|
import { Session } from "../../src/harness/session/session.js";
|
||||||
|
import { InMemorySessionStorage } from "../../src/harness/session/storage/memory.js";
|
||||||
|
import type { PromptTemplate, Skill } from "../../src/harness/types.js";
|
||||||
|
import type { AgentTool } from "../../src/types.js";
|
||||||
|
|
||||||
|
interface AppSkill extends Skill {
|
||||||
|
source: "project" | "user";
|
||||||
|
}
|
||||||
|
|
||||||
|
interface AppPromptTemplate extends PromptTemplate {
|
||||||
|
source: "project" | "user";
|
||||||
|
}
|
||||||
|
|
||||||
|
interface AppTool extends AgentTool {
|
||||||
|
source: "builtin" | "extension";
|
||||||
|
}
|
||||||
|
|
||||||
|
describe("AgentHarness", () => {
|
||||||
|
it("constructs directly and exposes queue modes", () => {
|
||||||
|
const session = new Session(new InMemorySessionStorage());
|
||||||
|
const env = new NodeExecutionEnv({ cwd: process.cwd() });
|
||||||
|
const initialModel = getModel("anthropic", "claude-sonnet-4-5");
|
||||||
|
const harness = new AgentHarness({
|
||||||
|
env,
|
||||||
|
session,
|
||||||
|
model: initialModel,
|
||||||
|
systemPrompt: "You are helpful.",
|
||||||
|
steeringMode: "all",
|
||||||
|
followUpMode: "all",
|
||||||
|
});
|
||||||
|
expect(harness.env).toBe(env);
|
||||||
|
expect(harness.agent.state.model).toBe(initialModel);
|
||||||
|
expect(harness.steeringMode).toBe("all");
|
||||||
|
expect(harness.followUpMode).toBe("all");
|
||||||
|
harness.steeringMode = "one-at-a-time";
|
||||||
|
harness.followUpMode = "one-at-a-time";
|
||||||
|
expect(harness.agent.steeringMode).toBe("one-at-a-time");
|
||||||
|
expect(harness.agent.followUpMode).toBe("one-at-a-time");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("preserves app resource types for getters and update events", async () => {
|
||||||
|
const session = new Session(new InMemorySessionStorage());
|
||||||
|
const env = new NodeExecutionEnv({ cwd: process.cwd() });
|
||||||
|
const model = getModel("anthropic", "claude-sonnet-4-5");
|
||||||
|
const harness = new AgentHarness<AppSkill, AppPromptTemplate, AppTool>({ env, session, model });
|
||||||
|
const skill: AppSkill = {
|
||||||
|
name: "inspect",
|
||||||
|
description: "Inspect things",
|
||||||
|
content: "Use inspection tools.",
|
||||||
|
filePath: "/skills/inspect/SKILL.md",
|
||||||
|
source: "project",
|
||||||
|
};
|
||||||
|
const promptTemplate: AppPromptTemplate = { name: "review", content: "Review $1", source: "user" };
|
||||||
|
const resources = { skills: [skill], promptTemplates: [promptTemplate] };
|
||||||
|
const updates: Array<{ resourcesSource?: string; previousSource?: string }> = [];
|
||||||
|
harness.subscribe((event) => {
|
||||||
|
if (event.type === "resources_update") {
|
||||||
|
updates.push({
|
||||||
|
resourcesSource: event.resources.skills?.[0]?.source,
|
||||||
|
previousSource: event.previousResources.skills?.[0]?.source,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
await harness.setResources(resources);
|
||||||
|
await harness.setResources(resources);
|
||||||
|
const resolved = harness.getResources();
|
||||||
|
|
||||||
|
expect(updates).toEqual([
|
||||||
|
{ resourcesSource: "project", previousSource: undefined },
|
||||||
|
{ resourcesSource: "project", previousSource: "project" },
|
||||||
|
]);
|
||||||
|
expect(resolved.skills?.[0]?.source).toBe("project");
|
||||||
|
expect(resolved.promptTemplates?.[0]?.source).toBe("user");
|
||||||
|
expect(resolved.skills).not.toBe(resources.skills);
|
||||||
|
expect(resolved.promptTemplates).not.toBe(resources.promptTemplates);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -1,64 +0,0 @@
|
|||||||
import { getModel } from "@earendil-works/pi-ai";
|
|
||||||
import { describe, expect, it } from "vitest";
|
|
||||||
import { NodeExecutionEnv } from "../../src/harness/execution-env.js";
|
|
||||||
import { createAgentHarness, createSession } from "../../src/harness/factory.js";
|
|
||||||
import { InMemorySessionStorage } from "../../src/harness/session/storage/memory.js";
|
|
||||||
|
|
||||||
describe("harness factories", () => {
|
|
||||||
it("creates sessions from storage", async () => {
|
|
||||||
const storage = new InMemorySessionStorage({
|
|
||||||
metadata: { id: "session-1", createdAt: "2026-01-01T00:00:00.000Z" },
|
|
||||||
});
|
|
||||||
const session = createSession(storage);
|
|
||||||
expect(session.getStorage()).toBe(storage);
|
|
||||||
expect(await session.getMetadata()).toEqual({ id: "session-1", createdAt: "2026-01-01T00:00:00.000Z" });
|
|
||||||
});
|
|
||||||
|
|
||||||
it("creates agent harnesses", () => {
|
|
||||||
const session = createSession(new InMemorySessionStorage());
|
|
||||||
const env = new NodeExecutionEnv({ cwd: process.cwd() });
|
|
||||||
const initialModel = getModel("anthropic", "claude-sonnet-4-5");
|
|
||||||
const harness = createAgentHarness({
|
|
||||||
env,
|
|
||||||
session,
|
|
||||||
model: initialModel,
|
|
||||||
systemPrompt: "You are helpful.",
|
|
||||||
steeringMode: "all",
|
|
||||||
followUpMode: "all",
|
|
||||||
});
|
|
||||||
expect(harness.env).toBe(env);
|
|
||||||
expect(harness.agent.state.model).toBe(initialModel);
|
|
||||||
expect(harness.steeringMode).toBe("all");
|
|
||||||
expect(harness.followUpMode).toBe("all");
|
|
||||||
harness.steeringMode = "one-at-a-time";
|
|
||||||
harness.followUpMode = "one-at-a-time";
|
|
||||||
expect(harness.agent.steeringMode).toBe("one-at-a-time");
|
|
||||||
expect(harness.agent.followUpMode).toBe("one-at-a-time");
|
|
||||||
});
|
|
||||||
|
|
||||||
it("updates and reads concrete resources", async () => {
|
|
||||||
const session = createSession(new InMemorySessionStorage());
|
|
||||||
const env = new NodeExecutionEnv({ cwd: process.cwd() });
|
|
||||||
const model = getModel("anthropic", "claude-sonnet-4-5");
|
|
||||||
const harness = createAgentHarness({ env, session, model });
|
|
||||||
const skill = {
|
|
||||||
name: "inspect",
|
|
||||||
description: "Inspect things",
|
|
||||||
content: "Use inspection tools.",
|
|
||||||
filePath: "/skills/inspect/SKILL.md",
|
|
||||||
};
|
|
||||||
const resources = { skills: [skill], promptTemplates: [{ name: "review", content: "Review $1" }] };
|
|
||||||
const updates: unknown[] = [];
|
|
||||||
harness.subscribe((event) => {
|
|
||||||
if (event.type === "resources_update") updates.push(event.resources);
|
|
||||||
});
|
|
||||||
|
|
||||||
await harness.setResources(resources);
|
|
||||||
const resolved = harness.getResources();
|
|
||||||
|
|
||||||
expect(updates).toEqual([resources]);
|
|
||||||
expect(resolved).toEqual(resources);
|
|
||||||
expect(resolved.skills).not.toBe(resources.skills);
|
|
||||||
expect(resolved.promptTemplates).not.toBe(resources.promptTemplates);
|
|
||||||
});
|
|
||||||
});
|
|
||||||
@@ -3,30 +3,39 @@ import { join } from "node:path";
|
|||||||
import { getModel } from "@earendil-works/pi-ai";
|
import { getModel } from "@earendil-works/pi-ai";
|
||||||
import { InMemorySessionStorage } from "../../src/harness/session/storage/memory.js";
|
import { InMemorySessionStorage } from "../../src/harness/session/storage/memory.js";
|
||||||
import {
|
import {
|
||||||
createAgentHarness,
|
AgentHarness,
|
||||||
formatSkillsForSystemPrompt,
|
formatSkillsForSystemPrompt,
|
||||||
loadSourcedPromptTemplates,
|
loadSourcedPromptTemplates,
|
||||||
loadSourcedSkills,
|
loadSourcedSkills,
|
||||||
NodeExecutionEnv,
|
NodeExecutionEnv,
|
||||||
|
type PromptTemplate,
|
||||||
Session,
|
Session,
|
||||||
|
type Skill,
|
||||||
} from "../../src/index.js";
|
} from "../../src/index.js";
|
||||||
|
|
||||||
type Source = { type: "project" | "user" | "path"; dir: string };
|
type Source = { type: "project" | "user" | "path"; dir: string };
|
||||||
|
type SourcedSkill = Skill & { source: Source };
|
||||||
|
type SourcedPromptTemplate = PromptTemplate & { source: Source };
|
||||||
|
|
||||||
const env = new NodeExecutionEnv({ cwd: process.cwd() });
|
const env = new NodeExecutionEnv({ cwd: process.cwd() });
|
||||||
const source = (type: Source["type"], dir: string) => ({ path: dir, source: { type, dir } });
|
const source = (type: Source["type"], dir: string) => ({ path: dir, source: { type, dir } });
|
||||||
const { skills: sourcedSkills } = await loadSourcedSkills<Source>(env, [
|
const { skills: sourcedSkills } = await loadSourcedSkills<Source, SourcedSkill>(
|
||||||
source("project", join(env.cwd, ".pi/skills")),
|
env,
|
||||||
source("user", join(homedir(), ".pi/agent/skills")),
|
[
|
||||||
source("path", join(env.cwd, "../../../pi-skills")),
|
source("project", join(env.cwd, ".pi/skills")),
|
||||||
]);
|
source("user", join(homedir(), ".pi/agent/skills")),
|
||||||
const { promptTemplates: sourcedPromptTemplates } = await loadSourcedPromptTemplates<Source>(env, [
|
source("path", join(env.cwd, "../../../pi-skills")),
|
||||||
source("project", join(env.cwd, ".pi/prompts")),
|
],
|
||||||
source("user", join(homedir(), ".pi/agent/prompts")),
|
(skill, source) => ({ ...skill, source }),
|
||||||
]);
|
);
|
||||||
|
const { promptTemplates: sourcedPromptTemplates } = await loadSourcedPromptTemplates<Source, SourcedPromptTemplate>(
|
||||||
|
env,
|
||||||
|
[source("project", join(env.cwd, ".pi/prompts")), source("user", join(homedir(), ".pi/agent/prompts"))],
|
||||||
|
(promptTemplate, source) => ({ ...promptTemplate, source }),
|
||||||
|
);
|
||||||
|
|
||||||
const session = new Session(new InMemorySessionStorage());
|
const session = new Session(new InMemorySessionStorage());
|
||||||
const agent = createAgentHarness({
|
const agent = new AgentHarness({
|
||||||
env,
|
env,
|
||||||
session,
|
session,
|
||||||
model: getModel("openai", "gpt-5.5"),
|
model: getModel("openai", "gpt-5.5"),
|
||||||
|
|||||||
Reference in New Issue
Block a user