feat(agent): add harness stream configuration
This commit is contained in:
Mario Zechner
@@ -31,6 +31,7 @@ Harness config is the latest runtime configuration set by the application or ext
|
||||
- tools
|
||||
- active tool names
|
||||
- resources
|
||||
- stream options
|
||||
- system prompt or system prompt provider
|
||||
|
||||
Getters return harness config. They do not return the snapshot used by an in-flight provider request.
|
||||
@@ -52,11 +53,15 @@ A turn snapshot is the concrete state used for one LLM turn. It is created by `c
|
||||
- thinking level
|
||||
- all tools
|
||||
- active tools
|
||||
- stream options
|
||||
- derived session id
|
||||
|
||||
Static option values are used directly. System-prompt provider callbacks are invoked once per `createTurnState()` call. All logic for that turn uses the same snapshot.
|
||||
|
||||
Resource arrays are shallow-copied when a snapshot is created. Individual skill and prompt-template objects are not deep-copied.
|
||||
|
||||
Stream options are shallow-copied when a snapshot is created. `headers` and `metadata` maps are shallow-copied; their values are not deep-copied. Credentials from `getApiKeyAndHeaders()` are resolved per provider request so expiring tokens can refresh, but the configured stream options and derived session id come from the current turn snapshot.
|
||||
|
||||
### Session
|
||||
|
||||
The session contains persisted entries only. Session reads return persisted state and do not include queued writes.
|
||||
@@ -125,9 +130,9 @@ At a save point the harness:
|
||||
|
||||
1. flushes pending session writes after the agent-emitted messages for that turn
|
||||
2. creates a fresh turn snapshot if the low-level loop may continue
|
||||
3. applies the fresh context/model/thinking-level state before the next provider request
|
||||
3. applies the fresh context/model/thinking-level/stream-options/session-id state before the next provider request
|
||||
|
||||
This lets model, thinking level, tool, resource, and system prompt changes made during a turn affect the next turn in the same run, while never mutating an in-flight provider request. The loop callbacks are not recreated at save points.
|
||||
This lets model, thinking level, tool, resource, stream option, and system prompt changes made during a turn affect the next turn in the same run, while never mutating an in-flight provider request. The loop callbacks are not recreated at save points.
|
||||
|
||||
The low-level loop converts harness `ThinkingLevel` to provider `reasoning` at the provider boundary:
|
||||
|
||||
@@ -188,12 +193,154 @@ Branch summary generation is part of the tree navigation operation.
|
||||
|
||||
Auto-compaction and retry decision points are not implemented in `AgentHarness` yet.
|
||||
|
||||
## Final lifecycle hardening todo
|
||||
## Implementation todo
|
||||
|
||||
Before treating `AgentHarness` as migration-ready, add a broad test suite that exercises listeners and hooks closing over the harness and calling public APIs during every relevant event:
|
||||
This list tracks the remaining work before treating `AgentHarness` as migration-ready.
|
||||
|
||||
### 1. Finish curated provider/stream configuration
|
||||
|
||||
Implemented so far:
|
||||
|
||||
- `AgentHarnessOptions.streamOptions` provides curated request configuration.
|
||||
- `getStreamOptions()` returns a shallow copy of current harness config.
|
||||
- `setStreamOptions()` replaces current harness config.
|
||||
- Stream options are snapshotted in `createTurnState()` and applied with `applyTurnState()`.
|
||||
- `headers` and `metadata` maps are shallow-copied when stream options are copied.
|
||||
- `sessionId` is derived from `session.getMetadata().id` in the turn snapshot.
|
||||
- The harness installs its own internal stream wrapper and calls `streamSimple()`.
|
||||
- The wrapper ignores raw incoming provider options except lifecycle-owned fields that must come from the low-level loop: `signal` and `reasoning`.
|
||||
- Credentials and auth headers from `getApiKeyAndHeaders()` are resolved per provider request.
|
||||
|
||||
Implemented provider hook behavior:
|
||||
|
||||
- `before_provider_request` runs before `streamSimple()` and can patch curated stream options for the current request only.
|
||||
- `before_provider_payload` maps to the underlying `pi-ai` `onPayload` and can inspect/replace provider-specific payloads.
|
||||
- `after_provider_response` maps to the underlying `pi-ai` `onResponse` and observes response status/headers before body consumption.
|
||||
- `AgentHarnessStreamOptionsPatch` has explicit deletion semantics:
|
||||
- top-level fields present with `undefined` clear that option.
|
||||
- `headers` and `metadata` patches may set individual keys to `undefined` to delete them.
|
||||
- `headers: undefined` or `metadata: undefined`, when explicitly present, clears the whole map.
|
||||
- Current-request stream option merge order is:
|
||||
1. snapshotted `streamOptions`
|
||||
2. auth headers from `getApiKeyAndHeaders()`
|
||||
3. `before_provider_request` patches, in hook registration order
|
||||
- `before_provider_request` does not patch `reasoning`; add that only if a concrete use case appears.
|
||||
|
||||
Still needed:
|
||||
|
||||
- Add tests proving stream options are snapshotted per turn and changes while busy affect only future provider requests/save-point snapshots.
|
||||
- Add tests proving provider hook patch/deletion/chaining semantics.
|
||||
|
||||
### 2. Design per-`AgentHarness` model registry
|
||||
|
||||
Not started.
|
||||
|
||||
Still needed:
|
||||
|
||||
- Decide how applications supply the model registry.
|
||||
- Decide whether the harness stores concrete `Model` objects, model references, or both.
|
||||
- Validate model selection against the registry.
|
||||
- Define model change semantics during active turns and save points.
|
||||
- Preserve current `setModel()` behavior until the registry model is designed.
|
||||
|
||||
### 3. Design generic hook/event extension mechanism
|
||||
|
||||
Current cleanup already done:
|
||||
|
||||
- Removed `AgentHarnessContext`.
|
||||
- Hooks receive only event payloads.
|
||||
- `emitHook(event)` derives the hook type from `event.type`.
|
||||
|
||||
Still needed:
|
||||
|
||||
- Define extension context shape.
|
||||
- Likely expose a harness facade plus a session facade rather than raw internals.
|
||||
- Decide which public harness APIs are allowed from each hook/event.
|
||||
- Decide whether hooks can mutate turn snapshots directly or only through explicit hook results/public APIs.
|
||||
- Clarify event payload semantics versus harness getter semantics.
|
||||
- Revisit `AgentHarnessOwnEvent` versus `AgentHarnessEvent`.
|
||||
- Define hook result chaining where it has clean transform semantics:
|
||||
- `before_provider_request`: each hook receives the stream options produced by previous hooks.
|
||||
- `before_provider_payload`: each hook receives the payload produced by previous hooks.
|
||||
- possibly `context`: each hook receives the messages produced by previous hooks.
|
||||
- possibly `tool_result`: each hook receives the result fields produced by previous hooks.
|
||||
- Do not chain hooks where semantics are policy-based or ambiguous until explicitly designed, such as `tool_call`, `session_before_compact`, `session_before_tree`, and `before_agent_start`.
|
||||
|
||||
### 4. Add explicit tool registry read/update semantics
|
||||
|
||||
Implemented so far:
|
||||
|
||||
- `setTools(tools, activeToolNames?)`
|
||||
- `setActiveTools(toolNames)`
|
||||
- invalid active tool names throw
|
||||
- generic common app tool shape via `AgentHarness<TSkill, TPromptTemplate, TTool>`
|
||||
- `QueueMode` exported from `Agent`
|
||||
- `AgentHarnessOptions.steeringMode` / `followUpMode`
|
||||
- live `steeringMode` / `followUpMode` getters/setters
|
||||
- queue modes are immediate/live, matching coding-agent behavior
|
||||
|
||||
Still needed:
|
||||
|
||||
- Add `getTools()` semantics.
|
||||
- Add `getActiveTools()` semantics.
|
||||
- Decide and implement tool update observability events.
|
||||
- Include active-tool-only updates in the uniform runtime config observability plan.
|
||||
|
||||
### 5. Full `AgentHarness` lifecycle/state pass
|
||||
|
||||
Implemented so far:
|
||||
|
||||
- Removed constructor `void syncFromTree()`.
|
||||
- Removed `syncFromTree()`.
|
||||
- Added `createTurnState()`, `applyTurnState()`, and `executeTurn()`.
|
||||
- Low-level `AgentLoopConfig.prepareNextTurn` save-point update exists.
|
||||
- `prepareNextTurn` updates low-level context/model/thinking-level and harness-applied stream/session snapshot state.
|
||||
- The loop converts `ThinkingLevel` to provider `reasoning` internally.
|
||||
- `phase` replaces boolean idle.
|
||||
- Pending session writes are based on session-entry shapes without generated fields.
|
||||
- Pending session writes flush at save points, settlement, and failure cleanup.
|
||||
- `steer`, `followUp`, and `nextTurn` accept text plus optional images and create `UserMessage` internally.
|
||||
- `nextTurn` ordering is fixed: queued messages before the new user message.
|
||||
- Removed `liveOperationId`.
|
||||
- Removed `shell()`; use `harness.env`.
|
||||
|
||||
Still needed:
|
||||
|
||||
- Finalize phase/idle semantics.
|
||||
- Audit whether `settled` can fire too early.
|
||||
- Make session writes inside `settled` callbacks deterministic.
|
||||
- Audit follow-up behavior around `agent_end`.
|
||||
- Implement auto-compaction decision point.
|
||||
- Implement retry handling.
|
||||
- Ensure structural operations use consistent `try/finally` phase cleanup.
|
||||
- Verify `before_agent_start` hook semantics against coding-agent:
|
||||
- current behavior prepends returned messages.
|
||||
- decide whether replacement, prepend, append, or transform semantics are correct.
|
||||
- Decide if `before_agent_start` needs more turn info such as tools/tool snippets.
|
||||
- Document or change timing for model/thinking/stream-option events that may fire before queued session entries persist while busy.
|
||||
- Audit `abort()` barrier semantics.
|
||||
|
||||
### 6. Later coding-agent migration plan
|
||||
|
||||
Not started.
|
||||
|
||||
Still needed:
|
||||
|
||||
- Map coding-agent resources to sourced loaders.
|
||||
- Keep app-level resource dedupe/provenance outside the harness.
|
||||
- Adapt extension loader to the future hook/session facade.
|
||||
- Preserve UI/session behavior outside core.
|
||||
- Move coding-agent stream/auth/retry/header behavior onto the harness stream configuration and provider hooks.
|
||||
|
||||
### 7. Final lifecycle hardening suite
|
||||
|
||||
Before treating `AgentHarness` as migration-ready, add a broad test suite that exercises listeners and hooks closing over the harness and calling public APIs during every relevant event.
|
||||
|
||||
Needs broad tests for:
|
||||
|
||||
- runtime config setters from low-level lifecycle events and harness events
|
||||
- resource/tool/model/thinking updates during active turns and save points
|
||||
- uniform runtime config observability events for model, thinking, resources, tools, active tools, and stream options
|
||||
- resource/tool/model/thinking/stream-option updates during active turns and save points
|
||||
- session writes from listeners and hooks, including writes from `settled`
|
||||
- queue operations from turn events, tool events, and provider hooks
|
||||
- rejected structural operations while busy
|
||||
|
||||
Reference in New Issue
Block a user