diff --git a/docs/roadmap/execution-controls.md b/docs/roadmap/execution-controls.md new file mode 100644 index 0000000..c527713 --- /dev/null +++ b/docs/roadmap/execution-controls.md @@ -0,0 +1,257 @@ +# Execution Controls For Coordinated Turns + +Status: proposed follow-up to v0.22.0. This document describes the target behavior +and implementation work; the significant choices at the end remain open. No code +changes are authorized by this roadmap. Public names and signatures are not yet +selected. + +## Purpose + +Enable applications using `Engine.Coordinate` to select a fresh timeout for each +continuation and retain the exact prepared request before generation. These are +execution capabilities useful for bounded operations and optional diagnostics, +without assumptions about prompt content or application workflows. + +[Audita's request](audita-request.md) supplies the motivating requirements. +Audita already uses one child context across explicit preparation and execution, +and retains prepared details for diagnostics. The goal is to make those controls +available during coordinator-owned execution without requiring a downstream +replacement for PromptKit's turn loop. + +PromptKit owns history assembly, execution, cancellation propagation, and +observations. Applications select budgets, interpret results, run validators, +schedule branches, and own diagnostic destinations, disclosure policy, and write +failure handling. This work adds no artifact store or reporting framework. + +## Current Baseline + +[PromptKit v0.22.0](../releases/v0.22.0.md) supplies captured exchanges, +continuation, sequential coordination, and explicit execution groups. + +- The [coordinator](../../coordinate.go) attaches the current exchange and composes + request/coordinator observers, then calls `Engine.Run` with the parent context. + Its callback returns a request, not an execution context or prepared handle. +- Ordinary [run execution](../../internal/usecase/runner.go) resolves the target, + obtains logical admission, completes preparation, and generates and validates + the response. Admission is released before the next-turn callback. +- Explicit [prepared execution](../../prepared_execution.go) already supplies + detached `Details()` and executes the retained snapshot. Preparation and + execution are separate calls; preparation does not reserve backend capacity. +- Ordinary [observations](../internal/observation.md) are metadata-only. Ordinary + runs observe preparation failures; split preparation itself emits no execution + events. A claimed prepared handle starts its observation lifecycle at execution. + +A timeout around the entire coordinator includes all turns and application +callbacks. Provider timeouts cover a different scope. Neither provides a fresh +whole-operation budget for each selected turn. Separately calling `Prepare` +would duplicate preparation and would not bind the inspected details to the +request actually executed. An exchange cannot supply a failed turn's prepared +request when execution returns no result. + +## Target End State + +### Optional Operation Timeout + +An application can select a duration for each requested continuation. When that +continuation starts, establish a fresh child deadline before preparation and use +it for the entire operation: preparation, admission waiting, retries, generation, +structural validation, and structural repair. Do not restart the budget between +phases or attempts. + +Respect parent cancellation and earlier parent deadlines. A local timeout ends +the affected operation and coordination under the existing operational-error +rule; it does not cancel the parent or siblings. Release child-context resources +before returning to the application's next-turn callback. Assessment calls from +that callback retain their own application-selected contexts and budgets. + +A completed operation does not become canceled merely because its child context +is cleaned up after execution. Preserve the existing coordinator behavior for +completed responses, parent cancellation before callback delivery, and the single +undelivered response. No automatic replay or new recovery policy is added. + +No selected timeout means existing context behavior. This adds no framework-wide +recommended duration or mandatory cap. Additional-turn, transport-retry, +structural-repair, and provider-generation budgets keep their existing meanings. +Cancellation remains cooperative; a deadline does not forcibly terminate +application callbacks or injected collaborators that ignore their context. + +### Opt-In Prepared Details Before Generation + +Provide explicitly selected access to caller-owned details of the logical request +that will execute, after successful preparation and before its first generation +attempt. Reuse the existing prepared-details representation and credential +redaction where practical. Include assembled captured history, rendered follow-up +messages, caller-appended messages, and ordinary resolved execution metadata. + +Use the execution's prepared snapshot rather than rendering or reading inputs +again. Retaining or modifying delivered details must not mutate the execution. +The application can retain them after provider failure or cancellation even when +no ordinary result or exchange is returned. If preparation fails, no complete +snapshot is required. Cancellation before notification does not promise delivery; +its precise boundary is proposed in Open Decision 3. + +This access exposes logical prepared content, not raw HTTP envelopes or internal +structural-repair requests. Keep it separate from metadata-only observers and +execution-group summaries. It is not another next-turn callback: ordinary +execution may already hold backend admission at the notification point. + +### Compatibility And Composition + +Applications selecting neither capability retain existing behavior. Preserve +conversation assembly, capture, final-turn delivery, branching, and grouping +contracts. Request observers, coordinator observers, and explicitly selected +groups continue receiving ordinary execution facts without duplicate lifecycle +events or counting attempts twice. + +Avoid changing the coordinator to an explicit public prepare/run sequence merely +to obtain details: that would change ordinary admission and preparation-failure +observation behavior. Reuse the execution path and its existing ownership +boundaries. Exact public controls and split-preparation semantics remain open +below; do not silently ignore a newly supplied option on one entry point. + +## Work From The Baseline + +1. **Settle the public lifecycle.** Resolve the open decisions below and write + small ordinary-run, coordinated-turn, and split-preparation usage sketches. + Specify omitted/zero/negative timeout handling using ordinary duration + conventions, callback invocation timing, and retained callback cleanup. + Choose names and signatures without adding a broader lifecycle-hook framework. +2. **Add operation-budget handling.** Establish and clean up the child scope at + the selected execution boundary. Preserve earlier parent deadlines and public + error identity through facade mapping and terminal observation classification. + Ensure cleanup does not affect the coordinator parent or subsequent callbacks. +3. **Expose the prepared snapshot.** Deliver detached public details from the + existing prepared state before generation. Cover both ordinary execution and + the selected prepared-execution lifecycle without a second preparation pass. + Preserve credential handling and release resources on cancellation or panic. + Avoid snapshot-copy work when the capability is not selected. +4. **Integrate and verify coordination.** Have ordinary execution honor the + selected controls on the callback-returned request under the recommended API. + Preserve capture and observer composition; verify the next-turn callback runs + with the parent context after admission and child-budget cleanup. +5. **Document and demonstrate.** Update public GoDoc, the + [consumer guide](../consumers/pkg-promptkit.md), relevant internal lifecycle + documentation, and a focused offline example when implementation lands. + Show how applications retain diagnostics on failure and bound independent + assessment calls. Reconcile Audita's scenarios against the final API. + +This is an additive runtime feature; no collection-format or ConfigKit changes +are needed. Keep implementation proportional to the two requested controls. + +## Validation Targets + +Use offline public-boundary tests and existing execution collaborators, with +controlled cancellation and synchronization rather than fragile timing margins. +Cover the consumer request's [acceptance scenarios](audita-request.md#acceptance-scenarios) +and the lifecycle choices ultimately selected: + +- Successive turns receive fresh budgets: their combined duration may exceed one + operation budget without exhausting either individual operation. +- Preparation, admission waiting, retries, validation, and repair share the same + operation deadline. Local expiry stops coordination without canceling parent + or sibling work; earlier parent cancellation/deadlines remain effective. +- Next-turn assessment uses the parent context after release, including another + call to a backend with capacity one. Timeout identity survives public wrapping. +- Prepared details precede generation, contain the assembled continuation, and + remain available after later failure. Mutation of the delivered copy or source + changes after preparation do not change the executed snapshot. +- No duplicate preparation, provider call, notification per internal repair, or + observer lifecycle is introduced. Existing credential redaction remains intact. +- Preparation failure, cancellation around notification, and callback panic follow + the selected delivery rules and do not strand admission or retained resources. +- Split preparation follows its documented budget and callback lifecycle. Neither + discarded nor unsuccessfully claimed handles start execution notifications. +- Observers/groups retain ordinary facts on success and failure. No-option use, + completed-result delivery, and final-turn behavior remain compatible. + +## Open Decisions + +### 1. Narrow Execution Controls Or Caller-Delegated Execution? + +**Recommendation:** add optional per-request execution controls honored by ordinary +`Run` and thus by `Coordinate`, including a timeout and prepared-details callback. +This keeps history, capture, admission, errors, and observers in their existing +owners and makes the capability useful outside coordination. A caller can select +different controls on each returned request; an additional coordinator-wide +configuration layer is not needed for the demonstrated use case. + +**Viable alternative:** let the coordinator delegate a turn to a caller-provided +executor or prepared-execution adapter. This would reuse Audita's existing adapter +and accommodate other execution customizations. It also requires a larger contract +for exchange assembly, capture, observer composition, handle cleanup, and admission +release. Prefer it only if concrete API sketches show it is materially simpler +than the narrow controls, not merely more extensible. + +### 2. What Does A Timeout Mean For Split Preparation And Execution? + +**Recommendation:** bound each `Run` invocation from before preparation. For +`PrepareExecution`, retain the selected duration as an execution control and apply +it when `RunPrepared` starts; preparation itself continues using its supplied +context. Do not retain a live timer across an idle prepared handle. Explicitly +explain that this bounds the remaining execution, not preparation already done. +Callers needing one total budget across both calls can continue passing the same +caller-created deadline context, as Audita does today. + +This preserves the distinction between executing now and preparing for later, +while letting a prepared handle carry the same selected execution controls. It +requires clear documentation: the duration alone does not retroactively cover +split preparation. `Prepare` and `ValidateRequest` likewise do not become timed +executions merely because they accept request-shaped input. + +**Viable alternative:** start a budget during `PrepareExecution` and carry its +remaining deadline into `RunPrepared`. This directly bounds preparation and +execution together without caller context plumbing. It also counts idle time, +requires timer/context ownership and discard semantics, and changes what a +prepared handle retains. This is useful if a consumer needs a whole prepared +operation lifetime, but exceeds the current coordinated-turn requirement. + +### 3. What Is The Prepared-Details Callback Contract? + +**Recommendation:** a synchronous notification receiving the operation context +and a detached prepared value, with no error return. Invoke it once after the +snapshot is ready and before generation, only when execution proceeds to that +point. For prepared handles, retain the callback until execution and clear it +on claim/discard like the existing observer; inspection remains available earlier +through `Details()` without triggering the notification. + +Check context before notification and again before generation. Include callback +time in the operation budget and retain ordinary cooperative cancellation and +panic propagation, with resource cleanup on unwind. Do not add background delivery +or stronger terminal-event guarantees. The application can retain details and +perform optional writes under its own policy; it need not turn a diagnostic write +failure into an inference failure. + +Under the current ordinary run path this notification executes while logical +admission is held. Document that fact rather than promise the next-turn callback's +capacity-one nested-execution behavior. Do not prohibit arbitrary callback work; +applications must account for the held capacity and operation budget. Releasing +and reacquiring admission solely for notification is not necessary for this scope. + +**Viable alternative:** let the callback return an error to veto generation. +This supports mandatory audit persistence or an application pre-generation check, +with ordinary cleanup and error propagation. It also makes the callback an +execution gate and requires failure classification semantics absent from the +optional-diagnostics request. Select it if that broader control has a concrete +consumer need; optional write failures can still be handled locally either way. + +### 4. How Precisely Should Local Timeout Errors Be Identified? + +**Recommendation:** preserve standard context matching and provide a narrow, +inspectable operation-timeout identity when this newly selected budget causes +termination. Keep parent cancellation/deadlines and provider-local failures under +their existing identities. The marker must survive public wrapping and must not +be inferred from any arbitrary `DeadlineExceeded` returned by a collaborator. +Choose its Go representation during API design; no new error-report hierarchy +or terminal-event category is implied. + +This lets callers distinguish the requested local budget from an earlier parent +or provider deadline without retaining the internal child context. Define cause +selection around races using the child context's recorded cause; do not promise +perfect ordering of independently simultaneous cancellations. + +**Viable alternative:** return only existing context errors and let callers +inspect their parent context. This is smaller and often sufficient when the +parent remains active. However, `DeadlineExceeded` alone may also originate in +a provider timeout, and a parent may become canceled before the caller inspects +it. It supplies weaker attribution; choose it only if Audita confirms that this +meets its classification requirement.