From 0e4f590f5aaa1ddf670cc29c277d396d8d055409 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sat, 14 Nov 2026 17:15:46 +0000 Subject: [PATCH] Define the execution controls implementation plan --- docs/roadmap/audita-request.md | 139 +++++++++ docs/roadmap/execution-controls.md | 52 ++-- docs/roadmap/implementation.md | 441 +++++++++++++++++++++++++++++ 3 files changed, 596 insertions(+), 36 deletions(-) create mode 100644 docs/roadmap/audita-request.md create mode 100644 docs/roadmap/implementation.md diff --git a/docs/roadmap/audita-request.md b/docs/roadmap/audita-request.md new file mode 100644 index 0000000..fafdf0f --- /dev/null +++ b/docs/roadmap/audita-request.md @@ -0,0 +1,139 @@ +# Audita request: coordinator execution controls + +Status: proposed follow-up to PromptKit v0.22.0. This document describes +consumer requirements; public API design and implementation remain upstream +decisions. + +## Purpose + +PromptKit v0.22.0 supplies the conversation capture, continuation, and execution +groups Audita requested. Audita can adopt those capabilities through its existing +execution adapter. To also adopt `Engine.Coordinate` and remove its local turn +loop, Audita needs two capabilities during coordinator-owned execution: + +1. A fresh, application-selected timeout for each continuation operation. +2. Access to the exact prepared request before generation, including for turns + that subsequently fail without returning a result. + +Both capabilities apply to applications that run bounded model operations and +retain optional diagnostic evidence. Neither requires PromptKit to understand +transcripts, correction proposals, or domain validation. + +## Current workflow and integration gap + +Audita's role-bound adapter currently derives a timeout context from its caller, +calls `PrepareExecution`, retains `PreparedExecution.Details()`, and executes +that same handle with `RunPrepared`. The operation deadline covers preparation, +capacity waiting, transient retries, generation, and structural repair. Each +model operation receives its own budget; application validation may make +additional, separately bounded model calls. + +When debug evidence is enabled, Audita retains the prepared messages and +effective settings alongside the available response or failure information. +It owns artifact destinations, application-specific redaction, and handling of +optional diagnostic write failures. The same adapter supplies ordinary execution +observers and classifies operation errors for the application. + +In v0.22.0, the [coordinator](../../coordinate.go) accepts a callback returning a +`RunRequest`, then directly calls `Engine.Run` with the coordinator's context. +The callback cannot supply a fresh execution context or a prepared handle. +There is no callback exposing the actual prepared request before that run. + +A timeout around the entire coordinator bounds the conversation and application +callbacks together; it does not preserve a separate timeout for each operation. +Provider generation timeouts do not cover the whole operation either. +Separately calling `Prepare` before returning a request duplicates preparation +and does not bind the inspected snapshot to the execution. Successful exchange +inspection cannot supply a failed turn's prepared request when no new exchange +is returned. Existing metadata-only observations do not contain that content. + +## Requested behavior + +### Per-operation deadlines for coordinated turns + +Allow the application to select a timeout for each requested continuation, with +these semantics: + +- Establish a fresh deadline when that continuation starts, before its + preparation. Cover preparation, admission waiting, retries, generation, and + structural repair under the same operation budget. +- Respect caller cancellation and any earlier parent deadline. Expiry of a + local operation budget must not cancel the parent context or independent + branches. +- End the operation's timeout scope before invoking the application's next-turn + callback. Assessment calls made there can retain their own operation budgets. +- Preserve useful error identity so Audita can distinguish a local deadline + from cancellation of the caller. An execution timeout can end coordination + under the existing operational-error rule; no automatic turn replay or new + recovery policy is requested. + +The existing additional-turn budget, transport retry budget, structural-repair +budget, and provider generation timeout retain their separate meanings. + +### Exact prepared-request access + +Provide an opt-in way to obtain caller-owned details of the actual prepared +continuation before its first generation attempt: + +- Include the assembled captured history, follow-up definition messages, + appended messages, and ordinary resolved execution metadata, using the + existing [prepared-details contract](../../prepared_execution.go) where useful. +- Expose the snapshot that execution will use, without a second rendering or + input-reading pass. Access to details must not mutate that execution. +- Let the application retain those details even if generation subsequently + fails or is canceled and returns no ordinary result or captured exchange. + If preparation itself fails, a complete prepared snapshot is not required. +- Preserve existing credential-redaction semantics. Keep content-bearing access + explicitly selected and separate from ordinary metadata-only observations. + +Audita needs the logical prepared request, not raw HTTP envelopes or an archive +of intermediate structural-repair requests. It can handle optional debug writes +itself without requiring PromptKit to own artifact storage or reporting policy. + +## API direction and ownership + +Prefer small execution-level controls that the coordinator honors through its +ordinary execution path. An optional operation timeout and a prepared-details +callback are one possible shape, not proposed exported names or signatures. +Their applicability to ordinary and prepared execution should follow a coherent +lifecycle rather than introducing a second coordinator-specific executor. + +An alternative is allowing a coordinator turn to delegate execution through a +caller-controlled prepared-execution path. That could reuse Audita's adapter, +but exposes more execution responsibility and requires clear ownership of +capture, observers, admission release, and prepared-handle cleanup. Evaluate +that option only if it is simpler than the narrow controls above. + +PromptKit should continue owning history assembly, turn progression, execution, +and observations. Audita retains response interpretation, validator calls, +branch policy, diagnostic storage, and application error classification. Existing +request observers, coordinator observers, and execution groups must remain +usable together. This request does not reopen the accepted conversation history, +final-turn delivery, or grouping contracts. + +## Acceptance scenarios + +1. Two successful continuation operations each finish within their own timeout, + while their combined duration exceeds one operation's budget. Both succeed + under an active parent context. +2. Preparation or admission waiting consumes the same budget as generation and + structural repair. Local expiry returns a recognizable deadline error and + stops that coordinator without canceling its parent or a sibling branch. +3. An earlier parent deadline remains effective. A validator called from the + next-turn callback runs after admission is released, including with backend + capacity one, under its own application-selected deadline. +4. The application receives the prepared continuation details before generation. + A later provider failure returns no result, but the retained details still + describe that failed operation. Mutating a details copy does not change the + messages executed. +5. Diagnostic access uses the same prepared snapshot that executes, including + when source inputs change afterward. It does not trigger another preparation + or provider call. +6. Existing observers and selected execution groups continue receiving ordinary + execution facts once per their existing contracts, including failure paths. + Applications that select neither new capability retain existing behavior. + +The consumer implementation motivating this request is Audita's +`internal/framework/llm/adapter.go`, especially `RoleClient.CompleteStructured`. +This gap affects full coordinator adoption; it does not block adopting v0.22.0 +capture, continuation, or execution groups independently. diff --git a/docs/roadmap/execution-controls.md b/docs/roadmap/execution-controls.md index d692358..6e0ba3b 100644 --- a/docs/roadmap/execution-controls.md +++ b/docs/roadmap/execution-controls.md @@ -1,8 +1,8 @@ # Execution Controls For Coordinated Turns -Status: all four design decisions accepted; ready for API and staged -implementation planning from v0.22.0. No code changes are authorized by this -roadmap. Public names and signatures are not yet selected. +Status: all four design decisions accepted. This document defines the target +state from v0.22.0; [implementation.md](implementation.md) selects the concrete +API and staged implementation work. The feature is not yet implemented. ## Purpose @@ -139,37 +139,17 @@ controls need not take effect during `Prepare` or `ValidateRequest`; accepting request-shaped input there does not itself justify rejecting such controls or starting a timer. Follow the accepted split-preparation and callback semantics. -## Work From The Baseline +## Affected Areas -1. **Design the public API.** Write small ordinary-run, coordinated-turn, and - split-preparation usage sketches under the accepted lifecycle below. Specify - omitted/zero/negative timeout handling using ordinary duration conventions, - and express the selected callback timing and cleanup in the public contract. - Document the entry-point distinctions and conditional diagnostic availability. - Choose names and signatures without adding a broader lifecycle-hook framework. -2. **Add operation-budget handling.** Establish and clean up the child scope at - the selected execution boundary. Preserve earlier parent deadlines and public - error identity through facade mapping and terminal observation classification. - Determine the outcome before terminal delivery and prevent observer latency - or child-context cleanup from changing it afterward. - Ensure cleanup does not affect the coordinator parent or subsequent callbacks. -3. **Expose the prepared snapshot.** Deliver detached public details from the - existing prepared state before generation. Cover both ordinary execution and - the selected prepared-execution lifecycle without a second preparation pass. - Preserve credential handling and release resources on cancellation or panic. - Avoid snapshot-copy work when the capability is not selected. -4. **Integrate and verify coordination.** Have ordinary execution honor the - selected controls on the callback-returned request under the accepted approach. - Preserve capture and observer composition; verify the next-turn callback runs - with the parent context after admission and child-budget cleanup. -5. **Document and demonstrate.** Update public GoDoc, the - [consumer guide](../consumers/pkg-promptkit.md), relevant internal lifecycle - documentation, and a focused offline example when implementation lands. - Show how applications retain diagnostics on failure and bound independent - assessment calls. Reconcile Audita's scenarios against the final API. +The target spans request controls, execution-context lifetime, detached prepared +details, public error projection, and ordinary coordinator integration. It is an +additive runtime feature; no collection-format or ConfigKit changes are needed. +Public GoDoc will own exact API contracts, the +[consumer guide](../consumers/pkg-promptkit.md) will explain use, and internal +lifecycle documentation will describe the implementation when it lands. -This is an additive runtime feature; no collection-format or ConfigKit changes -are needed. Keep implementation proportional to the two requested controls. +The implementation plan owns API selections, task sequencing, concrete changes, +and validation steps. Keep that work proportional to the two requested controls. ## Validation Targets @@ -213,8 +193,8 @@ and the accepted lifecycle choices: ## Accepted Decisions All four decisions below are accepted. No significant product decisions remain -open. Exact public names, signatures, timeout-value conventions, and the Go -representation of timeout identity remain for API and implementation planning. +open. The implementation plan selects public names, signatures, timeout-value +conventions, and the Go representation of timeout identity. ### 1. Narrow Execution Controls Or Caller-Delegated Execution? @@ -295,8 +275,8 @@ inspectable operation-timeout identity when this newly selected budget causes termination. Keep parent cancellation/deadlines and provider-local failures under their existing identities. The marker must survive public wrapping and must not be inferred from any arbitrary `DeadlineExceeded` returned by a -collaborator. Choose its Go representation during API design; no new -error-report hierarchy or terminal-event category is implied. +collaborator. Its Go representation is selected in the implementation plan; +no new error-report hierarchy or terminal-event category is implied. This lets callers distinguish the requested local budget from an earlier parent or provider deadline without retaining the internal child context. Define cause diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md new file mode 100644 index 0000000..1a8dee9 --- /dev/null +++ b/docs/roadmap/implementation.md @@ -0,0 +1,441 @@ +# Execution Controls Implementation Plan + +## Objective And Sprint Scope + +Implement the accepted target in [execution-controls.md](execution-controls.md) +from the v0.22.0 execution architecture. The feature roadmap owns intent and the +four accepted lifecycle decisions; this document selects the API and the work +needed to implement them. [Audita's request](audita-request.md) supplies the +consumer scenarios. + +Work through Stage 1 through Stage 8 in order. Each stage is scoped for one Luna +coding-agent prompt. This is one end-to-end sprint: intermediate stages may +temporarily fail to compile or leave wiring incomplete. Do not build temporary +compatibility layers merely to make intermediate commits green. There are no +intermediate human-review or external-publication gates. Completion requires +the assembled feature and its validation, not just completion of individual +stages. + +Read [development.md](../development.md), the [agent policy](../policy/agents.md), +and the applicable architecture, testing, and documentation policies before +work. Preserve unrelated changes. Follow the existing commit policy for finished +units; report intermediate validation honestly. This plan authorizes no release, +push, dependency update, configuration-format change, or new consumer workflow +beyond the execution controls described here. + +## Selected API And Semantics + +Add these fields to the existing public `RunRequest`, without introducing an +execution-options object, coordinator executor, or general hook framework: + +```go +OperationTimeout time.Duration `json:"-" yaml:"-"` +OnPrepared func(context.Context, PreparedRun) `json:"-" yaml:"-"` +``` + +Add one public sentinel, alongside the current execution errors: + +```go +var ErrOperationTimeout = errors.New("operation timeout exceeded") +``` + +The spelling of an error message is not a contract. No exported timeout error +type, new observer event, or new failure-category enum is needed. + +### Entry-Point Contract + +| Entry point | OperationTimeout | OnPrepared | +| --- | --- | --- | +| `Run` | A positive value starts a fresh child budget before preparation, covering preparation and execution. | Notify once if execution reaches the prepared point before first generation. | +| `Coordinate` | Each returned request receives ordinary `Run` semantics with a fresh scope. | Ordinary per-request notification; no coordinator-specific copy of the option. | +| `PrepareExecution` | Retain the value privately, without starting a timer. Preparation uses its supplied context. | Retain the selected function privately without invoking it. | +| `RunPrepared` | Start the retained positive duration after the successful claim, before execution callbacks, credential revalidation, and admission. | Notify once when the claimed execution reaches the prepared point. | +| `Prepare`, `ValidateRequest` | Do not start a timer; use the supplied context as today. | Do not invoke or retain the notification. | +| `PreparedExecution.Details`, `Discard` | Do not start a timer. | Do not invoke the notification. Discard drops its retained reference. | + +- A positive duration opts in. Zero and negative durations select no additional + timeout. This follows a nonpositive-means-unselected convention, avoids a new + request rejection rule, and leaves the parent deadline effective. There is no + default operation duration, upper bound, profile setting, or engine setting. +- The provider's existing generation timeout remains independent. Do not copy + `OperationTimeout` into `ExecutionTarget`, provider JSON, or prepared/result + metadata. Neither control changes prompt hashes or configuration inspection. +- The budget is continuous across preparation, capacity waiting, notification, + generation, transient retries/backoff, validation, and structural repair. It + is not restarted between phases, attempts, or repair generations. +- Split preparation retains a duration, not a context, deadline, timer, or + cancel function. Idle time does not count. A shared caller-created deadline + remains the way to bound preparation and later execution together. +- A nil notification selects no diagnostic copying. A selected function receives + the operation context and a detached, credential-redacted `PreparedRun` value + representing the logical request. Its nested mutable values belong to the + application. Reuse the existing projection used by `Details()`. +- Notification is synchronous, has no error return, and runs after successful + preparation and admission, before the first generation. Logical admission is + still held. Its time consumes the selected operation budget. It may retain + details for later diagnostics; PromptKit does not own writes or write errors. +- Check context before notification and again before generation after it + returns. A callback may block or ignore context; do not introduce a goroutine + to interrupt it. Callback panic propagates after deferred resource cleanup, + without a new terminal-event guarantee. +- Callback selection is copied as a function value, not as a deep copy of its + closure. Shared callback state is application-synchronized; separate runs can + invoke a common callback concurrently. Later reassignment of the original + request field cannot replace a prepared handle's retained function. +- Notification is conditional on reaching that execution point. Admission or + preparation failure can prevent it. Do not render again, reorder admission, + or promise a diagnostic snapshot for every failed operation. Split callers + can still inspect a successfully prepared handle through `Details()`. + +### Timeout Attribution And Completion + +Use the standard library's `context.WithTimeoutCause` with a distinct cause +value for each selected scope. Record the scope's own cause identity so an +inherited timeout from an enclosing operation cannot be mistaken for this +invocation's timer. Do not classify by error text or by `DeadlineExceeded` +alone. No strict ordering of independently racing cancellations is promised. + +When an execution fails through its context deadline and the child context's +recorded cause is this scope's own timer cause, return an error matching both +`ErrOperationTimeout` and `context.DeadlineExceeded`. Preserve existing operation +and collaborator error identities. An earlier parent deadline/cancellation or +a provider deadline must not acquire the local marker merely because it is a +deadline error. An unrelated failure must not acquire it merely because the +timer has also expired. Parent-provided error causes, if any, retain their +existing behavior; this feature need not reinterpret them. + +Finalize attribution from the operation's returned error before canceling its +child scope and before terminal observer delivery. A nil execution error remains +a completed result, including a completed structural rejection. Do not add a +post-terminal `ctx.Err()` check that converts it into an error. Slow terminal +observers and deferred context cleanup cannot change the chosen outcome. This +does not strengthen cancellation into forced interruption or a guarantee that +every context-ignoring collaborator will return by the deadline. + +The terminal event continues using the existing deadline failure category for +a timeout error. Groups continue aggregating those ordinary terminal facts. +The child scope is cleaned up before `Run`/`RunPrepared` returns to the next-turn +callback; that callback receives the coordinator parent context, not the child. +Existing parent cancellation and undelivered-result rules remain unchanged. + +## Implementation Map + +Use existing owners rather than adding a new public package: + +- Public declarations and contract: [types.go](../../types.go), + [engine.go](../../engine.go), and [prepared_execution.go](../../prepared_execution.go). +- Public conversion and error projection: [convert.go](../../convert.go), + [errors.go](../../errors.go), and [observations.go](../../observations.go). +- Internal request values: [domain.go](../../internal/domain/domain.go). +- Timeout lifecycle and notification orchestration: `internal/usecase`, chiefly + [runner.go](../../internal/usecase/runner.go) and + [prepared_execution.go](../../internal/usecase/prepared_execution.go). +- Existing private operation-error representation: + [operation_error.go](../../internal/usecase/operation_error.go). +- Ordinary coordinator integration: [coordinate.go](../../coordinate.go) and + [internal coordinator](../../internal/usecase/coordinate.go). + +The root remains a facade. Keep timer construction and cause-attribution logic +in a small internal helper; the facade may establish that scope and project its +errors while managing its existing observation lifecycle. Internal packages must +not import the root package or depend on the public `PreparedRun` representation. +Private helper names and equivalent internal organization may change; the +selected public behavior and ownership boundaries are what matter. + +## Stage 1: Add The Request Contract And Conversion + +Outcome: the selected API and internal request plumbing exist, ready for the +execution stages to consume. + +1. Add `OperationTimeout`, `OnPrepared`, and `ErrOperationTimeout` to the public + declarations above. Write their GoDoc from the entry-point and ownership + contract in this plan, explicitly separating operation and provider timeouts. +2. Extend `domain.RunRequest` with the duration and an internal callback of shape + `func(context.Context, *domain.PreparedRun)`. Document the callback argument + as borrowed read-only internal state. It is not a public mutable snapshot. +3. Extend `toDomainRunRequest` to copy the duration and capture the selected + public callback function in a local variable. Only when it is non-nil, adapt + it by projecting the supplied prepared state through `fromDomainPreparedRun` + and passing the resulting value. This projection is the public ownership + boundary; do not deep-copy the domain snapshot first as well. +4. Keep these fields out of public prepared/result JSON and opaque handle + details. Request conversion does not start a timer, invoke a callback, or + reject an execution-only control on preparation-only entry points. +5. Add the private timeout error identity in `internal/usecase`, using the same + existing separation between internal and public sentinels. Add its public + projection to `errors.go` when introducing the attribution wrapper in Stage 2. + +Validation: focused conversion tests should establish option forwarding and +conditional adaptation. Reuse existing nested-value projection tests instead of +retesting every existing `PreparedRun` field. Compilation may remain incomplete +until the execution wiring lands; do not claim the feature works at this stage. + +## Stage 2: Bound Ordinary Execution And Preserve Its Error Outcome + +Outcome: `Engine.Run` owns one continuous optional budget and returns the same +failure classification it publishes to observers. + +1. Add a small internal operation-scope helper using `WithTimeoutCause`. For + nonpositive durations, reuse the parent context without a timer. For positive + durations, retain a fresh per-invocation cause token and a cancel function. + Use a real distinct identity, not a shared sentinel or zero-sized token whose + pointer identity cannot distinguish scopes. +2. Give the helper an error-finalization operation: for a non-nil returned error, + require both a match to the operation context's deadline error and equality + of its recorded cause with this scope's token before adding the private + timeout identity. Leave nil errors and unrelated errors unchanged. Preserve + the original error as a cause. +3. Reuse `usecase.OperationError` with the private timeout category to carry the + marker around the existing error, rather than creating a public error + hierarchy. Project that category to `ErrOperationTimeout` through the + existing `publicOperationError` machinery. Keep the original phase category + and context cause inspectable through the nested chain. +4. Establish the ordinary scope in the facade's `Run` entry, after masking + inherited execution-observation state and before start-event delivery and + request preparation. Both observed and unobserved runs use it. Pass its + context through request conversion, internal engine delegation, admission, + preparation, and all generation/validation work. Do not also start a second + timer inside `Runner.Run`. +5. Adjust the private facade `run` path as needed to finalize the execution + error before applying public error projection once. On normal return, freeze + attribution and the returned result/error, clean up the child, then classify + and deliver the existing terminal event. Preserve normal observation of + request conversion and preparation failures. Handle all early returns; install + cleanup before any user callback can panic. +6. Do not change provider timeouts, retry eligibility, repair limits, scheduling, + or existing result timing conventions to implement this feature. A timer + expiring while terminal observation runs is not another execution failure. + +Validation: narrow helper tests cover disabled scopes, local expiry, parent +deadline/cancellation, inherited operation causes, unrelated errors, and nil +completion. Add public `Run` tests proving the marker survives phase wrapping +and agrees with an ordinary deadline terminal event. Provider-only deadline +errors while the operation child is active must not gain the marker. + +## Stage 3: Retain Controls Across Split Preparation + +Outcome: prepared execution carries the selected duration and notification, +without keeping a live operation scope while idle. + +1. Extend `preparedExecutionPayload` with the duration and internal notification. + `PrepareExecution` copies them from the converted request only after its + ordinary successful preparation. Preparation itself uses its supplied + context, without starting the selected timer or calling the notification. +2. Extend payload clearing for both fields. Preserve the existing atomic claim, + owner checks, foreign/reused-handle behavior, and durable redacted details. + A discard or completed/unwound claim drops PromptKit's callback reference. +3. After a successful `Runner.RunPrepared` claim, establish the scope with the + retained duration, before the retained observer's start delivery, credential + revalidation, and admission. Defer payload and scope cleanup before callback + work. A failed claim neither starts notification nor changes existing silent + invalid-handle behavior. +4. Pass the child through execution and finalize the returned error with the + same helper before cleanup and facade terminal classification. Keep this + lifecycle in the internal prepared runner rather than borrowing duration or + callback state from an unclaimed handle in the facade. +5. `RunPrepared`'s facade projects the resulting error and emits its one existing + terminal event after resource release. Do not reapply the timer at the facade + or retain the preparation context. Namespace views keep their shared-owner + behavior and use the handle's original selections. + +Validation: extend prepared-execution coverage to show that preparation and an +idle delay longer than the selected duration do not consume its execution +budget; later execution does. Reassignment of the original request does not +change the retained duration/callback. Cover earlier caller deadlines and a +shared caller deadline across preparation and execution. Existing claim-race, +discard, foreign-owner, credential, and error-identity tests remain applicable. +Notification invocation itself is completed in Stage 4. + +## Stage 4: Notify From The Actual Shared Prepared Snapshot + +Outcome: ordinary and prepared execution expose one detached logical request +before first generation, without repeating preparation. + +1. Thread the internal notification from `Runner.Run` and the claimed payload + into the existing shared `executePreparedRun` entry. At that entry, preparation + is complete and logical admission is held on both paths. +2. When selected, check the operation context, invoke the internal adapter with + the actual retained prepared state, and check context again before entering + generation. A context failure at this boundary follows the existing + generation operation-error wrapping, preserving context identity and gaining + local-timeout identity only through the scope finalizer. +3. Invoke once per executing logical request, not per transport retry or repair. + Do not call `Prepare`, `Details`, artifact readers, renderers, or schema + preparation again to produce the notification. With nil `OnPrepared`, avoid + constructing a public details copy. +4. Keep the prepared state's API key redacted. Generation still receives its + private direct credential through the existing separate path. Preserve the + complete assembled messages, schema, settings, input hashes, session, + ancestry, and preparation metadata in the delivered public value. +5. Leave logical admission held; do not release/reacquire it around notification. + Invoke application code without holding the handle mutex or a capacity + manager lock. Preserve panic propagation while defers release the lease, + timer, and claimed payload. A panic does not make a handle reusable. + +Validation: one ordinary and one split public test establish ordering before +generation and exact callback selection. Add focused context-before/after +notification tests and an internal cleanup regression if the public capacity +probe alone cannot demonstrate payload cleanup. More extensive ownership and +failure assertions belong to Stage 6, not duplicate tests here. + +## Stage 5: Verify Continuous Budgets Across Execution Phases + +Outcome: tests demonstrate that one scope reaches all relevant collaborators, +with correct attribution and observation on failure. + +1. Add focused tests for preparation expiry through a context-aware injected + artifact reader, generation-capacity waiting on a real limited backend, + interrupted built-in retry backoff, and structural repair expiry. Verify + standard context matching, local marker, and retained phase identity where + that phase already supplies one. Cover ordinary and split paths without + mechanically duplicating every case on both. +2. Use a narrow internal runner test for validation-phase expiry, since validation + collaborators are not a public injection API. Check the same deadline reaches + preparation, validation, and generation; do not add a new public testing seam. +3. Test that notification time consumes the same budget and that expiry there + prevents generation. Cover an earlier parent deadline and parent cancellation + separately from a provider's own deadline. Preserve original injected error + identities and do not relabel a non-context error after timer expiry. +4. On both ordinary and split public execution, exercise a terminal observer + that outlasts the operation budget after successful completion. The result + stays successful and the terminal remains successful. Include the timeout + failure case to ensure public error mapping happens before terminal delivery. +5. After expiry, prove capacity remains usable and the caller parent stays + active. Respect the current capacity model: logical admission may reject + immediately, while admitted work can wait for an active generation permit. + Do not introduce a new blocking admission queue to satisfy a test label. + +Use channels for ordering and `testing/synctest` for controlled elapsed-time +scenarios where its bubble rules apply. This repository's Go minimum supports +it. Prefer in-memory fixtures and controlled transports; avoid real short sleeps +or close-to-deadline assertions. Reuse the existing generation tests for retry +policy rather than building a second retry matrix. There is no production clock +framework, live provider probe, allocation threshold, or performance gate here. + +Validation: run focused use-case/generation tests and the new public execution +control tests under the race detector. The new cases should protect deadline +propagation and classification, not private helper call sequences. + +## Stage 6: Verify Notification Ownership And Failure Lifecycles + +Outcome: applications can safely retain diagnostic details without altering +execution or acquiring new lifecycle guarantees. + +1. Add a representative continuation request containing history, follow-up + messages, appended cache-control metadata, input hashes, session, settings + with nested provider values, and a structured schema. Inside `OnPrepared`, + retain and mutate the delivered mutable values; the model request, captured + exchange, and later prepared-handle `Details()` remain unchanged. Use + synthetic direct/environment credentials and verify their values are absent + from diagnostic details while the existing client credential path still works. +2. Demonstrate snapshot fidelity when mutable input/schema sources change after + preparation and before execution. Reuse existing frozen-preparation fixtures; + notification must describe the executed snapshot, not reread the changed + sources. A later provider failure or cancellation may return no result, while + the application still retains the delivered details. +3. Verify absence of notification for preparation failure and ordinary admission + rejection, without requiring extra reads to produce diagnostics. For split + execution cover invalid/foreign/discarded/reused handles and pre-notification + cancellation. `Details()`, `Prepare`, `ValidateRequest`, and preparation of an + executable handle never notify or start this execution timer. +4. Verify that retries and structural repair do not repeat notification. Reuse + an existing retry-plus-repair integration scenario with an added notification + assertion rather than duplicating its full event/usage test elsewhere. +5. Recover a notification panic only in test code, then verify ordinary capacity + is reusable and a claimed handle stays consumed. Do not require a terminal + event after panic or impose a prohibition on callback reentrancy. A capacity + one probe documents held admission rather than promising nested success. + +Validation: run affected public contract and private payload-cleanup tests under +the race detector. Mutable callback state in concurrent tests must be protected +by the test; do not make PromptKit serialize application callbacks globally. + +## Stage 7: Verify Coordinated Turns And Observer Composition + +Outcome: coordination obtains the controls through ordinary `Run`, retaining +its existing branching and delivery semantics. + +1. Check the coordinator's existing request-value copy preserves both new + controls while it replaces `ContinueFrom` and composes observers. No separate + coordinator timeout, executor, or public prepare/run sequence is needed. +2. Add a controlled-time, two-continuation success case. Each turn finishes + within its own timeout while their combined execution exceeds one turn's + duration. Callback details contain each turn's assembled history; no timer + from an earlier turn controls a later turn. +3. In the next-turn callback, verify the parent context is active and the prior + child context has been cleaned up. Run an explicitly selected assessment on + the same backend with concurrency one and queue capacity zero, using its own + timeout. This differs from `OnPrepared`, where admission is still held. +4. Exercise a locally timed-out continuation: coordination returns its ordinary + operational error with the marker, does not fabricate an outcome or completed + exchange, and leaves the parent and a sibling continuation usable. Retained + details are available if notification occurred before failure. Do not promise + them if execution stopped earlier. +5. Compose request and coordinator observers with an explicit execution group. + Check one ordinary lifecycle per selected run, normal terminal usage/attempt + totals, and no implicit membership for assessment calls. Reuse existing + final-turn and parent-cancellation/undelivered-result cases to verify that + cleanup and slow terminal observation have not changed delivery semantics. + +Validation: run the coordinator, observation, group, and new execution-control +contract cases together under the race detector. Keep tests at the public +boundary for the assembled behavior; the internal coordinator loop need not +be redesigned solely because its ordinary run gained options. + +## Stage 8: Document, Demonstrate, And Validate The Assembled Feature + +Outcome: the implementation and its supported consumer workflow are complete, +with documentation in the canonical owners. + +1. Finish GoDoc on `RunRequest`, `ErrOperationTimeout`, `Run`, `Prepare`, + `ValidateRequest`, `PrepareExecution`, `RunPrepared`, and `PreparedExecution`. + Cross-reference the owning field/error contract rather than duplicating its + entire definition. Explain conditional notification, held admission, + cooperative cancellation, split preparation, callback ownership, and outcome + finalization before terminal observation. +2. Update the [consumer guide](../consumers/pkg-promptkit.md) with small ordinary, + coordinated, and split usage sketches linked to GoDoc. Show a caller-owned + retained `PreparedRun` on failure, `errors.Is` checks for both timeout + identities, and a separate budget for callback-owned assessment. Do not add + persistence, log destinations, redaction policy, or a mandatory diagnostic + write gate. +3. Extend the maintained [observation example](../../examples/go-library/observe/main.go) + and its semantic test with a positive operation budget on continuation, + selected prepared-details retention, and a separately budgeted next-turn + assessment. Keep it offline, synthetic, and successful; Stage 5/6 tests own + failure demonstrations. Do not add another standalone example framework. +4. Update [runner documentation](../internal/runner.md), + [observation documentation](../internal/observation.md), and the + [component inventory](../internal/overview.md) only where responsibilities or + lifecycle descriptions changed. No ConfigKit, collection-format, provider + payload, or architectural policy change is required by this work. +5. Review Audita's six acceptance scenarios and the feature roadmap's validation + targets against the completed tests. Consolidate duplicate tests; do not + expand this into a full cross-product of all entry points, phases, observers, + and callback behaviors. Retain existing no-option regression coverage. +6. Select the newest installed Go toolchain per the development guide. Format + changed Go files, run `go test -race ./...` and `go vet ./...` with + `GOTOOLCHAIN=local` and `GOWORK=off`, and run `git diff --check`. The full Go + suite includes the maintained examples; no second example execution gate is + needed. Use writable caches as required by the development environment. + Validate local links and inspect the full diff for unintended files/secrets. + +Do not edit published v0.22.0 release notes, create a new release/tag, or require +CI or external publication to finish the sprint. Release preparation and +retirement of these roadmaps are subsequent work. + +## Completion Criteria + +- Ordinary and coordinated runs apply one optional whole-operation budget; + split handles retain duration and callback selection without live idle timers. +- Deadline attribution survives public wrapping, distinguishes this scope's + recorded timer cause, and agrees with terminal observation before cleanup. +- Notification supplies a detached redacted view of the actual logical prepared + request, once before generation when reached, without repeated preparation. +- Failure, cancellation, and panic paths preserve existing resource, handle, + error, observation, grouping, and continuation behavior. +- The maintained example, canonical documentation, focused tests, full race + suite, and vet are complete. No-option use stays compatible. + +No significant open questions remain. The API and nonpositive-duration choices +above complete the planning decisions left by the accepted feature roadmap.