base-layer/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Prepare v0.22.0 release notes and retire completed roadmaps

Browse Source
This commit is contained in:
Eric Rakestraw
2026-11-14 14:33:41 +00:00
parent 892cb74e6e
commit 65546d794e
6 changed files with 70 additions and 1453 deletions
Download Patch File Download Diff File Expand all files Collapse all files

67
docs/releases/v0.22.0.md Normal file
View File

@@ -0,0 +1,67 @@
# Promptkit v0.22.0
This minor release adds captured conversation exchanges, optional sequential
coordination, and explicit execution-group summaries. Applications can continue
or branch a conversation without reconstructing rendered history, while keeping
response interpretation and workflow decisions in application code.
## Capture And Continue Conversations
Set `RunRequest.CaptureExchange` to capture a completed run's effective messages
and final assistant content. Supply its `RunResult.Exchange` as `ContinueFrom`
on a new request to continue it. Exchanges are reusable, support independent
branches and different engines, and expose detached history through `Details()`.
Prepared execution also supports capture of the exact snapshot it executes.
Internal structural-repair instructions and intermediate candidates are not
conversation history. A completed result with failed structural validation can
still be captured; an operational error returning no result creates no exchange.
Follow-ups select their own profiles, output contracts, credentials, and sessions
under ordinary request rules. See [capture and continuation guidance](../consumers/pkg-promptkit.md#capture-and-continue-a-conversation)
and the [public exchange contract](../../exchange.go).
## Coordinate Application-Directed Turns
`Engine.Coordinate` advances one conversation branch using an application callback
and an additional-turn budget. The application selects the next request or stops;
Promptkit handles turn progression and cancellation. Each completed run releases
backend admission before the callback, allowing assessment calls to use the same
backend. Stop and budget exhaustion are control-flow outcomes, not judgments of
response acceptance. Transport retries and structural repairs remain separate
from the turn budget.
See the [coordination walkthrough](../consumers/pkg-promptkit.md#coordinate-application-directed-turns)
and [coordinator GoDoc](../../coordinate.go) for callback and cancellation behavior.
## Collect Selected Execution Facts
`ExecutionGroup` collects explicitly selected execution observations, while
`ComposeObservers` combines groups with existing diagnostic observers. Detached
group snapshots report member identities, terminal coverage, attempt/retry/repair
counts, and available token totals without counting a run more than once.
Terminal coverage remains distinct from token-usage completeness; membership is
not inferred from ancestry or nested calls.
See [execution-group guidance](../consumers/pkg-promptkit.md#collect-selected-execution-facts),
the [group GoDoc](../../execution_group.go), and the updated
[offline observation example](../../examples/go-library/observe/main.go).
## Efficiency And Compatibility
History assembly and inspection avoid intermediate full-history copies. Observer
composition retains storage only for selected callbacks and avoids fan-out
closures for zero or one selected callback.
Capture and coordination are opt-in; existing execution workflows and
configuration formats need no migration from v0.21.0. The Go consumer minimum
remains 1.27.0, and dependency versions are unchanged. Exported request, prepared,
result, and observation structs gained fields; consumers using positional struct
literals must switch to keyed literals. Continuation metadata adds `parent_run_id`
to prepared/result JSON when applicable; the opaque exchange itself is excluded.
Exchanges are in-process capabilities, not serialized restartable sessions.
```sh
go get go.maximumdirect.net/promptkit@v0.22.0
```
[COPYRIGHT](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.22.0/COPYRIGHT) | [LICENSE](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.22.0/LICENSE) | [ATTRIBUTION](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.22.0/docs/oss/attribution.md)

169
docs/roadmap/audita-request.md
View File

@@ -1,169 +0,0 @@
# Audita request: conversation continuation and application-directed repair
Status: proposed upstream features, requested by Audita. Public API design and
implementation remain for PromptKit to determine.
This request is based on Audita's use of PromptKit v0.21.0. It proposes two
related capabilities: captured conversation continuations, followed by bounded
repair driven by application-supplied validation feedback. The goal is to move
reusable execution machinery into PromptKit and remove its local equivalents
from Audita.
## Audita's workflow
Audita polishes transcripts using module-specific proposal, validation, and
repair prompts:
1. A proposal request supplies transcript segments and receives a structured
collection of proposed corrections.
2. Audita evaluates each correction using deterministic and LLM-backed
validators. Accepted corrections are retained for later application.
3. Each actionable rejection starts an independent repair branch containing the
original rendered request, the full final assistant response, and a follow-up
instruction rendered from that module's repair prompt definition.
4. The follow-up identifies one correction and supplies its current content,
available before-and-after transcript context, and validation findings. The
model can revise or withdraw that correction. This response uses a different
schema from the original collection of proposals.
5. A revision passes through the validators again. Further attempts retain that
branch's history and receive updated feedback. Acceptance, withdrawal,
exhaustion, or failure ends the branch; independent approvals are preserved.
Audita owns the meaning of a correction, the validation rules, repair
eligibility, and eventual application to the transcript. Conversation handling
and the bounded revision lifecycle are useful beyond transcript processing.
## Existing support and the remaining gap
PromptKit already supports [structural output repair](../consumers/pkg-promptkit.md#repair-a-structured-result),
[appended messages](../consumers/pkg-promptkit.md#append-already-rendered-messages),
prepared execution, profiles, execution observations, and backend capacity
management. These allow Audita to implement the workflow today.
Audita still captures exchanges, reconstructs the original prompt, verifies its
rendered prefix, assembles branch histories, renders follow-up instructions,
and coordinates repeated application validation and revision. The requested
APIs would make those responsibilities supported PromptKit capabilities.
## 1. Captured conversations and independent continuations
Provide a public way to capture a rendered request and its full final assistant
response, then continue that exchange without making the consumer reconstruct
the original prompt and check that its messages are unchanged.
Audita needs to:
- Preserve the original rendered message content, roles, and cache metadata
when continuing an exchange.
- Start multiple independent branches from the same exchange. A branch's later
instructions and responses must not alter the original exchange or another
branch's history.
- Render follow-up instructions using ordinary PromptKit prompt definitions and
variables, and append them to the selected branch. Audita's repair prompts
remain operator-replaceable through its existing PromptKit configuration.
- Select a follow-up profile and output contract independently of the initial
request. The first response may contain many items while a continuation
returns one revised item or a withdrawal.
- Retain the relationship between the original exchange, the follow-up prompt
definition, and each execution in inspection and observation data.
An in-process capability is sufficient for Audita. Persistence, restartable
sessions, and a conversation database are not required by this request. The
existing [prepared execution handle](../../prepared_execution.go) is single-use;
the request does not assume that copying or reusing it creates a continuation.
## 2. Bounded application-directed repair
Build a reusable revision coordinator on the continuation capability. Audita
should supply its assessment and request-building behavior while PromptKit
handles the repeated execution lifecycle.
The coordinator needs to support:
- Starting with an already-assessed candidate and its rejection findings, so
Audita need not repeat the initial batch validation simply to enter repair.
- Application-supplied assessment of each revision, including deterministic
checks and checks that make further PromptKit calls. Findings may come from
multiple validators; their meaning and explanatory content remain with the
application.
- Application-selected follow-up prompts, variables, profiles, and response
contracts. Audita supplies the candidate and context, and interprets the
response as a revision or withdrawal. PromptKit need not prescribe an Audita
correction schema or a universal feedback format.
- Reassessment before a revised candidate can be accepted, with fresh feedback
for the next attempt and accumulated history confined to that branch.
- A caller-selected additional-revision budget and caller context, with clear
terminal outcomes covering acceptance, withdrawal, exhaustion, unavailable
results, and interruption. Callers retain decisions about repair eligibility
and whether an operational failure should terminate other work.
- Attempt observations, execution relationships, and available usage accounting
sufficient to report a repair sequence without recreating its lifecycle
tracking in Audita.
The revision budget must remain distinct from transport retry and structural
repair budgets. A structurally valid response can still fail Audita's checks;
each revision generation can itself need structural repair. Accounting should
distinguish those layers and avoid double-counting shared initial work.
Application validators must be able to call the same engine and backend without
being blocked by capacity retained by their parent operation. Coordinating
ordinary completed runs is one possible approach. PromptKit should determine
the implementation in light of its [execution and admission lifecycle](../internal/runner.md#run-flow).
This is broader than exposing the current structural-repair hook: the response
schema can change between proposal and repair, validation may perform model
calls, and each rejected item has its own history and disposition.
## Ownership boundary
PromptKit would own captured exchanges, branch continuation, the bounded repair
lifecycle, and associated execution observations. Audita would retain:
- Transcript segments, proposal identities, glossary and protected-term rules,
editorial judgments, and meaning checks.
- Validator selection and ordering, actionable-rejection policy, feedback
content, and module-specific proposal, validation, and repair prompts.
- Before-and-after transcript previews, preservation of accepted sibling
proposals, application ordering, and final transcript mutation.
- Application reports, debug artifact destinations, and CLI behavior.
The separate idea of reconciling missing or duplicate items in batch responses
is outside this request. The two requested capabilities should be usable
through the supported public facade without an Audita dependency or a general
workflow framework. Exact types, callbacks, and configuration syntax are left
to PromptKit's API design.
## Suggested delivery and acceptance scenarios
Deliver conversation continuation first, allowing Audita to replace its capture
and replay machinery. Then add application-directed repair and replace the
corresponding local coordination. Existing prompt definitions and domain
validators should remain usable through the new APIs.
The following scenarios would demonstrate that the capabilities meet Audita's
needs:
1. An initial response contains three proposals. One is accepted; the other two
receive independent repair conversations containing the same original
exchange. Revising either branch leaves the accepted proposal and the other
branch unchanged.
2. A follow-up uses a separate prompt definition, profile, and single-item
response schema. Inspection identifies the follow-up and its originating
exchange.
3. A revision fails multiple application checks, receives their updated
feedback, and passes on a later attempt. An LLM-backed check can use the same
engine and backend even when backend capacity is limited to one.
4. Withdrawal, exhausted revision budget, unavailable validation, and
cancellation produce distinguishable outcomes. An unaccepted revision is
never returned as an accepted candidate, and a recoverable branch failure
leaves independent approvals available to the application.
5. A revision generation needs both a transport retry and structural repair.
Observations distinguish these from application-directed revision attempts
and expose the available usage without counting the original exchange once
per branch.
For implementation reference, Audita currently owns these mechanics in
`internal/framework/llm/adapter.go`,
`internal/framework/structuredoutput/call.go`, and
`internal/framework/runner/repair.go`. These are examples of the consumer need,
not proposed PromptKit package boundaries or public API shapes.

448
docs/roadmap/conversation-continuation.md
View File

@@ -1,448 +0,0 @@
# Conversation Continuation And Execution Groups
Status: target behavior accepted; concrete API choices and implementation steps
are recorded in the [implementation plan](implementation.md). This document
defines the target behavior and responsibility boundaries. Execution groups are
independently usable.
## Purpose And Responsibility Boundary
PromptKit will let applications continue logical conversations, optionally
coordinate successive turns, and summarize explicitly selected executions.
These capabilities remove repeated history reconstruction, turn lifecycle work,
and observation aggregation without interpreting prompt or response content.
[Audita's request](audita-request.md) supplies consumer evidence and acceptance
scenarios. Repair, analysis, clarification, and branching exploration should all
use the same content-neutral capabilities. New concepts and names must be useful
without translating an application's domain into repair terminology. This does
not rename or expand PromptKit's existing structural-output repair functionality.
| Capability | PromptKit owns | Application owns |
| --- | --- | --- |
| Captured exchanges and continuation | Logical history, reusable branches, ordinary follow-up execution, exchange details and parent metadata | Which exchange to continue and which request to execute |
| Optional sequential coordinator | One branch's turn progression, additional-turn budget, callback delivery and cancellation checks | Response interpretation, next action, branch scheduling and broader failure policy |
| Optional execution groups | Explicitly selected observations, deduplication and execution-fact summaries | Membership selection, domain identifiers, final-report timing and presentation |
Continuation is independently usable. The coordinator builds on it; applications
can instead use their own loops. Groups work with ordinary runs as well as
continuations and require neither capture nor a coordinator. Selecting a group
does not change scheduling, capacity, cancellation, retries, or validation.
The scope is in-process use. Persistence, restartable workflows, cross-process
resumption, a conversation registry, batch reconciliation, and a branching-workflow
scheduler are outside this feature. Applications schedule branches, join results,
and decide whether one branch's failure should affect other work. This initial
scope is not a permanent prohibition on future capabilities.
## Captured Exchanges And Continuation
### Capture And Availability
Capture is opt-in per ordinary or prepared execution. Ordinary execution returns
the exchange alongside its usual result. A prepared handle retains the capture
choice; executing that exact prepared snapshot creates the exchange. Preparation
alone creates no completed exchange. Continuation implies capture so its result
can itself be continued.
Capture the request actually executed, including the complete rendered messages
with their content, roles, and cache metadata, followed by the full final
assistant content returned by the run. Consumers must not reconstruct the request
or pair a response with separately prepared data. “Full” means content supported
by PromptKit, not a new raw provider-envelope API.
Capture represents logical conversation history. Internal structural-repair
instructions and intermediate candidates are excluded; only the final returned
assistant content follows the original request. Existing attempt observations
remain separate from conversation content.
| Execution outcome | Capture behavior |
| --- | --- |
| Completed response satisfying its output contract | Return the captured exchange. |
| Completed response with failed structural validation, including repair exhaustion | Return the exchange with the ordinary validation result intact. |
| Operational error with no completed result | Create no new exchange; the preceding exchange remains usable. |
| Completed response obtained before cancellation prevents coordinator delivery | Preserve that single undelivered response under the coordinator return contract. |
Capture does not imply structural validity or application acceptance. A completed
response with failed validation follows the normal coordinator callback path,
subject to cancellation; it is not reclassified as an operational error.
### Ownership And Reuse
An exchange is an immutable, reusable snapshot held by the caller. Continuing it
adds the rendered follow-up messages and final response to a new exchange, leaving
the source and independent branches unchanged. No previous templates or input
files need to be read or rendered again.
Avoid unnecessary repeated copying of retained history where practical, while
preserving independent branches and detached inspection values. Internal sharing
of immutable history is an implementation option, not a required representation
or a public allocation or complexity guarantee.
Another engine instance in the same process can continue the exchange, applying
its own configuration to the new request. The originating engine is not required.
This permits application engine replacement without promising serialized or
cross-process resumption.
Completed exchanges retain no backend admission lease, direct credential, or
execution callback. Ordinary Go memory management controls their lifetime; there
is no new `Close` or `Discard` lifecycle. The existing
[prepared execution handle](../../prepared_execution.go) remains a separate,
single-use execution capability.
### Exchange Details
Provide explicit access to the held exchange's logical message history, including
its final assistant content, originating run identity, and immediate parent
reference. Reuse existing run identifiers; no mandatory conversation, branch, or
sequence identifiers are introduced.
Follow the general pattern of `PreparedExecution.Details()`. Details must be
generically renderable through ordinary encoders. Reading or modifying returned
values must not affect retained history or another branch. Inspection reads the
snapshot without rendering, loading inputs, or making provider calls.
Existing results and preparation details continue to supply ordinary execution
information. Exchange inspection does not create a second comprehensive report,
a result archive, or an engine-level conversation lookup API. `InspectPrompt`,
`InspectProfile`, and `InspectBackend` keep their resource-inspection boundary.
Captured message content remains separate from metadata-only observations.
## Optional Sequential Coordinator
The coordinator starts from a captured exchange and executes one branch using
an application callback, a caller-selected additional-turn budget, and context.
The callback selects a new ordinary request, stops, or returns an error. It can
keep application state in a closure; PromptKit defines no domain-state or
feedback schema.
### Progression And Budget
With an active context, invoke the initial callback once, even with a zero-turn
budget. It receives the starting exchange and remaining budget; the starting
exchange is not a newly delivered response. Existing application findings can
be used without repeating earlier assessment.
After each continuation, deliver its completed response once through the same
callback before starting another execution, including the final permitted
response. Internal retries and structural repairs do not create extra callbacks.
| Callback action | Coordinator behavior |
| --- | --- |
| Stop | Return application-directed completion, including after the last permitted response. |
| Continue with budget remaining | Start the selected continuation. |
| Continue without budget remaining | Return exhaustion without executing the requested turn. |
| Error | Stop and propagate the error, subject to context checks. |
The budget permits at most N additional continuation executions to start.
Starting a continuation consumes one turn even if preparation or generation
fails. The initial exchange and callback invocations consume no turns; internal
transport retries and structural repairs stay within one turn. The first
operational error stops coordination, without refunds or recovery machinery.
Using the full budget alone does not mean exhaustion: exhaustion means the
application requested more work than the budget permits. The budget does not
bound provider attempts, cost, or work inside callbacks. Applications wanting
no callback work can skip the coordinator entirely.
### Cancellation And Failures
Check context immediately before every callback, including the initial callback.
If cancellation is already observed, skip delivery and return the context error
with its identity preserved. Retain a completed response already obtained but
not yet delivered; do not recover raw output from an underlying failed call that
returned no completed result.
Callbacks receive the same context and cooperate with cancellation. Check it
again after the callback returns and before starting another execution.
Cancellation may race with invocation; PromptKit does not forcibly terminate
callback code or promise callback delivery despite cancellation or panic.
An execution or callback error stops coordination. Preserve ordinary Go error
identities, including `errors.Is`/`errors.As`. Do not automatically replay the
failed continuation or callback. Ordinary transport retries and structural repair
remain execution-internal. Earlier deliveries and application work remain intact;
the application owns any broader consequences.
### Minimal Return Contract
Return only:
- An ordinary Go error for execution failure, callback failure, or cancellation.
- A small normal outcome distinguishing application-directed stop from budget
exhaustion; neither outcome is an error or a claim about content acceptance.
- An optional single completed response when cancellation prevented callback
delivery, reusing existing response types where practical.
Delivery means callback invocation, not successful processing or acceptance. If
the callback returns an error, its response is already delivered and is neither
returned as undelivered nor replayed. Previously delivered responses are not
archived in the termination result. No additional failure hierarchy, failure-stage
fields, run-ID references, or aggregate reporting is added to this contract.
Existing response and observation fields remain available; group summaries are
obtained separately.
## Shared Execution And Accounting Rules
### Follow-Up Configuration
A continuation is an ordinary new execution supplied with captured history.
Resolve its prompt, profile, credentials, output contract, and request overrides
using the receiving engine's existing rules. In particular:
- Use an explicit request profile or the follow-up prompt's default profile,
with ordinary missing-profile behavior when neither is available.
- A supplied output contract completely replaces the default; do not merge it
with or inherit the preceding execution's contract.
- Resolve credentials for the new execution; retained history does not preserve
the original direct key.
- Select session ID from the new request or follow-up prompt's session template.
Do not implicitly inherit it. Applications may explicitly reuse a session ID;
ancestry does not depend on one.
History introduces no additional execution-setting precedence. The follow-up
can change model or response schema without altering the original exchange.
Assemble each continuation's messages in this order:
1. The captured logical history, including its final assistant content.
2. The rendered follow-up prompt definition messages, in definition order.
3. The follow-up request's caller-appended messages, in caller order.
Preserve authored roles and message content; do not move messages based on role.
Continuation adds no history-size limit or automatic truncation. Upstream
context-window or other request rejections follow ordinary generation-error
behavior.
### Capacity And Application Work
Compose completed executions. Finish each turn and release its logical admission
lease before calling application code. Releasing only the active generation
permit is insufficient under the existing [run flow](../internal/runner.md#run-flow).
Each later execution acquires ordinary admission.
The coordinator holds no backend capacity while awaiting callback work.
Callbacks may inspect responses, consult data, wait for application input, or
call the same engine and backend, including when backend capacity is one.
### Ancestry And Per-Run Accounting
Each continuation retains its own originating run ID and exposes its immediate
parent exchange's run ID. Parent metadata describes history provenance; group
membership describes application-selected accounting. Neither implies the other.
A validator can join a continuation's accounting group without extending its
conversation.
Reuse ordinary per-run usage and observations, including attempt, retry, repair,
failure-path, and usage-presence/completeness semantics. Preserve the existing
[observation boundary](../internal/observation.md): arbitrary nested calls are
independent executions. Do not infer membership from a callback's purpose or
context.
If initial run A supplies branches B and C, its usage belongs only to A. B and C
do not inherit that cost. Usage reported for resending captured history in B and
C still belongs to those new runs. Count selected runs once each; never add
terminal totals to the attempt events they already summarize.
## Optional Execution Groups
### Explicit Membership And Observer Composition
Use an observer-compatible collector attached explicitly to selected executions.
Prepared requests retain the selection; membership starts when execution emits
observations. Preparation alone, discarded handles, and failed claims do not
enroll runs through observations.
Coordinator selection covers its continuation runs. Callback-initiated calls
require their own explicit selection. Neither nested context nor a captured
starting exchange automatically enrolls another run.
Support an application's existing diagnostic observer alongside one or more
collectors through a small observer-composition helper. Preserve synchronous
delivery and existing panic behavior. Do not introduce asynchronous queues,
panic recovery, or guaranteed delivery to later observers after an earlier one
panics; interrupted delivery can leave accounting incomplete.
A run may explicitly belong to multiple independent groups. Each group counts
its run identity once. Applications obtain operation-level and overall views by
attaching the corresponding collectors to each selected run. Overlapping totals
are not generally additive; no group hierarchy, inherited membership, or merging
facility is needed.
### Concurrent Snapshots And Lifetime
Provide detached, internally consistent snapshots from a caller-held group while
selected executions and snapshot reads may occur concurrently. Reading or
modifying a snapshot must not change the collector or another snapshot. There
is no mandatory close, seal, or wait lifecycle. Applications own scheduling and
decide when to read or discard the group.
Distinguish observed runs with terminal observations from those without them.
Missing terminal observations mean incomplete accounting; the group does not
infer whether work is active, delivery was interrupted, or another cause applies.
It promises no terminal delivery beyond ordinary observation semantics.
“All observed members have terminal observations” does not mean the application
operation is complete. More runs can join later, and PromptKit cannot know whether
the application selected all relevant calls. The application chooses when a
snapshot represents its final report.
### Summary Contents
The initial summary exposes:
- Member run identities and terminal-observation coverage.
- Existing provider-attempt, transient-retry, and structural-repair counters.
- Available token totals with per-field known/completeness information.
Aggregate one terminal observation per run. These observations already summarize
retries, repairs, and available failure-path usage, including executions returning
no ordinary result. Do not reconstruct totals from attempts.
Keep terminal coverage distinct from usage completeness. A terminal observation
may itself have incomplete usage. Two finished runs with fully reported usage
provide a useful known subtotal, but a third observed run lacking a terminal
event leaves group accounting incomplete. Preserve existing usage semantics and
expose missing coverage separately; unavailable evidence must not become known
zero.
Live attempt-level totals and duration aggregation are omitted from the initial
implementation. Applications can use existing events for progress displays. If
durations are later added, summed execution durations must be distinguished from
elapsed group time. Groups supply execution facts, not monetary cost, application
outcomes, or an application-wide reporting framework. Applications retain domain
mappings, interpretation, and report assembly.
## Consumer Sketches
These are pseudocode, not selected exported names or implemented Go APIs.
Each begins with an already captured exchange and uses ordinary follow-up request
selection. Both omit application-specific error handling, including checks for
failed structural validation before schema-specific decoding.
### Analysis Loop
The application records responses and selects its next question or decides that
its investigation is complete. PromptKit retains history and executes the turns.
```text
analysis = application.start_analysis(initial_exchange)
choose_next(ctx, turn):
if turn.new_result exists:
analysis.record(turn.new_result)
question = application.choose_next_question(analysis)
if question is absent:
return Stop()
return Continue(request(
prompt_id = "research/investigate",
vars = {"question": question}
))
termination, error = coordinate(
ctx, start = initial_exchange,
additional_turns = 3, next = choose_next
)
application.present_analysis(analysis, termination, error)
```
### Audita Branch
Audita starts with an already-assessed candidate and existing findings. Only
Audita interprets acceptance, withdrawal, or assessment unavailability.
```text
branch = audita.branch(candidate, existing_findings)
choose_next(ctx, turn):
if turn.new_result exists:
response = audita.decode_response(turn.new_result)
if audita.is_withdrawal(response):
branch.mark_withdrawn()
return Stop()
branch.update_candidate(response)
assessment = audita.assess(ctx, engine, branch.candidate)
branch.record(assessment)
if audita.should_stop(assessment):
return Stop()
return Continue(request(
prompt_id = branch.configured_follow_up_prompt,
vars = audita.follow_up_variables(branch)
))
termination, error = coordinate(
ctx, start = initial_exchange,
additional_turns = 3, next = choose_next
)
audita.record_branch_outcome(branch, termination, error)
```
The initial callback uses existing findings without repeating batch assessment.
`audita.assess` may call the same backend after the turn's admission lease is
released. Those calls join any group only through their own explicit selection.
Other candidates can start independent branches; Audita owns sibling outcomes.
Both sketches process the final permitted response through the normal callback.
Applications retain their received results; the coordinator's termination value
is not an archive. These examples should remain usable through the same generic
API without manual history assembly or per-turn lifecycle bookkeeping.
## API And Implementation Ownership
The [implementation plan](implementation.md) selects the concrete API and steps
for reaching this target. Both consumer sketches must remain expressible without
manual history assembly or per-turn lifecycle bookkeeping. Execution groups
remain independent of capture and coordination; the plan's ordering does not
introduce a product dependency between them.
## Acceptance Scenarios
Use [Audita's acceptance scenarios](audita-request.md#suggested-delivery-and-acceptance-scenarios)
alongside analysis and branching-conversation examples. Validate each capability
at its public boundary without adding guarantees beyond the selected behavior.
### Capture And Continuation
- Ordinary and prepared opt-in capture uses the actual executed snapshot;
preparation alone produces no exchange.
- Internal structural repair contributes only the full final assistant content
to history. Final validation failure still permits capture; an operational
failure without a completed result does not.
- Independent branches preserve prior history and outcomes, including after
original inputs change or disappear and when continued by another engine.
- Follow-up configuration, credentials, session selection, and complete output
overrides follow ordinary rules without implicit inheritance.
- Details encode generically, expose originating and parent identities, and are
detached from retained state; inspection performs no rendering or external reads.
### Coordination
- Initial callback, zero-turn behavior, last-response delivery, stop versus
exhaustion, and execution-start budget consumption follow the specified rules.
- Cancellation before delivery retains only an already-obtained undelivered
response; callback errors do not replay delivered responses or earlier work.
- Failed structural validation follows normal response delivery, while operational
errors preserve Go error identities and stop progression.
- Same-backend callback calls at capacity one proceed after logical admission
release. Internal retries/repairs do not create extra conversation callbacks.
- Both consumer sketches work without PromptKit interpreting content or owning
application outcomes, branch scheduling, or a response archive.
### Observations And Groups
- Per-run usage and failure diagnostics preserve ordinary presence/completeness
semantics and parent references distinguish turns and branches.
- Explicit ordinary, prepared, coordinator, and callback membership works alongside
an existing observer; unrelated nested calls remain independent.
- Concurrent collection and snapshots preserve detached, consistent results.
More members may join after a snapshot with full terminal coverage.
- Overlapping groups count each run once within each group. Shared initial work,
newly reported replay usage, and attempt/terminal events are not double counted.
- Known subtotals, explicit zeros, incomplete usage, and missing terminal coverage
remain distinguishable, including on failure paths.
- Observer composition preserves synchronous delivery and panic behavior without
stronger terminal-delivery promises.

659
docs/roadmap/implementation.md
View File

@@ -1,659 +0,0 @@
# Conversation Continuation And Execution Groups Implementation Plan
## Objective And Sprint Rules
Implement the accepted [feature roadmap](conversation-continuation.md), using
[Audita's request](audita-request.md) as consumer evidence. The feature roadmap
owns product behavior and boundaries; this plan selects the API and work order.
Existing structural-output repair remains a separate, unchanged capability.
Each numbered stage is one Luna-sized implementation prompt. Execute them in
order during one end-to-end sprint. Intermediate stages need not compile or
constitute independently releasable changes. Do not build compatibility shims or
temporary implementations solely to make a stage green. Carry a concise record
of remaining wiring into the next stage; the final stage must leave the whole
repository coherent and validated. There are no intermediate human-review,
external-publication, provider-access, or dependency-release gates.
Start with [development guidance](../development.md). Follow the applicable
[agent](../policy/agents.md), [architecture](../policy/architecture.md),
[testing](../policy/testing.md), and [documentation](../policy/documentation.md)
policies. Preserve the accepted [configuration boundary](../adr/0001-configuration-boundary.md).
This sprint needs no configuration format, dependency, Go minimum, release
tooling, or licensing changes. It does not authorize tagging or publication.
Use real fast internal collaborators and replace only external boundaries in
tests. Select focused checks during stages and run the complete checks at the
end. Do not duplicate existing retry, repair, schema, or capacity test matrices.
Protect the new interactions with those mechanisms.
## Current Baseline And Integration Points
- The root facade in `engine.go` converts public requests, masks inherited
observation state, invokes the internal engine, maps errors, and finishes
observations after the runner releases admission. Preserve this boundary.
- `internal/usecase/runner.go` has shared resolution and completion preparation
phases. `completePreparationWithStructuredOutput` currently combines rendered
definition messages with appended messages and hashes the resulting prompt.
`executePreparedRun` is shared by ordinary and prepared execution and returns
the final `RawOutput`, including after exhausted structural repair.
- `internal/usecase/prepared_execution.go` retains a single-claim payload,
validation plan, direct credential, retry budget, and observer separately from
durable prepared details. Claims remain bound to the originating execution
owner. Details survive execution and discard.
- `internal/domain/message.go` owns message cloning and cache-control ownership.
Reuse it rather than inventing a second normalization contract.
- `observations.go`, `internal/domain/observation.go`, and
`internal/observation/recorder.go` own value-only events, run IDs, terminal
counters, and presence-aware usage. Terminal observations already include
completed work preceding operational failure. Do not total result usage or
attempt events on top of terminal totals.
- `internal/engine/engine.go` applies namespace views to new caller selections.
Captured history requires no original-engine lookups or namespace rewriting.
- Root conversion, `json.go`, formatting, and contract tests must account for
additions without changing existing timing, credential, or error behavior.
## Selected Public API
Implement these names and shapes in the root `promptkit` package. Code below
specifies additions and complete new types, not replacement definitions for
existing types. Keep all implementation machinery under `internal/`.
### Capture, Continuation, And Ancestry
```go
// Add to RunRequest.
CaptureExchange bool
ContinueFrom *Exchange `json:"-" yaml:"-"`
// Add to RunResult.
Exchange *Exchange `json:"-" yaml:"-"`
ParentRunID string `json:"parent_run_id,omitempty"`
// Add to PreparedRun.
ParentRunID string `json:"parent_run_id,omitempty"`
// Add to ExecutionEvent, following its existing field/tag style.
ParentRunID string
// Exchange has private state only.
type Exchange struct { /* private immutable snapshot reference */ }
type ExchangeDetails struct {
RunID string `json:"run_id"`
ParentRunID string `json:"parent_run_id,omitempty"`
Messages []RenderedMessage `json:"messages"`
}
func (e *Exchange) Details() ExchangeDetails
```
- `Run` with `CaptureExchange: true` returns `result.Exchange`. A non-nil
`ContinueFrom` supplies history and implies capture, regardless of the Boolean.
No separate continuation execution method is necessary.
- `Prepare`, `ValidateRequest`, and `PrepareExecution` accept the same request
fields and assemble the same effective messages. They create no completed
exchange. `PrepareExecution` retains capture selection and ancestry privately
for `RunPrepared`; plain preparation only returns its ordinary metadata.
- Nil `ContinueFrom` means no history. A non-nil, zero-value exchange is not a
captured exchange and fails as `ErrInvalidRequest` before source/model work.
`Details` on a nil or zero-value exchange returns a zero details value,
matching the prepared-handle accessor pattern.
- Details contain the complete logical history ending in the final assistant
content. They are detached and encodable with ordinary encoders. Give the
opaque handle constant `String` and `GoString` representations, following
`PreparedExecution`. Serializing ordinary results does not serialize the
capability or automatically disclose the additional conversation history;
callers explicitly encode `Details()` when desired. Do not add deserialization,
persistence, reconstruction, lookup, or lifecycle APIs.
- `ParentRunID` is the immediate source exchange's originating run ID, or empty
for initial runs. Expose it in prepared details, completed results, exchange
details, and ordinary observation events. Reuse the run ID already assigned to
the execution. It is metadata, not a provider request parameter, session ID,
group ID, or new source of execution settings.
- The follow-up's `PromptID`, definition hash, selected profile/backend, and input
hashes describe the new request. The rendered-prompt hash covers its complete
effective message sequence and selected session, using the existing hash
function. Do not hash capture flags or run identities into content equality.
### Optional Coordinator
```go
type ConversationTurn struct {
Exchange *Exchange
Result *RunResult
RemainingTurns int
}
type CoordinateRequest struct {
Start *Exchange
AdditionalTurns int
Next func(context.Context, ConversationTurn) (*RunRequest, error)
Observer ExecutionObserver
}
type CoordinationOutcome string
const (
CoordinationStopped CoordinationOutcome = "stopped"
CoordinationExhausted CoordinationOutcome = "exhausted"
)
type CoordinationResult struct {
Outcome CoordinationOutcome `json:"outcome,omitempty"`
Undelivered *RunResult `json:"undelivered,omitempty"`
}
func (e *Engine) Coordinate(ctx context.Context, req CoordinateRequest) (CoordinationResult, error)
```
- `Next` returning `(nil, nil)` stops. A non-nil request asks to continue; an
error stops with that error. No separate next-action enum or domain-outcome
taxonomy is needed. A zero request pointed to by a non-nil pointer is still
a continuation request, subject to ordinary validation if execution starts.
- `Start` must hold a captured exchange, `Next` must be non-nil, and
`AdditionalTurns` must be nonnegative; invalid values match `ErrInvalidRequest`.
A nil/uninitialized engine follows the existing `ErrInvalidConfig` convention.
Zero is a useful budget, not a default that expands into extra turns.
- Initial callback state has `Exchange: Start`, `Result: nil`, and the full
remaining budget. Later states contain the newly completed result and exchange
with the remaining start budget, including zero after the last permitted run.
The initial exchange is not presented as a newly delivered response.
- The coordinator copies each selected request value and sets its `ContinueFrom`
to the current exchange. It owns this one branch's progression, so a callback's
`ContinueFrom` does not redirect that branch. The rest of the request uses
ordinary rules. Callers wanting independent branches use `Run` themselves.
- Compose the request's observer first and the coordinator's observer second.
The latter selects all continuation runs, not the starting run or nested
callback calls. Applications explicitly compose additional group observers.
- On error leave `Outcome` empty. Set `Undelivered` only for a completed new
response withheld from callback invocation because cancellation was observed.
That result includes its exchange. A callback that was invoked has received
its response even if it errors or cancels. Return no result archive, latest
delivered response, failure-stage taxonomy, or coordinator accounting totals.
- Callbacks receive caller-owned public values. Retain the coordinator's current
immutable exchange separately, so replacing fields in delivered values cannot
rewrite retained history or the next continuation's ancestry.
### Observer Composition And Execution Groups
```go
func ComposeObservers(observers ...ExecutionObserver) ExecutionObserver
// Zero value ready for use; contains private synchronization/state only.
type ExecutionGroup struct { /* private collector */ }
func (g *ExecutionGroup) Observe(event ExecutionEvent)
func (g *ExecutionGroup) Snapshot() ExecutionGroupSummary
type ExecutionGroupMember struct {
RunID string `json:"run_id"`
TerminalObserved bool `json:"terminal_observed"`
}
type ExecutionGroupSummary struct {
Members []ExecutionGroupMember `json:"members"`
TerminalCoverageComplete bool `json:"terminal_coverage_complete"`
Attempts uint64 `json:"attempts"`
TransientRetries uint64 `json:"transient_retries"`
RepairCalls uint64 `json:"repair_calls"`
Usage ObservedTokenUsage `json:"usage"`
}
```
- Attach a group with `Observer: group.Observe`, or use `ComposeObservers` for
diagnostics and multiple groups. No request group field, implicit context
membership, constructor, registry, close, seal, or wait method is needed.
- Composition copies the supplied function slice, skips nil functions, and
invokes the remainder synchronously in supplied order. With no non-nil
functions return nil. Do not deduplicate functions or recover panics; a panic
prevents delivery to later observers. Closures retain their ordinary identity.
- Groups enroll a run when any ordinary event for that run is observed. The
first terminal event seen for that identity contributes counters and usage
exactly once, including when it is the first observed event. Repeated delivery
must not change totals or regress terminal coverage. Handling contradictory
caller-fabricated events is not a new public validation contract.
- Summary counters and token subtotals use terminal observations only. Pending
runs contribute no invented attempt counts or known zero usage. Members without
terminals remain identifiable. `TerminalCoverageComplete` says only that every
currently observed member has a terminal observation; it says nothing about
future members or application completion.
- For each token field, sum known terminal values; `Known` is the OR of terminal
`Known` flags and `Complete` is the AND of terminal `Complete` flags. These flags
describe received terminal subtotals; missing terminal coverage is separate.
If members exist but none has a terminal, usage is unknown/incomplete. An empty
group has complete terminal coverage and known, complete zero totals. Preserve
explicit provider zero and existing per-run failure-path usage semantics.
- Snapshots are detached and internally consistent under concurrent observation
and reads. There is no promised member ordering; tests compare by identity.
The mutex-bearing group must not be copied after first use, as with ordinary
synchronized Go values. Overlapping group totals are not generally additive.
## Internal Implementation Decisions
### Immutable History And Execution-Owned Messages
Put the internal immutable exchange value and history operations in
`internal/domain/exchange.go`. The public handle privately references this value;
internal packages never import the root facade. Retain only history and the two
run identities, not engine owners, definitions, requests, credentials, validation
plans, results, admission leases, or observer/callback closures.
Use a simple immutable parent-plus-turn representation: a parent exchange,
detached new-turn messages, the final assistant message, and a cached total
message count. Initial exchanges have no parent. New exchanges retain only their
new suffix rather than another flat copy of all ancestors. Flatten iteratively
into one correctly sized, caller/execution-owned destination when preparing a
provider request or returning details; do not recurse through arbitrary depth.
Reuse message/cache-control clone helpers at mutable boundaries. This is an
internal choice, not a public allocation or asymptotic guarantee. Avoid unrelated
history caches, ropes, eviction, limits, or pooling.
Prepare the new prompt normally, then assemble history, new definition messages,
and appended messages in the accepted order before hashing. Captured history
is not normalized or rendered again. Save capture/parent state separately in the
private prepared operation/payload. Prepared metadata contains `ParentRunID` and
the already assembled messages, not the exchange capability or capture settings.
Pass the private capture state through the shared execution function. After
successful final validation processing, capture the new-turn suffix from the
executed snapshot and the exact final `RawOutput`; failed structural validation
is still a completed result. Do not capture internal repair instructions or
intermediate candidates. On an operational error returning no result, create no
exchange. Do not change underlying execution cancellation or partial-result
rules to recover an assistant message.
### Coordinator Placement And Boundary Adapters
Put the loop in `internal/usecase/coordinate.go`, not in root business logic.
Use one small internal function generic over request/result value types, with
typed callback and execution functions and a `*domain.Exchange` current state.
Its internal turn/termination values contain only the fields projected by the
public API. This lets the facade inject ordinary `Engine.Run` and the application
callback without importing facade types internally or converting an entire run
result to domain and back just for delivery. No generic public API is added.
The execution adapter accepts `(context, currentExchange, selectedRequest)` and
returns `(completedResult, nextExchange, error)`. It creates a fresh public
exchange wrapper, copies the request, supplies the current history and composed
observer, and calls `e.Run`. Obtain the next internal exchange before delivering
the result to application code. The callback adapter projects a fresh public
wrapper and the current result into `ConversationTurn`. The shared loop does not
inspect results, interpret output, or access generation/capacity directly.
Use this exact progression:
1. With no completed response pending initially, check context before callback.
2. Invoke `Next` with the current exchange, pending result if any, and remaining
budget. Invocation marks that pending result delivered.
3. Check context again; cancellation wins over the callback's action/error.
Otherwise return a callback error unchanged, stop on nil request, or continue.
4. If a request was selected with no remaining budget, return exhaustion before
converting/preparing that request. Otherwise check context before execution,
consume one start from the budget, and call the execution adapter.
5. Return an execution error unchanged; retain no partial output. On a completed
result, update the private current exchange and repeat from step 1.
At step 1, cancellation with a pending new completed response returns that
single response in `Undelivered`; cancellation before the initial callback
returns none. Callback errors and cancellation observed after invocation return
no undelivered response. Panics propagate naturally without recovery. These
context checks do not forcibly interrupt callbacks or eliminate races.
Calling ordinary `Run` per turn ensures its logical admission and terminal
observation finish before `Next`. Do not wrap the whole coordinator in an
admission lease or an execution recorder. Nested calls remain independent.
### Group Collector Placement And Cost
Implement the collector in `internal/observation/group.go`, using domain events
and summary values. The root group is a thin adapter with private internal state;
root summary/member types are independent public declarations, not aliases.
Store a map keyed by run identity, a terminal-seen bit per member, and aggregate
terminal counters/usage under one mutex. Do not retain full event histories,
provider content, or per-attempt arrays. Event updates use map lookup rather than
rescanning all members. Snapshot copies the map's members and totals under the
lock without invoking callbacks. Observer composition is also internal logic
with thin root conversion; do not round-trip every event through both public and
domain representations simply to fan out callbacks.
## Stage 1 — Implement Immutable Exchange Storage
**Status:** Complete.
**Scope:** internal history value and ownership only.
1. Add the internal exchange representation and construction/flattening helpers
described above. Keep fields private and expose only the internal accessors
needed for run identities, message count, capture, and detached details.
2. Clone only new mutable suffix data when capturing; safely share immutable
parents. Preserve all role, content, empty-content, and cache metadata values.
3. Add focused internal domain tests for initial history, a second turn, two
independent branches, and mutation of source messages/cache control and
returned copies. Include a many-turn flattening case without asserting a
specific private layout or allocation count.
**Validation:** `go test ./internal/domain` when the package is buildable.
These tests own focused history behavior; the final suite verifies its assembly.
## Stage 2 — Add Public Handles And History-Aware Preparation
**Status:** Complete; capture state is carried through execution.
**Scope:** facade request conversion and the shared preparation path; generation
capture is the next stage.
1. Add `Exchange`, `ExchangeDetails`, their accessor and opaque formatting, the
request fields, and the prepared/result/observation ancestry fields selected
above. Add private domain request and prepared-operation capture state.
2. Extend `toDomainRunRequest` to translate a valid handle and capture choice.
Reject non-nil empty handles as invalid requests; do not impose engine binding
on exchanges. Preserve existing appended-message validation and copying.
3. Update completion to prepend captured messages before newly rendered messages
and caller suffixes. Compute hashes only after final assembly, leaving the
follow-up's settings, session, profile selection, and schema rules ordinary.
4. Update `PreparedRun` conversion and JSON behavior for optional `ParentRunID`.
Keep capture/private history references out of returned prepared details.
`ValidateRequest` inherits this through its existing preparation delegation.
5. Extend preparation contract coverage to assert exact order, roles, cache
metadata, hash sensitivity to prior history, and unchanged definition hashes.
Check nil/empty handle behavior and that preparation performs no model work.
**Validation:** focused root preparation/JSON tests and affected use-case tests.
Do not add a separate preparation or validation algorithm for continuation.
## Stage 3 — Capture Ordinary And Prepared Executions
**Status:** Complete.
**Scope:** completed-result creation and prepared payload lifetime.
1. Carry capture/parent state through `preparedOperation` and
`preparedExecutionPayload` to the shared `executePreparedRun` path. Keep it
out of the retained details object so discard can release execution-only
references. Preserve claim ownership, validation-plan freezing, direct-key
clearing, observer identity, and retry budgets.
2. Create the exchange from the request actually executed and exact final response
after the ordinary execution has a result. Initial capture is opt-in;
continuation always captures. Attach it and `ParentRunID` in result conversion.
3. Capture final failed validation/repair exhaustion as well as valid results.
On generation, validation-operation, or repair-generation errors returning no
result, create no exchange and leave the source exchange reusable.
4. Keep `RunResult.Exchange` excluded from ordinary JSON/YAML and preserve existing
result timing marshal/unmarshal behavior. Details are the explicit content
encoding path. The exchange's private state stores no execution target,
credential, result archive, callback, or lease.
5. Add focused tests for default no-capture, ordinary capture, prepared capture,
discarded/failed-claim silence, final repaired content without repair chatter,
failed validation capture, and operational-error absence of a new exchange.
Reuse existing prepared lifecycle and mutating-client fixtures.
**Validation:** targeted root capture, prepared-execution, and repair tests plus
`go test ./internal/usecase` as the touched package becomes coherent.
## Stage 4 — Propagate Ancestry Through Ordinary Observations
**Status:** Complete.
**Scope:** immediate-parent metadata without accounting changes.
1. Add parent metadata to the internal recorder's event construction and the root
event projection. Set it before ordinary start delivery and before a prepared
start after successful claim. Preserve it when selection identities update.
A small internal setter before `Start` is sufficient; do not add context-based
ancestry inheritance or new run IDs.
2. Propagate parent identity through start, attempt, retry-delay, and terminal
events. Initial runs and unrelated nested calls keep an empty parent. Preserve
terminal error classification and public error identities.
3. Cover observed ordinary/prepared continuation, preparation silence, operational
failure after continuation selection, and an independent nested call using
existing observation fixtures. Verify no history/content enters events.
**Validation:** focused observation and prepared-observation contract tests and
`go test ./internal/observation ./internal/generation`.
## Stage 5 — Verify Independent Continuation At The Public Boundary
**Status:** Complete.
**Scope:** assembled continuation behavior before adding coordination.
1. Add an external-package contract test that captures an initial exchange and
executes two branches, then continues one branch again. Assert exact outgoing
messages and immediate ancestry without changing the initial or sibling data.
2. Continue the exchange through another engine, including a scoped view. Select
different follow-up prompt/profile/output settings and session rules, and
verify that neither previous settings nor original direct credentials leak
into the new execution. Use synthetic credentials, not real environment data.
3. Remove or alter original artifact files after initial capture; continuation
still uses captured history. New follow-up inputs retain ordinary reads and
errors. Exercise prepared follow-up freezing separately from exchange reuse.
4. Mutate details and received result fields and use a mutating injected client
to prove ownership. Encode details generically and check exact final assistant
content, including empty/whitespace content when the output contract allows it.
5. Verify overlapping continuation runs can share the source exchange under the
race detector. Compare behavior, not a required scheduling order.
**Validation:** focused root continuation/capture tests under `-race`. This stage
must exercise ordinary shared preparation/execution rather than a mock history
reconstruction implemented by the test itself.
## Stage 6 — Implement The Internal Sequential Coordinator
**Status:** Complete.
**Scope:** the small generic loop and its deterministic state-machine tests.
1. Add the internal loop and turn/termination values described above, injecting
typed callback and run functions. Do not add a generic workflow framework or
a new generation collaborator.
2. Implement initial callback delivery, turn-start consumption, final-response
delivery, explicit stop versus exhaustion, and first-error termination.
3. Implement context checks and the single undelivered-response slot exactly as
specified. Do not prepare an exhausted request or replay delivered work.
4. Unit-test zero turns, stop on last response versus request for another turn,
execution and callback errors preserving `errors.Is`/`errors.As`, cancellation
before initial callback, cancellation withholding a completed result, and
cancellation/error after callback invocation. Use deterministic hooks, not
sleeps. Add one panic propagation check without terminal-delivery promises.
**Validation:** focused `go test ./internal/usecase -run Coordinate` (name tests
accordingly). The loop tests use cheap execution functions; provider behavior
remains covered by integration tests.
## Stage 7 — Expose Coordination Through The Facade
**Status:** Complete; public coordinator tests passed with observer composition.
**Scope:** public types, argument checks, and ordinary-execution adapters.
1. Add the selected coordinator types and `Engine.Coordinate`, with GoDoc for
callback state, budgets, cancellation, errors, and normal outcomes.
2. Validate coordinator arguments without looking up old resources. Project
callback state through fresh public handle wrappers. Preserve current private
exchange state independently of any callback mutation.
3. Supply the run adapter using `Engine.Run` with the receiving engine/view.
Set current history on a value copy; the coordinator does not mutate the
callback's `RunRequest`. Never add enclosing admission or an observer recorder.
4. Wire its observer selection to composition in Stage 9; do not build a second
temporary observer mechanism to make this stage independently complete.
5. Add basic public tests for invalid arguments, zero-budget initial stop,
ordinary continuation, callback error identity, and a callback modifying the
result/handle it received without changing subsequent retained history.
**Validation:** focused coordinator tests when wiring is buildable. Carry the
explicit observer-composition dependency forward, without an intermediate gate.
## Stage 8 — Verify Coordinator Capacity And Cancellation Boundaries
**Status:** Complete; focused race validation passed with observer composition.
**Scope:** the subtle assembled execution lifecycle.
1. Configure one backend with concurrency one and waiting capacity zero. Have the
callback make a same-engine/backend validation run after receiving a turn.
Assert it succeeds, proving the logical admission lease as well as the active
permit was released. Use ordinary admission, not test-only capacity bypasses.
2. Deliver a structurally invalid completed response normally, including one
whose structural repairs were exhausted. The application callback decides
whether to stop or select a new request; PromptKit adds no acceptance meaning.
3. Exercise operational failure stopping one coordinator while a sibling exchange
remains usable. Verify typed provider/capacity errors cross unchanged.
4. Trigger cancellation after a completed run but before callback delivery using
a terminal observer; retain its result/exchange once as undelivered. Separately
cancel inside the callback and assert no replay or undelivered duplicate.
5. Check that internal retry/repair attempts create only one delivery for that
continuation and do not consume extra turns. Reuse existing deterministic
HTTP/repair fixtures, not live providers.
**Validation:** focused root coordination tests under `-race`, with bounded test
contexts as failure guards, not timing-dependent synchronization.
## Stage 9 — Add Observer Composition
**Status:** Complete.
**Scope:** fan-out helper and final coordinator-observer wiring.
1. Implement internal synchronous fan-out over a copied function slice and the
root `ComposeObservers` wrapper. Keep public/domain adapters proportionate;
a small internal generic callback-composition helper can operate directly on
either event value type without repeated conversion.
2. Honor nil filtering, supplied order, no function deduplication, and ordinary
panic propagation. The helper holds no lock while invoking application code.
3. Complete coordinator observer composition, selecting each request observer
before the coordinator observer. Do not change original callback request
values. Callback-initiated calls remain explicitly selected independently.
4. Test nil/empty composition, caller mutation of the original observer slice,
ordered delivery, early panic, and simultaneous independent executions. Do
not promise stronger callback ordering across different runs.
**Validation:** focused composition and coordinator observer tests, including
the root observation-boundary tests.
## Stage 10 — Implement Terminal-Based Group Aggregation
**Status:** Complete.
**Scope:** internal collector, usage semantics, and concurrent snapshots.
1. Add domain summary/member values and the zero-ready internal group collector
under `internal/observation`. Implement map enrollment, terminal deduplication,
incremental terminal counters, and detached snapshots under one mutex.
2. Implement the selected known/completeness and coverage rules for all five
token fields. Do not sum `RunResult.Usage`, reconstruct usage from attempt
events, or convert missing terminals into zero evidence.
3. Cover empty groups, members without terminals, terminal-only enrollment,
duplicate terminal and repeated ordinary events, explicit zeros, wholly
unknown fields, partially known terminal usage, and terminal failures retaining
earlier completed usage. Use small event values rather than an HTTP test for
every arithmetic combination.
4. Concurrently observe independent runs and read snapshots, checking each
snapshot's counts, coverage, and member identities agree. Mutate a returned
snapshot and prove later snapshots are independent. Do not require snapshot
member ordering or introspect locks.
**Validation:** `go test -race ./internal/observation` with focused new group
tests. No background worker, callback invocation, or collector lifecycle is added.
## Stage 11 — Expose Groups And Verify Explicit Membership
**Status:** Complete.
**Scope:** thin public collector adapter and assembled accounting.
1. Add the public group/member/summary types and root event/summary conversions.
A private zero-ready internal group field keeps the public zero value usable.
Document synchronization, snapshot ownership, and no-copy-after-use semantics.
2. Exercise `group.Observe` alone and composed with a diagnostic observer and a
second group. Cover ordinary runs without capture, captured runs, prepared
execution, coordinator continuations, and separately selected callback calls.
3. Assert preparation/discard/failed claim creates no membership. The initial
exchange is not automatically enrolled by coordination; unselected nested
calls remain absent. Parent metadata and group membership remain independent.
4. In one assembled scenario, select initial A and branches B/C, plus a validator
into overlapping groups. Include a built-in transient retry, structural
repair, and a failure after completed usage. Check counters and known subtotals
once per identity, including replay usage reported by the provider for B/C.
5. A composed observer panic may leave an observed member without a terminal.
Verify snapshots remain usable, expose missing coverage, and do not manufacture
completion. Snapshotting inside an observer is safe because group locks are
not held while calling other observers.
**Validation:** focused root group/coordinator/observation tests under `-race`.
Reuse existing reliable-execution fixtures and avoid duplicating their complete
retry/repair matrices.
## Stage 12 — Demonstrate The Consumer Workflows
**Status:** Complete.
**Scope:** practical API use and one maintained offline example.
1. Extend `examples/go-library/observe` and its semantic assertions to demonstrate
captured proposal history, a coordinator follow-up, an application-owned
assessment call, and explicit group selection alongside diagnostic observation.
Keep its deterministic transport and existing retry/usage protections; no paid
calls or real credentials are needed.
2. The example's assessment remains application policy, not a library validator
hook. Use a closure to retain assessed state, and show stop versus exhausted
budget without returning an application verdict from PromptKit.
3. Add concise consumer-guide examples for direct branching and prepared capture,
plus both roadmap sketches translated to the selected callback shape. The
analysis sketch records a new result then asks another question or returns nil.
The Audita-shaped sketch uses existing findings initially, assesses each new
result, and either returns nil or an ordinary follow-up request. Domain-specific
functions remain illustrative caller code.
4. Show group use with ordinary/prepared requests and `CoordinateRequest.Observer`,
including explicit selection for a nested validator and an independent group
summary. Demonstrate checking structural validation before schema-specific
decoding; stop/exhaustion does not imply content acceptance.
**Validation:** `go test ./examples/go-library/...` and affected public example
tests. Review snippets against actual declarations; full copyable setup stays in
the maintained example rather than duplicated in prose.
## Stage 13 — Finalize Contracts, Documentation, And Sprint Validation
**Status:** Complete. Full offline race suite, vet, and diff checks passed with Go 1.27.3.
**Scope:** complete the target and remove sprint-only rough edges.
1. Review every new public declaration and GoDoc against this plan and the feature
roadmap. Confirm ordinary APIs still use existing error, timing, prepared-claim,
credential, namespace, usage, and cancellation behavior. Verify result/details
serialization and constant opaque formatting without introducing a restartable
exchange format or redacting caller-authored message content.
2. Update the canonical consumer guide, `doc.go` where orientation needs it, and
affected internal runner, observation, and component-overview documents.
Document capacity release for conversation callbacks separately from ordinary
per-attempt observer callback constraints. Configuration and integration docs
need only changes necessary to describe the implemented message composition;
ancestry is not an upstream wire field or configuration setting.
3. Remove temporary helpers, placeholder implementations, unused experimental
API choices, and intermediate compile/test failures. Do not remove unrelated
planned/deferred work or publish a release in this sprint. Release-note/version
selection and roadmap retirement can follow during release preparation.
4. With the newest locally available Go toolchain selected as the development
guide prescribes, format changed Go files, then run:
```sh
go test -race ./...
go vet ./...
git diff --check
```
The full suite includes the maintained examples; do not add a redundant
separate build or provider connectivity gate. Check added/changed local
documentation links, using the existing tooling where practical rather than
introducing another release validator.
5. Review the complete diff and Git status for unintended files or credentials.
Confirm no new dependency, workspace override, configuration parser, provider
parameter, or strengthened restriction/guarantee slipped into the work.
Commit completed units with plain-English messages under the agent policy.
**Completion evidence:** the implemented facade supports ordinary/prepared
capture, cross-engine independent continuations, coordinator callbacks after
admission release, cancellation's single undelivered response, and independent
observer-based groups. Public behavior tests cover those boundaries and terminal
usage completeness; the complete offline race suite and vet pass. No product or
API decisions are deferred to an implementation stage.

177
docs/roadmap/remediation.md
View File

@@ -1,177 +0,0 @@
# Conversation Continuation Review Remediation
## Review Scope
Reviewed all 13 stages of the [implementation plan](implementation.md) against
the [accepted target](conversation-continuation.md), implementation, tests, and
documentation. Baseline: `5c135c5`, including the implementation commits after
`5cc3fa0`.
The implementation retains the intended boundaries: immutable parent-linked
history, ordinary preparation and execution for continuations, a separate
sequential coordinator, and explicitly selected observer-based groups. No runtime
correctness defect or inadvertently strengthened restriction was identified.
The initial finding concerns the durable public documentation in Stage 13.
Validation completed with the locally installed Go 1.27.3 toolchain,
`GOTOOLCHAIN=local`, `GOWORK=off`, and a writable temporary build cache:
- `go test -race ./...` passed, including the maintained examples.
- `go vet ./...` passed.
- `git diff --check` passed.
### Algorithmic Follow-Up
A subsequent static audit of all 13 stages, at the same implementation baseline,
identified the two efficiency opportunities below. No additional ordinary
correctness bug or avoidable quadratic-time algorithm was found. No real-world
performance data was collected.
The planned algorithms remain fixed: suffix-only immutable capture, iterative
history flattening, sequential coordination, synchronous observer delivery, and
map-based incremental group aggregation. The findings address extra allocation
and copying around those choices, not replacements for them. Complexity statements
describe implementation costs, not new public performance guarantees.
For orientation, capture is linear in the new turn's message count; flattening is
linear in retained message count; coordinator bookkeeping is constant per turn,
excluding application callbacks and ordinary execution. Group updates are
expected constant time per event for fixed-length run identities, with linear
membership storage and linear full snapshots. An observer dispatch is linear in
its selected callbacks. These implementations do not rescan all prior turns or
group members on each incremental update.
## 1. Complete The Public Exchange Capture And Continuation Contract
**Priority:** P2. **Status:** Complete. **Plan stage:** 13.
**Completion:** Expanded the public request, exchange, details, and prepared-handle
GoDoc and linked the consumer walkthrough to those owners. Checked comments
against capture, preparation, and repair behavior; local links and diff verified.
**Locations:** [request fields](../../types.go),
[exchange declarations and accessor](../../exchange.go), and
[consumer guidance](../consumers/pkg-promptkit.md#capture-and-continue-a-conversation).
The `CaptureExchange` and `ContinueFrom` GoDoc currently provides only a short
description of opt-in capture, invalid handles, and fresh follow-up settings.
The `Exchange` and `Details` comments describe reuse and detached history, but do
not define its exact contents or explain that inspection uses only retained
state. Important selected semantics remain documented only in temporary roadmaps
or prose/internal guidance:
- History contains the effective original request and exact final assistant
content, preserving roles and cache metadata; intermediate structural-repair
candidates and corrective instructions are excluded.
- Continuation assembles captured history, rendered follow-up definition
messages, and caller-appended messages in that order, without re-rendering or
reading the original inputs. It adds no history-size limit or truncation.
- `PrepareExecution` retains capture selection for execution of its snapshot;
preparation and concrete-request validation alone produce no completed exchange.
- `Details` reads the retained snapshot without rendering, filesystem reads, or
provider calls. Its run identities identify the originating execution and
immediate parent execution.
These behaviors are implemented. The gap is that the exported declarations and
GoDoc do not yet own their complete contract as required by the documentation
policy and the final implementation stage. Retiring the roadmap would otherwise
leave consumers and future maintainers without a complete canonical reference.
**Required remediation:** Document these existing semantics at the appropriate
request fields, exchange type/details, and prepared-execution declarations, using
GoDoc cross-references instead of repeating the same contract across declarations.
Link the new consumer walkthrough to those public owners. Preserve the selected
scope: do not add persistence, limits, stronger serialization guarantees, or
private storage/allocation promises while documenting the behavior.
**Verification:** Check the comments against the existing capture, continuation,
prepared-execution, ownership, and repair tests and implementation. Validate the
affected documentation links and diff. This is documentation remediation; it
does not require changing runtime behavior or duplicating existing tests.
## 2. Avoid Intermediate History Copies During Assembly And Inspection
**Priority:** P3. **Status:** Complete. **Plan stages:** 2 and 5.
**Completion:** Preparation now fills one history-plus-suffix buffer. Inspection
projects indexed history directly into public messages, cloning exposed cache
metadata once. The parent-linked representation and public types are unchanged.
Focused race-enabled history, continuation, preparation, and ownership tests passed.
**Locations:** `completePreparationWithStructuredOutput` in
[the runner](../../internal/usecase/runner.go), `Exchange.Messages` in
[history storage](../../internal/domain/exchange.go), and
[`Exchange.Details`](../../exchange.go) with `fromDomainRenderedMessages` in
[facade conversion](../../convert.go).
Let H be the captured history's message count and S the newly rendered and
caller-appended message count. Continuation preparation currently:
1. allocates and copies S messages into the new suffix;
2. materializes H history messages into a detached intermediate slice; and
3. allocates the final H + S slice and copies both intermediate slices again.
Thus assembly copies 2H + 2S message entries, rather than the H + S needed to
populate the final execution-owned buffer. Each pass also clones non-nil
cache-control structs. The public `Details` path similarly materializes a
detached domain history and then traverses it again to allocate public messages
and clone cache controls for a second time. Strings share immutable content;
these are message-header and cache-struct copies, not duplicate text-byte buffers.
**Complexity:** Current and proposed assembly are both Θ(H + S) in time and
auxiliary space; details remain Θ(H). The opportunity removes an extra linear
pass and temporary allocations, not an asymptotic quadratic defect. Successive
full-history requests still entail work proportional to the sum of their history
lengths. This audit does not propose truncation or a different
conversation protocol.
**Potential refactoring:** Add an internal fill/flatten helper that writes history
and the new message parts directly into one correctly sized final domain slice.
For inspection, consider an internal indexed traversal/projection helper that
fills the public destination without first constructing a complete domain slice.
Keep the immutable parent-linked representation, separate public types, and
detachment of every exposed mutable cache-control value. Prioritize the simple
assembly change; avoid adding a general projection framework solely to eliminate
the inspection intermediate.
**Verification:** Reuse history ordering, cache-control mutation, branching,
prepared-execution, and mutating-client tests. Review the copy/allocation paths
statically. Do not add public allocation guarantees or exact-allocation tests.
## 3. Size Observer Storage For Selected Callbacks
**Priority:** P3. **Status:** Complete. **Plan stage:** 9.
**Completion:** Observer composition counts selected callbacks before allocating,
sizes retained storage to that count, and handles zero/single selections without
a fan-out closure. Sparse-selection ownership and duplicate-delivery tests pass.
The complete offline race suite, maintained examples, vet, and diff checks passed
with Go 1.27.3 after all remediations.
**Location:** [`Compose`](../../internal/observation/compose.go).
`Compose` allocates `selected` with capacity equal to every supplied observer
slot before filtering nil callbacks. The returned closure retains that backing
array even when only a small subset of slots survives. An all-nil input also
allocates the full temporary array before returning nil.
Let K be the number of supplied slots and A the number of non-nil callbacks,
including intentionally repeated callbacks. Construction is Θ(K) time with
Θ(K) allocated storage, and a nonempty composition retains Θ(K) storage.
Dispatch is already Θ(A) and does not need changing. Selected-function storage
can instead be Θ(A), with constant storage for A = 0, while preserving Θ(K)
construction time.
**Potential refactoring:** Count non-nil callbacks first, then allocate exactly A
slots and fill them in order. This trades a second linear scan for avoiding
storage proportional to disabled slots. Alternatively, append into an initially
nil slice for a simpler one-pass filter with amortized Θ(K) construction and
Θ(A) retained storage, accepting growth allocations for dense inputs. A
zero/single-callback fast path can also avoid an unnecessary fan-out closure.
The usual coordinator supplies only two slots, so its benefit is small; this is
a low-priority general-helper scalability opportunity, not a release blocker.
**Verification:** Preserve copied function values, nil skipping, duplicate
delivery, supplied order, and ordinary panic propagation. Existing composition
and prepared-observer ownership tests protect those contracts; no benchmark or
new performance gate is required.