promptkit/docs/roadmap/implementation.md

# Reliable Execution Implementation Plan

## Scope And Working Rules

Implement the target in [reliable-execution.md](reliable-execution.md). This plan
is for sequential execution by a coding agent, one numbered stage per prompt.
Stages intentionally separate transport refactoring, retry behavior, lifecycle
instrumentation, public integration, and consumer evidence. Do not merge stages
merely because their production changes look small.

Read all [repository policies](../development.md#initial-orientation), especially
the [agent](../policy/agents.md), [architecture](../policy/architecture.md),
[documentation](../policy/documentation.md), [testing](../policy/testing.md), and
[release](../policy/release.md) policies. Preserve unrelated changes. Commit each
completed, validated stage with a plain-English message and report remaining
work. Do not push, tag, publish, rewrite history, or change dependencies. Licensing
is complete. No new live-provider checks or release gates are needed.

Use focused tests at each stage; run race tests for concurrency changes. Keep
intermediate commits building, and expose public controls only when their
behavior works. Update owning documentation alongside implemented behavior,
not ahead of it. Exact internal helper names may change without approval; the
consumer semantics and explicit choices below must not drift silently.

## Decisions Shared By All Stages

### Public Controls And Prepared Execution

- Add `Config.TransientRetries *int` and `RunRequest.TransientRetries *int`.
  Nil selects two additional attempts at construction or inherits the engine
  value on a request. Zero disables retries. Reject negative values with
  `ErrInvalidConfig` or `ErrInvalidRequest`, respectively. Do not invent a small
  maximum; count actual attempts with uint64 and convert the nonnegative budget
  before adding the initial attempt, avoiding int overflow at MaxInt. Use
  overflow-safe duration arithmetic. Copy pointed values.
- Validate request retry values in `Prepare`, `PrepareExecution`, and `Run`.
  `Prepare` performs no retries; `PrepareExecution` freezes the effective retry
  count privately. Do not add retry settings to profile YAML, execution targets,
  provider payloads, prepared metadata, or prompt hashes.
- Add `RunRequest.Observer ExecutionObserver`, with
  `type ExecutionObserver func(ExecutionEvent)`. It has no error return and is
  excluded from JSON. Ordinary `Run` observes that invocation. Successful
  `PrepareExecution` retains the function in its private payload for the owning
  `RunPrepared`; `Prepare` and `PrepareExecution` do not emit execution events.
  No new `RunPrepared` parameters or parallel execution API are introduced.
- `RunPrepared` observation starts only after successful atomic claim. Invalid,
  foreign, reused, discarded, or nil handles cannot supply an execution observer
  and emit no events. Preparation failures return their existing errors without
  execution events. Ordinary `Run` does observe pre-generation errors, including
  facade conversion errors and a nil engine if an observer was supplied.
- Clear retained observers and private execution settings on claim completion
  or discard along with the existing private payload. Never invoke observers
  while holding the prepared-handle mutex. A callback is retained by identity;
  its closure is not cloned. Document caller synchronization responsibilities.
- Injected `LLMClient` values are never automatically retried, even when configured
  retry values are positive. Values are still validated. A custom HTTP client
  used by the built-in implementation does receive built-in retries; explicit
  zero disables them when a transport already owns retries.

### Retry Semantics

Only HTTP 429, 500, 502, 503, and 504 permit a retry. Transport failures, timeout,
malformed envelopes, invalid requests, and nil collaborator responses do not.
Content repair remains separate. Each repair generation gets the same configured
transient budget as initial generation, never a budget shared across the run.

For retry ordinal k starting at 1, use equal jitter in [base/2, base], where
base is min(500 milliseconds × 2^(k-1), 8 seconds). Cap before doubling to avoid
overflow. These are documented defaults, not public clock/backoff options.
Internal tests can supply a clock, jitter function, and cancellable wait function
through private constructors; do not change package globals or sleep in tests.

Parse `Retry-After` as unsigned decimal seconds or an HTTP date via
`http.ParseTime`, following [RFC 9110 section 10.2.3](https://www.rfc-editor.org/rfc/rfc9110.html#section-10.2.3).
Use the first header value; trim whitespace; negative, fractional, or malformed
values use local backoff. Past dates supply zero additional minimum. Measure a
valid delay from response receipt, not from the end of body decoding; wait until
the later of that not-before time and the local backoff deadline. Never apply
the 8-second local cap to the provider minimum. Saturate unrepresentably large
valid delays safely rather than wrapping or treating them as permission to
retry early. Cancellation interrupts the wait; it must not busy-loop.

Check the caller context before waiting, before permit acquisition, and before
dispatch. Caller cancellation/deadline wins when observed at those boundaries.
Do not proactively reject long provider delays; wait cancellably. A context
without a deadline can wait a long time, which must be documented.

Existing HTTP-client and target generation timeouts apply separately to each
actual attempt after capacity admission. Retry waiting and capacity waiting use
the original caller context, not a previously canceled attempt context. No
new overall timeout is imposed. Never reset the caller's deadline.

Freeze built-in request payload, URL, and resolved authentication once per
generation sequence, before its first attempt; create a fresh HTTP request/body
reader and timeout context for every attempt. An absent optional key must remain
absent if the environment changes during retries. Repair is a new generation
sequence and may resolve the environment again, consistent with existing calls.
Do not freeze environment values at `PrepareExecution` time.

When R is zero, return the ordinary first failure, not retry exhaustion. When
R is positive and all R+1 attempts return eligible statuses, return exhaustion.
If a later failure is noneligible or the context stops the sequence, return that
failure rather than exhaustion. Count attempts actually dispatched, not waits
scheduled. For one run, the bound is (R+1)×(P+1), where P is the repair budget;
document the formula but do not compute it using overflow-prone `int` products.

### Error Contract

Add public `ErrTransport`, `ErrMalformedResponse`, and `ErrRetriesExhausted`.
Transport/malformed failures during generation also retain `ErrLLMGenerate`.
Preserve standard context identities and original transport causes for
`errors.Is`/`errors.As`. Do not expose new internal concrete error types as the
consumer contract or stringify sensitive transport errors while mapping them.

Add immutable `RetryExhaustedError` with `Attempts() uint64`, `Unwrap() error`,
`Is(error) bool` matching `ErrRetriesExhausted`, and redacted `Error`/`GoString`.
Map its cause recursively so `errors.As` still finds the existing public
`GenerationError` and status/detail accessors. Nil receivers and zero values
must be safe, consistent with existing typed errors. No stable JSON contract.
Do not let the current provider-error special case in `mapPublicError` discard
the exhaustion wrapper or outer validation/generation identities.

Injected errors retain their identities; do not guess transport or HTTP
classifications by text or structural resemblance. An injected client can use
the public sentinels deliberately. A nil successful collaborator response is
malformed generation. Completed structural rejection remains a normal result
with failed validation; operational validation errors remain errors with no
partial `RunResult`.

### Observation Values And Accounting

Define public values in `observations.go`, with separate internal equivalents
and facade conversion, not aliases. They have no stable serialized contract;
consumers translate them into application-owned records.

`ExecutionEvent` contains:

- `Kind ExecutionEventKind`: constants `ExecutionStarted`, `ExecutionAttempt`,
  `ExecutionRetryDelay`, `ExecutionFinished` (string values `started`, `attempt`,
  `retry_delay`, `finished`).
- `RunID string`, `Time time.Time`, and `PromptID`, `PromptVersion`, `ProfileID`,
  `BackendID`, `Model` string fields. Use the same RunID as a successful result.
  Start events for ordinary runs have no resolved identities; prepared runs
  may supply their already-resolved snapshot. Enrich later events as resolution
  succeeds. Do not label requested but unresolved values as selected values.
- `Generation uint64` (1 initial, 2 first repair), `Attempt uint64` (1-based within a
  generation), and `Dispatched bool`. Non-attempt events use zero indexes unless
  describing the retry delay after a specific failed attempt.
- `CapacityWait`, `GenerationDuration`, `RetryDelay`, `Duration time.Duration`.
- `Outcome ExecutionOutcome`: `success`, `invalid_output`, `error`, or empty
  when not applicable; corresponding constants use the `ExecutionOutcome` prefix.
- `Failure ExecutionFailureKind`, `StatusCode int`, `RetriesExhausted bool`.
  Failure constants use the `ExecutionFailure` prefix; `ExecutionFailureNone`
  is the zero value empty string. Other values are `canceled`, `deadline`,
  `transport`, `provider_status`,
  `malformed_response`, `capacity`, `invalid_config`, `invalid_request`,
  `prompt_not_found`, `profile_not_found`, `prompt_load`, `profile_load`,
  `artifact_load`, `prompt_render`, `validation`, `generation`, `other`.
  Missing credentials/profile selection use invalid_request. Classify context
  identities first, then specific causes before broad operation categories.
  Exhaustion is a separate flag, not a replacement for provider_status.
- `Usage ObservedTokenUsage` and terminal `Attempts`, `TransientRetries`,
  `RepairCalls uint64`. Nonterminal aggregate counters are zero.

`ObservedTokenUsage` has the existing five token field names, each an
`ObservedTokenCount{Value int; Known bool; Complete bool}`. An attempt field is
known/complete exactly when the provider supplied that field as a non-null
integer. An aggregate Value sums known contributions, Known means at least one
known contribution, and Complete means every dispatched attempt reported that
field. For zero dispatched attempts use known complete zero totals. Preserve
reported values without deriving totals or reconciling fields. Unknown fields
have Value zero. Counts on non-dispatched attempts are unknown and excluded from
usage aggregation. This distinguishes partial accounting from complete zero.

Add optional `GenerateResponse.UsagePresence *TokenUsagePresence` with JSON tag
`usage_presence,omitempty`; `TokenUsagePresence` contains five bool fields with
the corresponding snake_case JSON names. For nil from legacy injected clients,
nonzero fields are known and zero fields are unknown: the old interface cannot
distinguish omitted zero from measured zero. An explicit presence value permits
known zeros and governs all five fields. Copy it on conversion. Existing result
accounting still sums all Usage values regardless of this observation metadata;
`TokenUsage` and existing `RunResult` JSON remain unchanged. New
GenerateResponse JSON round-trips the additive field and omits nil. The built-in
adapter always supplies explicit presence; absent/null wire counts are unknown.

Emit one attempt-completion event per acquisition attempt, including interrupted
capacity waiting with Dispatched=false. GenerationDuration covers dispatch
through response decoding/closure, excludes capacity/backoff/callback time.
CapacityWait covers acquiring a permit, and is zero for the unlimited path.
Release the active permit before invoking observers. Dispatched means invoking
the prepared HTTP attempt or injected Generate, not a proven network packet or
billable request; hidden HTTP redirects/transport retries are not observable.

Emit retry-delay completion after each actual wait, including interrupted waits,
with elapsed RetryDelay and the failed attempt indexes. No delay event when no
wait starts. Emit terminal totals once on each normal return, including errors,
after admission release. Attempts counts dispatched calls; TransientRetries
counts dispatched attempts numbered above 1; RepairCalls counts generations
above 1 with at least one dispatched attempt. Do not count scheduled work.
Terminal phase-duration fields sum their respective measured phases; Duration
is wall-clock operation time
and can additionally include preparation, validation, callbacks, and other work.
Do not add terminal usage to attempt usage. Attempt success means generation
succeeded, not that later output validation accepted it.

Synchronous callbacks are ordered per operation, may overlap across operations,
and cannot return errors. Event values contain no mutable containers or error
objects. Never include endpoint URLs, headers, keys, session values, bodies,
validation diagnostics, or provider text. Metadata identifiers can still be
sensitive. Callback panics propagate; defer cleanup before first callback after
resource acquisition. Do not promise terminal events after panics or process
termination and do not invoke further callbacks while unwinding a panic.
Callbacks must return for the operation to progress; cancellation cannot forcibly
interrupt consumer code. Warn against blocking on work requiring the same run's
admission slot. No asynchronous delivery or durable history.

## Stage 1 — Expose Missing Generation Failure Categories

**Completed:** 2026-09-23.

Read `errors.go`, `generation_error.go`, `llm_adapter.go`, the LLM error types,
and public error-boundary tests. Add public transport/malformed sentinels and
map current built-in errors while preserving cause identity and redacted text.
Classify actual request execution failures as transport, not unrelated request
encoding failures merely sharing the old broad ErrRequestFailed sentinel.
Normalize nil successful injected responses into the malformed category. Do not
add retries or observers yet. Retain existing provider-status and validation
behavior. Add the error GoDoc and a short consumer-guide classification update.

Verify public tests for transport causes, caller cancellation/deadline, malformed
framing/choices, nil response, provider detail redaction, and injected arbitrary
errors. Reuse current fixtures; avoid duplicating the entire transport matrix.
Run `go test ./...`. Commit when mappings and existing tests pass.

## Stage 2 — Preserve Usage Presence Through The Adapter

**Completed:** 2026-09-23.

Read wire response decoding, domain/public response types, facade conversion,
and JSON contract tests. Add the presence types and optional public response
field above. Convert wire usage counts to pointer-aware decoding, retaining
existing integer values and response validation behavior. Explicit null and
omission are unknown; numeric zero is known. Invalid numeric types retain current
malformed-response handling. Apply the nil-presence fallback above only to
observation metadata, never to existing successful-result accounting.

Test absent usage, partial nested usage, nulls, zero, and all five reported
fields at the decoder boundary. Test public optional-field JSON round trips and
defensive copying once at the facade. Existing successful RunResult totals and
JSON must be unchanged. Document injected-client presence semantics in GoDoc.
No observation API yet. Validate affected packages and commit.

## Stage 3 — Separate Frozen HTTP Preparation From One Attempt

Refactor `internal/llm/OpenAICompatibleClient` without changing its one-shot
`Generate` behavior. Add an internal `PrepareGeneration` operation that validates
and encodes one request, resolves credentials once, and returns private frozen
state with an `Execute(ctx)` method. It creates a fresh body/request/deadline
on every Execute and closes every response before returning.

Return an internal attempt outcome containing response, error, HTTP status,
usage plus presence, and parsed provider not-before timing. Preserve known usage
from a strictly decoded successful envelope even if missing choices/content
subsequently make it malformed; do not salvage data from invalid framing or
non-success error bodies. Preserve current body-size limits and redaction.
The existing Generate method delegates once and returns no partial response on
error. After failed body reading/decoding, retain an observed attempt-context
cancellation/deadline instead of hiding it behind malformed-response formatting.
Timing/header data remains internal, not provider-error strings.

Implement Retry-After parsing here, but no sleeping or retry loop. Test frozen
payload/authentication across repeated Execute calls, including an initially
absent optional key; independent per-attempt timeout contexts; body closure;
usage on malformed choices; and header parsing with an explicit receipt time.
Use separate Execute contexts to verify a canceled attempt does not poison the
next call. All existing wire tests must pass. Commit this refactoring separately.

## Stage 4 — Implement The Internal Retry And Capacity Coordinator

Create `internal/generation`, depending on domain, llm, and capacity. It implements
the existing `llm.Client` consumed by the runner and repairer. For built-in
generation it prepares once through Stage 3, then loops over capacity-controlled
Execute calls. For injected generation it dispatches once. Constructor arguments
explicitly select the path; do not infer retry eligibility from arbitrary errors.

Expose an internal Manager active-permit acquisition method returning an
idempotent release function. Reuse pool logic; never build another queue. Keep
the current capacity client as a delegating one-shot wrapper until engine wiring
moves, sharing that acquisition primitive. The coordinator defers permit release
inside each attempt; no permit survives a wait or callback. Admission remains
runner-owned. Invalid frozen-request preparation dispatches nothing.

Implement the exact delay/context/count semantics above with private test seams.
Introduce an internal exhaustion wrapper retaining the cause and attempt count.
The coordinator accepts a resolved retry count on the internal GenerateRequest;
it has no public API or engine behavior change yet. Repair propagation arrives
in Stage 5. Leave observation hooks for later stages rather than adding no-op
public callbacks now.

Tests own the retry status table, R=0, exhaustion, a noneligible later failure,
context interruption, provider minimum versus jitter, overflow-safe backoff,
frozen absent/present credentials, and independent attempt deadlines. A blocked
backoff must allow another call to acquire the only active permit. Cancellation
and panic must not leak permits. Run focused race tests and commit.

## Stage 5 — Wire Retry Configuration Through Both Execution Paths

Add public/internal configuration and request fields. Thread the copied effective
retry count through ordinary resolution, private prepared payloads,
`executePreparedRun`, `newGenerationRequest`, and `RepairRequest`. Do not add it
to public GenerateRequest or provider settings: injected clients own their policy.
Keep one shared resolution helper for default/override/negative validation.

In `internal/engine.New`, assemble the generation coordinator around the selected
built-in or injected client and shared manager. Do not wrap it again in
capacity.NewClient: that would hold a permit over waits or deadlock reacquisition.
Initial generation and default repairer must receive the same coordinator.
Remove obsolete capacity-wrapper machinery/tests only when its distinct behavior
is covered by the coordinator and manager tests; do not retain two live schedulers.

Add the public exhaustion type and recursive error mapping. Update old tests
expecting one eligible HTTP failure to set explicit zero where their purpose is
not retry behavior, rather than making unrelated tests wait through backoff.
Document the newly active controls, default two retries, and per-attempt timeout
semantics now. Add public tests for request/engine precedence, copied pointers,
prepared freezing, repair budget multiplication, exhaustion cause access, and
injected-client single dispatch. Run full tests and focused race tests; commit.

## Stage 6 — Build The Internal Observation Recorder

Create `internal/observation` for invocation-local counters, presence-aware
aggregation, and synchronous dispatch. Internal event/value counterparts and
the callback function type belong in domain so domain requests/payloads can
carry them without importing the recorder package. The recorder depends only
on domain/standard library, not provider or root types. No global map, history
slice, mutex held across callbacks, goroutine, or background queue is needed.
Keep outcome classification at owning boundaries; recorder consumes normalized
failure metadata, never raw error strings.

Move/reuse run-ID generation here so observation and successful results have one
identity owner. Provide start/update/attempt/delay/finish operations and a private
context key to pass the invocation-local recorder through the existing internal
client interfaces. Context propagation is solely internal plumbing, not public
control policy. A nil observer avoids event delivery, but counts needed for errors
and existing results must still work. If run-ID creation fails, report an error
terminal event with an empty RunID rather than invent a valid identifier.

Test phase sums, dispatched-only counters, partial/unknown/zero usage, no double
counting, callback order, retained-value independence, and panic propagation.
Do not promise event delivery across panics. There is no public observer or runner
wiring in this stage. Update the internal inventory when the package lands.

## Stage 7 — Instrument Attempts And Retry Delays

Connect the coordinator to the internal recorder through context. Record measured
capacity wait even on interrupted acquisition, actual dispatch and outcome, usage
from Stage 3, and completed/canceled backoff. Release the active permit before
callback delivery. Collect injected response usage using presence from Stage 2;
arbitrary injected failures have unknown usage. Error response bodies are not
mined for usage.

Do not add an event for request freezing failure: the terminal operation will
report it without pretending a dispatch happened. A completed attempt callback
that cancels context must prevent another attempt. Measure phase end before its
callback so observer work is not reported as provider time. Feed normalized HTTP
and context classifications without exposing sensitive strings.

Tests cover retry then success, exhaustion, interrupted wait, canceled capacity,
success then later repair failure (using internal recorder plumbing), legacy
injected usage and explicit unknown fields, and callbacks outside permits/locks.
Use synchronization rather than elapsed-time thresholds. Run race tests; commit.

## Stage 8 — Add Ordinary-Run Lifecycle And Public Observation API

Add public observation types/conversion and RunRequest.Observer. The facade adapts
the function and delegates bookkeeping to the internal recorder. Start ordinary
Run observation before request conversion and nil-engine rejection; finish after
public error mapping so the terminal classification matches consumer errors.
Use the recorder's RunID/start in the runner instead of generating another ID.
Without an observer preserve existing Result timing conventions; with an observer
its Duration includes the facade boundary and can be slightly longer.

Enrich resolved identities at the runner's resolution boundary, before admission;
do not copy unresolved input strings into selected fields. Generation indexes are
passed through initial and repair paths. Ensure final totals include completed
generation usage before validation or repair errors. Emit finished only on normal
return, after all admission cleanup; do not use a defer that calls user callbacks
during panic unwinding. Keep `Run` signatures and no-partial-result behavior.

Tests cover source/conversion failure, nil engine, admission rejection, validation
operational failure, valid/invalid/skipped validation, success followed by repair
failure, matching RunID, redaction, and observer panic cleanup. Verify callback
mutation/retention cannot alter execution state. Update public GoDoc and consumer
guidance for ordinary observations only; run root and internal race tests; commit.

## Stage 9 — Observe Prepared Execution Without Weakening Its Lifecycle

Retain the adapted observer with the effective retries in the private prepared
payload. Claim atomically before starting observation, release the claim mutex,
install payload cleanup, then emit started. Preparation emits nothing; successful
claim followed by credential failure, capacity rejection, or cancellation still
emits finished on a normal return. Invalid/reused/foreign handles emit nothing
and must neither invoke nor discard another execution's callback.

Use the same recorder, identity enrichment, failure normalization, and terminal
accounting as ordinary execution. Keep callback work outside handle and manager
locks. Discard clears observer references without calling them. Concurrent calls
using one handle result in one executing observation sequence, not duplicate runs.

Add focused prepared public tests for frozen retry choice/observer, discarded and
foreign handles, claim races, no preparation events, later failures with usage,
and panic cleanup. Preserve existing preparation/result timing distinctions.
Document the prepared observer lifetime and exclusions. Run race tests; commit.

## Stage 10 — Verify Cross-Cutting Public Contracts

Review assembled behavior through the public facade using controlled HTTP
transports and small synthetic fixtures. Add only missing integration evidence:

- Two profiles sharing one finite backend coordinate initial, retry, and repair
  traffic. Backoff frees the active slot but not logical admission. Different
  backend pools remain independent; unlimited/endpoint-only paths still work.
- A sequence of transient failure, successful invalid output, repair transient
  failure, and successful valid output produces consistent counts, known partial
  usage, result usage, events, and error identities. Exercise exhaustion and
  cancellation as focused variants, not a Cartesian product.
- Frozen request/authentication and prepared source immutability survive retries.
  Noneligible errors never replay. Injected client attempts remain opaque.
- Safe default error/event formatting and JSON never disclose credential markers,
  request bodies, provider message markers, or retained callback internals.

Do not require fixed callback timing, random samples, exact error wording, or
internal helper-call sequences. Fix interaction defects in their owning layer.
Run `go test -race -vet=off ./...` and `go vet ./...`; commit the verified scope.

## Stage 11 — Add The Offline Proposal And Assessment Example

Add `examples/go-library/observe` with embedded prompt/schema assets, two
application-owned profiles sharing a finite custom backend, and one engine.
Use a controlled RoundTripper under the built-in client: return one eligible
status before a valid structured proposal, then a structurally valid assessment
that the application can accept/reject independently. Collect metadata events
and print a compact structured summary, including actual attempt/retry counts
and assessment outcome. Do not log prompt/response bodies or provider detail.
Accept the real short default backoff in this executable demonstration; unit
tests still use controlled waiting. Do not add public timing controls for it.

Keep cancellation/exhaustion/schema-invalid branch coverage in focused tests
from earlier stages, not a large example switchboard. Add this example once to
the existing release-check script with semantic output assertions; do not run
a redundant second integration suite there. Update the development guide,
testing policy's example inventory, and internal overview accordingly. Run the
example and its relevant tests; commit.

## Stage 12 — Complete Documentation And Final Validation

Reconcile GoDoc, consumer error/observer/retry guidance, provider integration
docs, and internal runner/LLM/capacity/generation/observation documentation with
the implemented design. Remove claims that no retries exist or that usage can
only be learned on success. Clearly distinguish observations from complete cost
accounting and explain nil legacy injected usage presence.

Document single-attempt migration via explicit zero, multiplication with repair
and application semantic retries, provider-delay/caller-deadline behavior,
callback lifetime/concurrency/panics, and no-partial-result compatibility. Do not
copy the same field reference into multiple documents. Link the new example.
Update Audita's wishlist status to implemented where verified. Do not invent a
release version or tag; versioned release notes and completed-roadmap retirement
belong to the later authorized release-preparation task.

Run `./scripts/release-check.sh` once for the final implementation, inspect the
diff for accidental contracts, dependencies, or artifacts, and verify no public
signature exposes internal types. Record results and any real remaining gaps.
Commit the final coherent documentation/integration work; report readiness for
review without publishing or waiting for CI.

## Open Questions

None requiring a maintainer decision remain. The approved roadmap policies and
the explicit API, accounting, timing, and lifecycle choices above are the basis
for this implementation. If implementation reveals a conflict with an existing
explicit contract, report the concrete conflict rather than silently adding a
restriction or weakening that contract.