Reconcile reliable execution documentation

This commit is contained in:
2026-09-23 17:29:12 +00:00
parent 1cc754c29e
commit cef8524b31
14 changed files with 152 additions and 48 deletions

View File

@@ -2,16 +2,16 @@
**Last updated:** 2026-09-23
**Status:** Proposed consumer requests, based on a review of Promptkit v0.10.0.
These are not approved features, implementation plans, delivery commitments, or
claims about current behavior. Priorities reflect Audita's prospective adoption;
Promptkit should evaluate them against its broader consumer needs.
**Status:** Requests 1–6 are implemented in source but not yet released.
These requests originated from a review of Promptkit v0.10.0. They retain
Audita's consumer context and priorities, not a current API specification or
a delivery commitment for Audita adoption.
Requests 1–5 are now incorporated into the active
[reliable-execution roadmap](reliable-execution.md), which owns the selected
Promptkit-native scope and remaining decisions. The requests below retain the
consumer context, not a separate implementation specification. Request 6 is
implemented in source as noted below.
Requests 1–5 were resolved through the
[reliable-execution roadmap](reliable-execution.md) and its
[implementation plan](implementation.md). The [consumer guide](../consumers/pkg-promptkit.md)
and public GoDoc own current behavior. No release or Audita adoption is
claimed here.
## Consumer Context
@@ -34,6 +34,9 @@ consumer outcomes only; API shape and implementation are left to Promptkit.
## 1. Transient-Failure Retries
**Implemented in source; not yet released.** The built-in HTTP path owns the
selected transient-status policy; injected clients keep their own policy.
**Priority: High.** Audita should not need its own general-purpose provider retry
policy to recover from temporary rate limiting or service failures.
@@ -66,6 +69,9 @@ with content repair. Permanent request failures must remain distinguishable.
## 2. Execution Observations, Including Failed Runs
**Implemented in source; not yet released.** Optional metadata callbacks cover
ordinary and claimed prepared execution, including normal error returns.
**Priority: High.** Audita needs to explain the time and calls consumed by a logical
operation, including when that operation never produces a successful result.
@@ -92,6 +98,10 @@ and failure accounting sufficiently complete for downstream diagnostics.
## 3. Actionable Failure Classification
**Implemented in source; not yet released.** Public sentinels and typed errors
preserve provider status, retry exhaustion, transport, malformed-response,
capacity, validation, and context distinctions.
**Priority: High; closely related to retries and observations.** Audita needs to
choose between failing a run, rejecting affected work, and recording a warning
without interpreting human-readable error messages.
@@ -110,6 +120,10 @@ Promptkit need not decide Audita's recovery or correction-acceptance policy.
## 4. Simple Per-Backend Capacity Limits
**Implemented in source; not yet released.** One engine-local backend pool
coordinates initial, retry, and repair attempts across profiles selecting it.
Audita must still configure finite limits for the paid backends it adopts.
**Priority: Adoption requirement, largely supported by existing facilities.**
Audita needs only two protections:
@@ -141,6 +155,12 @@ across a long run. A monetary budget or rate-limiting system is not requested.
## 5. Representative Consumer Example
**Implemented in source; not yet released.** The
[offline example](../../examples/go-library/observe/main.go) demonstrates the
built-in retry, structured proposal and assessment, metadata collection, and
separate application acceptance decision. Focused tests, not example switches,
cover cancellation and invalid-output branches.
**Priority: High as adoption evidence.** Provide a maintained offline example of a
structured proposal call followed by an independent LLM validation call, using
embedded prompts/schemas and separate application-owned execution profiles.
@@ -164,11 +184,10 @@ release containing this change and satisfy the applicable distribution obligatio
## Suggested Evaluation Order and Scope
Evaluate retries, failure classification, and execution observations together, then
use the representative consumer example to assess readiness and verify shared
per-backend limits. Existing capacity controls should suffice unless that exercise
reveals a concrete gap. Resolve licensing before selecting a release for Audita's
migration.
The implemented retry, classification, observation, and per-backend capacity
contracts can now be evaluated together with the representative example.
Audita still owns its migration decision and must select a later published
release before adopting this source state.
Existing facilities for embedded sources, application-owned profiles, prepared
execution, JSON-schema validation, and request-scoped credentials are useful

View File

@@ -472,6 +472,8 @@ example and its relevant tests; commit.
## Stage 12 — Complete Documentation And Final Validation
**Completed:** 2026-09-23.
Reconcile GoDoc, consumer error/observer/retry guidance, provider integration
docs, and internal runner/LLM/capacity/generation/observation documentation with
the implemented design. Remove claims that no retries exist or that usage can

View File

@@ -1,9 +1,10 @@
# Reliable And Observable Execution
**Status:** Accepted feature scope; not implemented. This document owns the
target end state and approved policy choices. The companion
[implementation plan](implementation.md) owns implementation steps and concrete
API and algorithm decisions.
**Status:** Implemented in source; not yet released. This document records the
accepted target and approved policy choices. The companion
[implementation plan](implementation.md) records implementation steps and
concrete API and algorithm decisions. Current contracts are owned by public
GoDoc and the linked consumer and integration documentation.
## Purpose
@@ -19,20 +20,21 @@ observations, shared capacity, and consumer evidence belong together because
they describe the same execution lifecycle. Licensing was handled separately.
Audita motivates the work but does not define an application-specific API.
## Existing Foundations
## Foundations At Planning Time
Promptkit already provides structural output validation and bounded repair,
prepared execution, engine-local backend capacity, run metadata, aggregate token
usage, and typed provider-status and capacity errors. Operational errors do not
return partial run results. The built-in client has no transient retry loop,
and existing integer usage fields do not distinguish missing reports from zero.
At the time this scope was accepted, Promptkit provided structural output
validation and bounded repair, prepared execution, engine-local backend
capacity, run metadata, aggregate token usage, and typed provider-status and
capacity errors. Operational errors did not return partial run results. The
built-in path had no transient retry loop, and existing integer usage fields
did not distinguish missing reports from zero.
The current contracts remain owned by public GoDoc, the
The implemented contracts are owned by public GoDoc, the
[consumer guide](../consumers/pkg-promptkit.md), and the
[provider integration contract](../integrations/openai-compatible-chat.md).
The [capacity document](../internal/capacity.md) and
[runner document](../internal/runner.md) describe the existing implementation.
This roadmap does not redefine those documents as already implementing new work.
This historical baseline does not override their current-state descriptions.
## Target End State