From cef8524b31f32ad931e44254debb372d8f177bc7 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Wed, 23 Sep 2026 17:29:12 +0000 Subject: [PATCH] Reconcile reliable execution documentation --- doc.go | 3 +- docs/consumers/pkg-promptkit.md | 16 +++++++ docs/integrations/openai-compatible-chat.md | 21 ++++++--- docs/internal/capacity.md | 13 ++++-- docs/internal/generation.md | 10 ++++- docs/internal/llm.md | 11 ++++- docs/internal/observation.md | 12 ++++-- docs/internal/overview.md | 6 +-- docs/internal/runner.md | 3 +- docs/roadmap/audita-feature-requests.md | 47 +++++++++++++++------ docs/roadmap/implementation.md | 2 + docs/roadmap/reliable-execution.md | 26 ++++++------ engine.go | 5 +++ observations.go | 25 ++++++++++- 14 files changed, 152 insertions(+), 48 deletions(-) diff --git a/doc.go b/doc.go index 899f56f..024d895 100644 --- a/doc.go +++ b/doc.go @@ -55,7 +55,8 @@ // [Backend], [RunRequest], [ArtifactRef], [ExecutionTargetOverride], [Profile], // [OpenAICompatibleProfileConfig], [ProfileInspection], // [PromptInputDefinition], [PromptInspection], [PreparedExecution], -// [CapacityError], and [GenerationError], do not have stable JSON +// [CapacityError], [GenerationError], [RetryExhaustedError], and +// [ExecutionEvent], do not have stable JSON // representations. Direct API keys are nevertheless excluded from JSON for // every public value. diff --git a/docs/consumers/pkg-promptkit.md b/docs/consumers/pkg-promptkit.md index c8cb089..a842c5c 100644 --- a/docs/consumers/pkg-promptkit.md +++ b/docs/consumers/pkg-promptkit.md @@ -201,6 +201,10 @@ repair-call, duration, and presence-aware usage totals. Its usage is an aggregate, not an additional billable attempt. A known zero differs from an unknown count, and incomplete totals must not be treated as complete cost accounting. A failed operation still returns no partial `RunResult`. +An injected client that leaves `UsagePresence` nil can report known nonzero +counts, but its zero counts remain unknown to observations; supply explicit +presence when measured zeros matter. Existing `RunResult.Usage` sums returned +values independently of observation presence. Callbacks run synchronously, in order for one operation, and can overlap for different operations. Synchronize shared callback state and avoid blocking on @@ -208,6 +212,9 @@ work that needs the same run's admission slot. Callback panics propagate; there is no durable event history or promise of a terminal event after panic. Events omit content and error objects, but selected identifiers may still be sensitive. See the [observation GoDoc](../../observations.go) for exact fields. +The [offline proposal and assessment example](../../examples/go-library/observe/main.go) +shows application-owned event collection and a separate semantic acceptance +decision using the built-in HTTP path without network access. ### Repair A Structured Result @@ -240,6 +247,9 @@ previous single-attempt behavior, or set `RunRequest.TransientRetries` to override one execution. A custom HTTP transport still uses this policy; injected `LLMClient` implementations retain their own retry policy. Application semantic retries are separate runs and can multiply provider traffic further. +The caller context covers capacity and retry waiting as well as attempts; +without a deadline a valid provider delay can be long. Promptkit does not +infer exactly-once or complete cost accounting from a retry outcome. See the [provider integration contract](../integrations/openai-compatible-chat.md#transient-status-retries) for eligible statuses, waits, timeouts, and provider timing. @@ -612,6 +622,12 @@ decoded or an injected client returns a nil successful response. Both retain `errors.Is` and `errors.As`. Caller cancellation and deadlines retain their standard context identities. Arbitrary injected-client errors retain their own identities without Promptkit guessing a failure category from their text. +An eligible built-in status that consumes a positive retry budget additionally +matches `ErrRetriesExhausted`; use `errors.As` to read +`RetryExhaustedError.Attempts()` and the final `GenerationError` status. With +explicit zero retries, the first eligible status is an ordinary generation +error, not exhaustion. Completed structural rejection remains a normal result +with failed validation, while an operational error returns no partial result. ## Application Boundary diff --git a/docs/integrations/openai-compatible-chat.md b/docs/integrations/openai-compatible-chat.md index 29edea8..90106fa 100644 --- a/docs/integrations/openai-compatible-chat.md +++ b/docs/integrations/openai-compatible-chat.md @@ -113,6 +113,14 @@ data, a second JSON value, absent choices, missing content, `null` content, non-string content, and size overflow are malformed responses and return no partial result. +Each of the five token fields retains whether the provider reported a +non-null integer, including zero. Omitted and `null` fields are unknown for +observations. A strictly decoded successful envelope can yield observed usage +even when a later missing choice or content makes the attempt malformed. +Invalid framing and non-2xx error bodies supply no observed usage; provider +work may still have occurred. Reported counts are not inferred, reconciled, +or proof of complete cost accounting. + For a non-2xx status, Promptkit recognizes one JSON document with a top-level object-valued `error` member. Its optional `message` and `type` fields must be strings, and `code` may be a string or JSON number. Valid supported fields are @@ -150,12 +158,13 @@ Promptkit regardless of this setting. A custom `http.Client` on the built-in path remains subject to these retries. Each retry uses equal jitter between half and all of an exponential local -delay, starting at 500 milliseconds and capped at 8 seconds. The first valid -`Retry-After` header value, as unsigned decimal seconds or an HTTP date, sets -a minimum not-before time measured from response receipt. Invalid values use -local delay; past dates impose no extra delay. The provider minimum is not -capped at 8 seconds. Retry waiting is cancellable and holds no active backend -permit. A caller without a deadline can wait a long time for provider timing. +delay, starting at 500 milliseconds and capped at 8 seconds. The first +`Retry-After` header value sets a minimum not-before time measured from +response receipt if it is valid unsigned decimal seconds or an HTTP date. +Invalid values use local delay; past dates impose no extra delay. The provider +minimum is not capped at 8 seconds. Retry waiting is cancellable and holds no +active backend permit. A caller without a deadline can wait a long time for +provider timing. The request body, selected URL, and resolved authorization are fixed across attempts of one generation. Each attempt creates a fresh HTTP request and diff --git a/docs/internal/capacity.md b/docs/internal/capacity.md index f812f35..5f3f320 100644 --- a/docs/internal/capacity.md +++ b/docs/internal/capacity.md @@ -77,12 +77,17 @@ list. Releasing a permit transfers it directly to the oldest remaining waiter before making it generally available. Pools do not order work relative to other backend IDs. -The coordinator passes injected generation requests, responses, and errors -through unchanged. The built-in HTTP client remains responsible for provider -transport behavior. The runner, rather than the +The coordinator does not retry injected clients; it normalizes a nil successful +response as malformed generation. The built-in HTTP client remains responsible +for provider transport behavior. The runner, rather than the capacity layer, sums all five usage fields from the initial response and every completed repair response into the successful run result. +An invocation-local observer receives attempt events after active permit +release, including canceled acquisition without dispatch. A retry wait holds +the logical admission lease but no active permit. Observer callbacks must not +block on work that requires the same run's admission slot. + ## Cancellation And Release Admission checks the caller context before reserving a slot. A call canceled @@ -116,6 +121,8 @@ release. The [external package capacity tests](../../capacity_contract_test.go) own the assembled public-engine behavior for configured limits, capacity errors, endpoint identity, engine independence, and injected clients. The +[reliable execution contract tests](../../reliable_execution_contract_test.go) +cover shared and independent backend pools during retry and repair. The [prepared-execution contract tests](../../prepared_execution_contract_test.go) own the public prepared-capacity boundary. The [root error-boundary tests](../../errors_internal_test.go) own preservation of diff --git a/docs/internal/generation.md b/docs/internal/generation.md index b4e3059..41491c1 100644 --- a/docs/internal/generation.md +++ b/docs/internal/generation.md @@ -29,8 +29,16 @@ including interrupted waits. Active permits are released before callbacks; generation duration ends before callback delivery. Built-in successful-envelope usage retains reported-field presence, while injected clients use explicit presence when supplied and otherwise treat zero fields as unknown. +Malformed successful envelopes can retain strictly decoded usage; error +response bodies are never mined for it. Arbitrary injected failures carry +unknown usage. Transport, timeout, malformed, and noneligible status failures +stop the sequence without retry exhaustion. Context is checked before each +wait, acquisition, and dispatch. Focused coordinator tests use controlled transport and wait functions. They cover eligibility, exhaustion, cancellation, backoff, request freezing, permit release, and the injected-client path. The capacity manager tests own permit -acquisition and idempotent release. +acquisition and idempotent release. Public +[integration tests](../../reliable_execution_contract_test.go) cover retry, +repair, observation, frozen prepared execution, and shared-backend behavior +through the assembled facade. diff --git a/docs/internal/llm.md b/docs/internal/llm.md index f4566f5..2277873 100644 --- a/docs/internal/llm.md +++ b/docs/internal/llm.md @@ -51,8 +51,9 @@ response on failure. when validating extra parameters. Backend registration consumes the same rule without making the model client depend on registry configuration. -The implementation has no retry loop, tool-call support, provider catalog, -inbound HTTP behavior, or durable session store. +This client does not schedule retries; the [generation coordinator](generation.md) +does so around its prepared attempts. The client has no tool-call support, +provider catalog, inbound HTTP behavior, or durable session store. ## Prepared Generation @@ -101,6 +102,12 @@ whitespace-only content. Missing choices, missing or `null` content, and non-string content are malformed responses. Output validation and correction eligibility remain outside this package. +Successful-envelope usage is decoded with independent presence bits for each +token field: omission and `null` are unknown, while a reported integer zero is +known. Strictly decoded usage can still be observed if later choice/content +checks fail; invalid framing or a non-success error body contributes none. +Existing successful-result accounting remains value-based. + For a non-success response, `ProviderHTTPError` retains the HTTP status and only normalized detail from the bounded recognized envelope. It retains `ErrUnexpectedStatus` through unwrapping. The client owns response closure; diff --git a/docs/internal/observation.md b/docs/internal/observation.md index 4d40568..d533b7c 100644 --- a/docs/internal/observation.md +++ b/docs/internal/observation.md @@ -4,9 +4,9 @@ aggregation, measured phase totals, and synchronous metadata event delivery. Its event values and callback type live in `internal/domain` so internal requests can carry them without depending on the recorder package. The -recorder currently supplies run IDs to the runner and receives attempt and -retry-delay measurements from the generation coordinator. Run lifecycle -instrumentation for ordinary `Run` runs at the facade and runner boundaries. +recorder supplies run IDs to the runner and receives attempt and +retry-delay measurements from the generation coordinator. Ordinary `Run` +lifecycle instrumentation spans the facade and runner boundaries. Prepared execution starts only after its private payload is atomically claimed; the facade receives that invocation's recorder privately and emits the terminal event after public error mapping. Failed claims and preparation emit nothing. @@ -17,6 +17,12 @@ callback. The caller supplies normalized failure metadata; the recorder never parses an error string or receives provider bodies, credentials, headers, or other content. Events are values with no mutable containers. Callback panics propagate to the caller. +Per-operation callbacks are ordered and synchronous; different operations may +invoke the same callback concurrently. The recorder marks each measured phase's +end before delivery, so callback work does not inflate provider duration. +Terminal phase durations sum measured phases, while wall duration may also +include preparation, validation, and callbacks. Panics do not promise a +terminal event. Attempt usage is known when its explicit presence bit is true. For legacy injected clients without presence metadata, nonzero counts are known and diff --git a/docs/internal/overview.md b/docs/internal/overview.md index a918265..51d78e4 100644 --- a/docs/internal/overview.md +++ b/docs/internal/overview.md @@ -20,7 +20,7 @@ contributor workflow and validation. | `internal/capacity` | Owns engine-local bounded execution admission and FIFO model-generation permits for limited backend IDs, including cancellation-safe waiter removal and client wrapping. | [Internal capacity management](capacity.md) | | `internal/domain` | Defines internal framework values for requests, artifacts, prompt definitions, profiles, execution targets, rendering, generation, and validation, and owns source-neutral invariants for shared execution settings, OpenAI-compatible base endpoints, session identifiers, and output contracts. Source parsing, required fields, other source-specific normalization, defaulting, and boundary-specific error classification remain with their callers. | [Domain declarations](../../internal/domain/domain.go), [endpoint invariant](../../internal/domain/endpoint.go) | | `internal/engine` | Assembles external catalogs, Promptkit-owned profiles, repositories, registry, capacity, collaborators, repairer, and the one use-case runner for each engine instance. | [Engine assembly](../../internal/engine/engine.go), [internal sources](sources.md#profiles-and-built-ins), [internal runner](runner.md) | -| `internal/generation` | Coordinates frozen built-in HTTP attempts and one-shot injected calls through active backend permits, including bounded status retries and cancellable backoff. | [Internal generation coordination](generation.md) | +| `internal/generation` | Coordinates frozen built-in HTTP attempts and one-shot injected calls through active backend permits, including bounded status retries, cancellable backoff, and attempt observations. | [Internal generation coordination](generation.md) | | `internal/defaults` | Defines application-neutral framework constants and constructs the default execution target. It contains no CLI, server, or inbound HTTP limits. | [Framework defaults](../../internal/defaults/defaults.go) | | `internal/filecatalog` | Provides deterministic YAML discovery and path helpers for operating-system filesystems and `fs.FS` sources. | [File catalog](../../internal/filecatalog/catalog.go) | | `internal/jsonvalue` | Validates and deeply copies bounded JSON-compatible extra-parameter and prepared-schema trees while preserving supported concrete value types and rejecting cycles or excessive depth and work. | [JSON values](../../internal/jsonvalue/jsonvalue.go) | @@ -30,8 +30,8 @@ contributor workflow and validation. | `internal/artifact` | Resolves ordinary inline and unrestricted caller-selected file references into copied artifacts with metadata and hashes. | [Internal sources and validation](sources.md) | | `internal/validate` | Validates basic, JSON, and JSON Schema output using operating-system filesystem or `fs.FS` schema sources and creates operation-local validation plans with canonical contained schema resources. | [Framework formats](../formats.md#schemas), [internal sources and validation](sources.md) | | `internal/llm` | Defines the internal generation boundary and implements outbound OpenAI-compatible chat requests from resolved execution targets, including bounded structured non-success response decoding, successful-response decoding, authentication, deadline handling, and ownership of the OpenAI-compatible reserved request-field policy. | [Internal model client](llm.md) | -| `internal/observation` | Owns invocation-local run IDs, presence-aware usage and phase accounting, and synchronous metadata event delivery for ordinary runs and generation attempts. | [Internal execution observation](observation.md) | -| `internal/usecase` | Resolves prompt definitions and hashes, profiles, backends, and targets for exact inspection and request settings for preparation, and coordinates ordinary execution and one-attempt prepared execution across internal sources, rendering, artifact loading, operation-local validation plans, generation, capacity, and bounded repair. | [Internal runner](runner.md), [prepared-execution implementation](../../internal/usecase/prepared_execution.go) | +| `internal/observation` | Owns invocation-local run IDs, presence-aware usage and phase accounting, and synchronous metadata event delivery for ordinary and claimed prepared runs and generation attempts. | [Internal execution observation](observation.md) | +| `internal/usecase` | Resolves prompt definitions and hashes, profiles, backends, and targets for exact inspection and request settings for preparation, and coordinates ordinary and single-claim prepared execution across internal sources, rendering, artifact loading, operation-local validation plans, generation, capacity, and bounded repair. | [Internal runner](runner.md), [prepared-execution implementation](../../internal/usecase/prepared_execution.go) | The internal engine assembles these internal components without exposing their representations. Consumers depend only on the root facade. diff --git a/docs/internal/runner.md b/docs/internal/runner.md index c8e7aba..99c685a 100644 --- a/docs/internal/runner.md +++ b/docs/internal/runner.md @@ -189,7 +189,8 @@ credential during execution, reaches generation and every repair attempt. Result usage is the field-wise sum of all five usage values from the initial response and every completed repair response. Final raw output, artifact, and validation state still come from the last candidate. A repair error returns no -partial run result or partial usage. +partial run result or partial result usage, although the invocation-local +observer retains completed-attempt usage and reports it in the terminal event. ## Failure Categories diff --git a/docs/roadmap/audita-feature-requests.md b/docs/roadmap/audita-feature-requests.md index 59880d6..87ed49d 100644 --- a/docs/roadmap/audita-feature-requests.md +++ b/docs/roadmap/audita-feature-requests.md @@ -2,16 +2,16 @@ **Last updated:** 2026-09-23 -**Status:** Proposed consumer requests, based on a review of Promptkit v0.10.0. -These are not approved features, implementation plans, delivery commitments, or -claims about current behavior. Priorities reflect Audita's prospective adoption; -Promptkit should evaluate them against its broader consumer needs. +**Status:** Requests 1–6 are implemented in source but not yet released. +These requests originated from a review of Promptkit v0.10.0. They retain +Audita's consumer context and priorities, not a current API specification or +a delivery commitment for Audita adoption. -Requests 1–5 are now incorporated into the active -[reliable-execution roadmap](reliable-execution.md), which owns the selected -Promptkit-native scope and remaining decisions. The requests below retain the -consumer context, not a separate implementation specification. Request 6 is -implemented in source as noted below. +Requests 1–5 were resolved through the +[reliable-execution roadmap](reliable-execution.md) and its +[implementation plan](implementation.md). The [consumer guide](../consumers/pkg-promptkit.md) +and public GoDoc own current behavior. No release or Audita adoption is +claimed here. ## Consumer Context @@ -34,6 +34,9 @@ consumer outcomes only; API shape and implementation are left to Promptkit. ## 1. Transient-Failure Retries +**Implemented in source; not yet released.** The built-in HTTP path owns the +selected transient-status policy; injected clients keep their own policy. + **Priority: High.** Audita should not need its own general-purpose provider retry policy to recover from temporary rate limiting or service failures. @@ -66,6 +69,9 @@ with content repair. Permanent request failures must remain distinguishable. ## 2. Execution Observations, Including Failed Runs +**Implemented in source; not yet released.** Optional metadata callbacks cover +ordinary and claimed prepared execution, including normal error returns. + **Priority: High.** Audita needs to explain the time and calls consumed by a logical operation, including when that operation never produces a successful result. @@ -92,6 +98,10 @@ and failure accounting sufficiently complete for downstream diagnostics. ## 3. Actionable Failure Classification +**Implemented in source; not yet released.** Public sentinels and typed errors +preserve provider status, retry exhaustion, transport, malformed-response, +capacity, validation, and context distinctions. + **Priority: High; closely related to retries and observations.** Audita needs to choose between failing a run, rejecting affected work, and recording a warning without interpreting human-readable error messages. @@ -110,6 +120,10 @@ Promptkit need not decide Audita's recovery or correction-acceptance policy. ## 4. Simple Per-Backend Capacity Limits +**Implemented in source; not yet released.** One engine-local backend pool +coordinates initial, retry, and repair attempts across profiles selecting it. +Audita must still configure finite limits for the paid backends it adopts. + **Priority: Adoption requirement, largely supported by existing facilities.** Audita needs only two protections: @@ -141,6 +155,12 @@ across a long run. A monetary budget or rate-limiting system is not requested. ## 5. Representative Consumer Example +**Implemented in source; not yet released.** The +[offline example](../../examples/go-library/observe/main.go) demonstrates the +built-in retry, structured proposal and assessment, metadata collection, and +separate application acceptance decision. Focused tests, not example switches, +cover cancellation and invalid-output branches. + **Priority: High as adoption evidence.** Provide a maintained offline example of a structured proposal call followed by an independent LLM validation call, using embedded prompts/schemas and separate application-owned execution profiles. @@ -164,11 +184,10 @@ release containing this change and satisfy the applicable distribution obligatio ## Suggested Evaluation Order and Scope -Evaluate retries, failure classification, and execution observations together, then -use the representative consumer example to assess readiness and verify shared -per-backend limits. Existing capacity controls should suffice unless that exercise -reveals a concrete gap. Resolve licensing before selecting a release for Audita's -migration. +The implemented retry, classification, observation, and per-backend capacity +contracts can now be evaluated together with the representative example. +Audita still owns its migration decision and must select a later published +release before adopting this source state. Existing facilities for embedded sources, application-owned profiles, prepared execution, JSON-schema validation, and request-scoped credentials are useful diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 77de6d5..623f82a 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -472,6 +472,8 @@ example and its relevant tests; commit. ## Stage 12 — Complete Documentation And Final Validation +**Completed:** 2026-09-23. + Reconcile GoDoc, consumer error/observer/retry guidance, provider integration docs, and internal runner/LLM/capacity/generation/observation documentation with the implemented design. Remove claims that no retries exist or that usage can diff --git a/docs/roadmap/reliable-execution.md b/docs/roadmap/reliable-execution.md index 45ea949..21bbcd2 100644 --- a/docs/roadmap/reliable-execution.md +++ b/docs/roadmap/reliable-execution.md @@ -1,9 +1,10 @@ # Reliable And Observable Execution -**Status:** Accepted feature scope; not implemented. This document owns the -target end state and approved policy choices. The companion -[implementation plan](implementation.md) owns implementation steps and concrete -API and algorithm decisions. +**Status:** Implemented in source; not yet released. This document records the +accepted target and approved policy choices. The companion +[implementation plan](implementation.md) records implementation steps and +concrete API and algorithm decisions. Current contracts are owned by public +GoDoc and the linked consumer and integration documentation. ## Purpose @@ -19,20 +20,21 @@ observations, shared capacity, and consumer evidence belong together because they describe the same execution lifecycle. Licensing was handled separately. Audita motivates the work but does not define an application-specific API. -## Existing Foundations +## Foundations At Planning Time -Promptkit already provides structural output validation and bounded repair, -prepared execution, engine-local backend capacity, run metadata, aggregate token -usage, and typed provider-status and capacity errors. Operational errors do not -return partial run results. The built-in client has no transient retry loop, -and existing integer usage fields do not distinguish missing reports from zero. +At the time this scope was accepted, Promptkit provided structural output +validation and bounded repair, prepared execution, engine-local backend +capacity, run metadata, aggregate token usage, and typed provider-status and +capacity errors. Operational errors did not return partial run results. The +built-in path had no transient retry loop, and existing integer usage fields +did not distinguish missing reports from zero. -The current contracts remain owned by public GoDoc, the +The implemented contracts are owned by public GoDoc, the [consumer guide](../consumers/pkg-promptkit.md), and the [provider integration contract](../integrations/openai-compatible-chat.md). The [capacity document](../internal/capacity.md) and [runner document](../internal/runner.md) describe the existing implementation. -This roadmap does not redefine those documents as already implementing new work. +This historical baseline does not override their current-state descriptions. ## Target End State diff --git a/engine.go b/engine.go index c3954a3..b4307a7 100644 --- a/engine.go +++ b/engine.go @@ -510,6 +510,11 @@ func (e *Engine) PrepareExecution(ctx context.Context, req RunRequest) (*Prepare // model attempts. Each dispatched attempt receives its own generation and // HTTP-client timeout; the caller context governs the full operation. // Injected LLMClient implementations are not retried automatically. +// With Observer, Run emits started before facade request conversion or a nil +// engine check, and finished on every normal return after public error mapping +// and resource release. The event RunID matches a successful result. Without +// Observer, result timing retains its existing convention; observed wall +// duration can be slightly longer because it spans the facade boundary. // // Run can return every error category documented by [Engine.Prepare], plus // ErrCapacityExceeded and ErrLLMGenerate. Built-in HTTP execution failures diff --git a/observations.go b/observations.go index bc0e473..6d16b70 100644 --- a/observations.go +++ b/observations.go @@ -10,7 +10,10 @@ import ( // ExecutionObserver receives immutable metadata events synchronously. Callers // must synchronize shared state if separate runs use the same observer. A -// callback must return for its run to progress; panics propagate to the caller. +// callback must return for its run to progress; cancellation cannot interrupt +// consumer code already in the callback. Do not block on work requiring the +// same run's admission slot. Panics propagate without a promise of a terminal +// event, and Promptkit keeps no asynchronous queue or durable history. type ExecutionObserver func(ExecutionEvent) // ExecutionEventKind identifies one point in an execution's lifecycle. @@ -57,7 +60,11 @@ const ( ) // ObservedTokenCount distinguishes an omitted count from a reported zero. -// Complete means every dispatched attempt reported the field. +// On an attempt, Known and Complete are true together only for a reported +// non-null integer. An aggregate Value sums known contributions; Known means +// at least one attempt reported the field, and Complete means every dispatched +// attempt did. With no dispatched attempts, aggregate zero is known and +// complete. Unknown fields have Value zero. type ObservedTokenCount struct { Value int Known bool @@ -78,6 +85,20 @@ type ObservedTokenUsage struct { // credentials, URLs, headers, or error objects. It has no stable JSON contract. // Identifiers themselves may be sensitive. Nonterminal aggregate counters are // zero; a terminal usage total must not be added to attempt usage. +// +// Generation is one for the initial generation and increases for repairs; +// Attempt is one-based within each generation. Dispatched means Promptkit +// invoked an HTTP attempt or injected Generate, not that a network packet was +// sent or billable work occurred. A canceled capacity acquisition emits a +// non-dispatched attempt with no usage. A retry-delay event describes the +// completed or interrupted wait after the indexed failed attempt. +// +// GenerationDuration excludes capacity waiting, retry delay, and callback +// work. Terminal phase durations sum measured phases; terminal Duration is +// wall time and may also include preparation, validation, and callbacks. +// Attempt success means generation succeeded, even if later validation rejects +// its output. Failure is classified metadata; RetriesExhausted is a separate +// flag and does not replace the underlying failure kind. type ExecutionEvent struct { Kind ExecutionEventKind RunID string