Expose ordinary run execution observations

This commit is contained in:
2026-09-23 17:11:09 +00:00
parent be07f6d873
commit 0e1ad784d1
10 changed files with 494 additions and 6 deletions

View File

@@ -173,6 +173,34 @@ semantics. The
[OpenAI-compatible integration contract](../integrations/openai-compatible-chat.md)
owns the built-in client's outbound HTTP behavior.
### Observe An Ordinary Run
Set `RunRequest.Observer` to receive synchronous metadata events from `Run`:
```go
request.Observer = func(event promptkit.ExecutionEvent) {
// Translate selected metadata into an application-owned record.
}
result, err := engine.Run(ctx, request)
```
An ordinary run emits `started` before request conversion, followed by any
attempt and retry-delay completions, then one `finished` event on a normal
return. `Prepare` and `PrepareExecution` emit no events. A start event has no
resolved prompt, profile, or model; later events gain identities only as
selection succeeds. The terminal event has dispatched-attempt, transient-retry,
repair-call, duration, and presence-aware usage totals. Its usage is an
aggregate, not an additional billable attempt. A known zero differs from an
unknown count, and incomplete totals must not be treated as complete cost
accounting. A failed operation still returns no partial `RunResult`.
Callbacks run synchronously, in order for one operation, and can overlap for
different operations. Synchronize shared callback state and avoid blocking on
work that needs the same run's admission slot. Callback panics propagate;
there is no durable event history or promise of a terminal event after panic.
Events omit content and error objects, but selected identifiers may still be
sensitive. See the [observation GoDoc](../../observations.go) for exact fields.
### Repair A Structured Result
Set a small additional-call budget when a structurally invalid result can be

View File

@@ -6,7 +6,7 @@ Its event values and callback type live in `internal/domain` so internal
requests can carry them without depending on the recorder package. The
recorder currently supplies run IDs to the runner and receives attempt and
retry-delay measurements from the generation coordinator. Run lifecycle
instrumentation is added at the runner boundary.
instrumentation from ordinary `Run` at the facade and runner boundaries.
A recorder keeps only counters, resolved identity fields, and the selected
callback. It has no global history, queue, goroutine, or mutex held across a

View File

@@ -11,7 +11,7 @@ contributor workflow and validation.
| Component | Implemented responsibility | References |
| --- | --- | --- |
| Root `promptkit` package | Provides the supported engine facade, source, backend-registration, and injection options, public request, result, prompt-inspection, and profile-inspection values, opaque prepared-execution handles, profile construction, extension interfaces, value conversion, redacted formatting, and typed capacity and generation error mapping. | [Package GoDoc](../../doc.go), [prepared execution](../../prepared_execution.go), [backend API](../../backends.go), [engine facade](../../engine.go) |
| Root `promptkit` package | Provides the supported engine facade, source, backend-registration, and injection options, public request, result, prompt-inspection, profile-inspection, and ordinary-run observation values, opaque prepared-execution handles, profile construction, extension interfaces, value conversion, redacted formatting, and typed capacity and generation error mapping. | [Package GoDoc](../../doc.go), [observations](../../observations.go), [prepared execution](../../prepared_execution.go), [backend API](../../backends.go), [engine facade](../../engine.go) |
| `examples/go-library/prepare` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, and `Prepare`. It is not a public library package. | [Example program](../../examples/go-library/prepare/main.go) |
| `examples/go-library/run` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, an injected deterministic model client, and `Run`. It is not a public library package. | [Example program](../../examples/go-library/run/main.go) |
| `internal/backend` | Constructs each engine's immutable registry from maintained definitions and consumer additions, validates and defensively copies definitions through the shared JSON-value package, and consumes the LLM-owned OpenAI-compatible reserved request-field rule. | [Backend registry](../../internal/backend/registry.go) |
@@ -29,7 +29,7 @@ contributor workflow and validation.
| `internal/artifact` | Resolves ordinary inline and unrestricted caller-selected file references into copied artifacts with metadata and hashes. | [Internal sources and validation](sources.md) |
| `internal/validate` | Validates basic, JSON, and JSON Schema output using operating-system filesystem or `fs.FS` schema sources and creates operation-local validation plans with canonical contained schema resources. | [Framework formats](../formats.md#schemas), [internal sources and validation](sources.md) |
| `internal/llm` | Defines the internal generation boundary and implements outbound OpenAI-compatible chat requests from resolved execution targets, including bounded structured non-success response decoding, successful-response decoding, authentication, deadline handling, and ownership of the OpenAI-compatible reserved request-field policy. | [Internal model client](llm.md) |
| `internal/observation` | Owns invocation-local run IDs, presence-aware usage and phase accounting, and synchronous internal metadata event delivery; generation and runner instrumentation are not yet connected. | [Internal execution observation](observation.md) |
| `internal/observation` | Owns invocation-local run IDs, presence-aware usage and phase accounting, and synchronous metadata event delivery for ordinary runs and generation attempts. | [Internal execution observation](observation.md) |
| `internal/usecase` | Resolves prompt definitions and hashes, profiles, backends, and targets for exact inspection and request settings for preparation, and coordinates ordinary execution and one-attempt prepared execution across internal sources, rendering, artifact loading, operation-local validation plans, generation, capacity, and bounded repair. | [Internal runner](runner.md), [prepared-execution implementation](../../internal/usecase/prepared_execution.go) |
The internal engine assembles these internal components without exposing their

View File

@@ -134,6 +134,14 @@ and validates that artifact with the plan compiled during completion. Invalid
generated content remains a validation result; an inability to generate or
perform validation is an operational error.
When the public facade installs an invocation-local observation recorder,
ordinary `Run` reuses its run ID and carries it to the generation coordinator
through context. Prompt and profile identities are enriched only after their
respective resolutions succeed, before admission. The facade starts observation
before request conversion and emits its terminal event after public error
mapping and after the runner's deferred admission release. Preparation and
prepared execution have no observation delivery yet.
Validation preparation and execution honor cancellation at every
Promptkit-controlled boundary and do not publish a partial plan or result.
Schema reads are bounded and context-checked between chunks; JSON decoding,

View File

@@ -382,6 +382,8 @@ Use synchronization rather than elapsed-time thresholds. Run race tests; commit.
## Stage 8 — Add Ordinary-Run Lifecycle And Public Observation API
**Completed:** 2026-09-23.
Add public observation types/conversion and RunRequest.Observer. The facade adapts
the function and delegates bookkeeping to the internal recorder. Start ordinary
Run observation before request conversion and nil-engine rejection; finish after