Expose ordinary run execution observations
This commit is contained in:
@@ -173,6 +173,34 @@ semantics. The
|
||||
[OpenAI-compatible integration contract](../integrations/openai-compatible-chat.md)
|
||||
owns the built-in client's outbound HTTP behavior.
|
||||
|
||||
### Observe An Ordinary Run
|
||||
|
||||
Set `RunRequest.Observer` to receive synchronous metadata events from `Run`:
|
||||
|
||||
```go
|
||||
request.Observer = func(event promptkit.ExecutionEvent) {
|
||||
// Translate selected metadata into an application-owned record.
|
||||
}
|
||||
result, err := engine.Run(ctx, request)
|
||||
```
|
||||
|
||||
An ordinary run emits `started` before request conversion, followed by any
|
||||
attempt and retry-delay completions, then one `finished` event on a normal
|
||||
return. `Prepare` and `PrepareExecution` emit no events. A start event has no
|
||||
resolved prompt, profile, or model; later events gain identities only as
|
||||
selection succeeds. The terminal event has dispatched-attempt, transient-retry,
|
||||
repair-call, duration, and presence-aware usage totals. Its usage is an
|
||||
aggregate, not an additional billable attempt. A known zero differs from an
|
||||
unknown count, and incomplete totals must not be treated as complete cost
|
||||
accounting. A failed operation still returns no partial `RunResult`.
|
||||
|
||||
Callbacks run synchronously, in order for one operation, and can overlap for
|
||||
different operations. Synchronize shared callback state and avoid blocking on
|
||||
work that needs the same run's admission slot. Callback panics propagate;
|
||||
there is no durable event history or promise of a terminal event after panic.
|
||||
Events omit content and error objects, but selected identifiers may still be
|
||||
sensitive. See the [observation GoDoc](../../observations.go) for exact fields.
|
||||
|
||||
### Repair A Structured Result
|
||||
|
||||
Set a small additional-call budget when a structurally invalid result can be
|
||||
|
||||
@@ -6,7 +6,7 @@ Its event values and callback type live in `internal/domain` so internal
|
||||
requests can carry them without depending on the recorder package. The
|
||||
recorder currently supplies run IDs to the runner and receives attempt and
|
||||
retry-delay measurements from the generation coordinator. Run lifecycle
|
||||
instrumentation is added at the runner boundary.
|
||||
instrumentation from ordinary `Run` at the facade and runner boundaries.
|
||||
|
||||
A recorder keeps only counters, resolved identity fields, and the selected
|
||||
callback. It has no global history, queue, goroutine, or mutex held across a
|
||||
|
||||
@@ -11,7 +11,7 @@ contributor workflow and validation.
|
||||
|
||||
| Component | Implemented responsibility | References |
|
||||
| --- | --- | --- |
|
||||
| Root `promptkit` package | Provides the supported engine facade, source, backend-registration, and injection options, public request, result, prompt-inspection, and profile-inspection values, opaque prepared-execution handles, profile construction, extension interfaces, value conversion, redacted formatting, and typed capacity and generation error mapping. | [Package GoDoc](../../doc.go), [prepared execution](../../prepared_execution.go), [backend API](../../backends.go), [engine facade](../../engine.go) |
|
||||
| Root `promptkit` package | Provides the supported engine facade, source, backend-registration, and injection options, public request, result, prompt-inspection, profile-inspection, and ordinary-run observation values, opaque prepared-execution handles, profile construction, extension interfaces, value conversion, redacted formatting, and typed capacity and generation error mapping. | [Package GoDoc](../../doc.go), [observations](../../observations.go), [prepared execution](../../prepared_execution.go), [backend API](../../backends.go), [engine facade](../../engine.go) |
|
||||
| `examples/go-library/prepare` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, and `Prepare`. It is not a public library package. | [Example program](../../examples/go-library/prepare/main.go) |
|
||||
| `examples/go-library/run` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, an injected deterministic model client, and `Run`. It is not a public library package. | [Example program](../../examples/go-library/run/main.go) |
|
||||
| `internal/backend` | Constructs each engine's immutable registry from maintained definitions and consumer additions, validates and defensively copies definitions through the shared JSON-value package, and consumes the LLM-owned OpenAI-compatible reserved request-field rule. | [Backend registry](../../internal/backend/registry.go) |
|
||||
@@ -29,7 +29,7 @@ contributor workflow and validation.
|
||||
| `internal/artifact` | Resolves ordinary inline and unrestricted caller-selected file references into copied artifacts with metadata and hashes. | [Internal sources and validation](sources.md) |
|
||||
| `internal/validate` | Validates basic, JSON, and JSON Schema output using operating-system filesystem or `fs.FS` schema sources and creates operation-local validation plans with canonical contained schema resources. | [Framework formats](../formats.md#schemas), [internal sources and validation](sources.md) |
|
||||
| `internal/llm` | Defines the internal generation boundary and implements outbound OpenAI-compatible chat requests from resolved execution targets, including bounded structured non-success response decoding, successful-response decoding, authentication, deadline handling, and ownership of the OpenAI-compatible reserved request-field policy. | [Internal model client](llm.md) |
|
||||
| `internal/observation` | Owns invocation-local run IDs, presence-aware usage and phase accounting, and synchronous internal metadata event delivery; generation and runner instrumentation are not yet connected. | [Internal execution observation](observation.md) |
|
||||
| `internal/observation` | Owns invocation-local run IDs, presence-aware usage and phase accounting, and synchronous metadata event delivery for ordinary runs and generation attempts. | [Internal execution observation](observation.md) |
|
||||
| `internal/usecase` | Resolves prompt definitions and hashes, profiles, backends, and targets for exact inspection and request settings for preparation, and coordinates ordinary execution and one-attempt prepared execution across internal sources, rendering, artifact loading, operation-local validation plans, generation, capacity, and bounded repair. | [Internal runner](runner.md), [prepared-execution implementation](../../internal/usecase/prepared_execution.go) |
|
||||
|
||||
The internal engine assembles these internal components without exposing their
|
||||
|
||||
@@ -134,6 +134,14 @@ and validates that artifact with the plan compiled during completion. Invalid
|
||||
generated content remains a validation result; an inability to generate or
|
||||
perform validation is an operational error.
|
||||
|
||||
When the public facade installs an invocation-local observation recorder,
|
||||
ordinary `Run` reuses its run ID and carries it to the generation coordinator
|
||||
through context. Prompt and profile identities are enriched only after their
|
||||
respective resolutions succeed, before admission. The facade starts observation
|
||||
before request conversion and emits its terminal event after public error
|
||||
mapping and after the runner's deferred admission release. Preparation and
|
||||
prepared execution have no observation delivery yet.
|
||||
|
||||
Validation preparation and execution honor cancellation at every
|
||||
Promptkit-controlled boundary and do not publish a partial plan or result.
|
||||
Schema reads are bounded and context-checked between chunks; JSON decoding,
|
||||
|
||||
@@ -382,6 +382,8 @@ Use synchronization rather than elapsed-time thresholds. Run race tests; commit.
|
||||
|
||||
## Stage 8 — Add Ordinary-Run Lifecycle And Public Observation API
|
||||
|
||||
**Completed:** 2026-09-23.
|
||||
|
||||
Add public observation types/conversion and RunRequest.Observer. The facade adapts
|
||||
the function and delegates bookkeeping to the internal recorder. Start ordinary
|
||||
Run observation before request conversion and nil-engine rejection; finish after
|
||||
|
||||
Reference in New Issue
Block a user