eric/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add invocation-local execution accounting

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-23 17:00:46 +00:00
parent 07a131bd33
commit 836919ec4b
8 changed files with 542 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files

31
docs/internal/observation.md Normal file
View File

@@ -0,0 +1,31 @@
# Internal Execution Observation
`internal/observation` owns one invocation's run ID, presence-aware token
aggregation, measured phase totals, and synchronous metadata event delivery.
Its event values and callback type live in `internal/domain` so internal
requests can carry them without depending on the recorder package. The
recorder currently supplies run IDs to the runner; attempt and lifecycle
instrumentation is added at their owning boundaries.
A recorder keeps only counters, resolved identity fields, and the selected
callback. It has no global history, queue, goroutine, or mutex held across a
callback. The caller supplies normalized failure metadata; the recorder never
parses an error string or receives provider bodies, credentials, headers, or
other content. Events are values with no mutable containers. Callback panics
propagate to the caller.
Attempt usage is known when its explicit presence bit is true. For legacy
injected clients without presence metadata, nonzero counts are known and
zeros are unknown. Aggregate values sum known contributions; `Known` means
at least one reported value, while `Complete` means every dispatched attempt
reported the field. Zero dispatched attempts produce known, complete zero
totals. Non-dispatched acquisitions contribute capacity wait but no usage or
generation duration. Terminal totals count dispatched attempts, transient
attempts above the first in a generation, and repair generations with at least
one dispatch. A terminal event is an aggregate and must not be added to its
attempt events.
The [recorder tests](../../internal/observation/recorder_test.go) own usage
presence, phase sums, event ordering, empty-run accounting, identity snapshots,
and callback panic behavior. The [generation coordinator](generation.md) owns
attempt and retry-delay measurements once connected to this recorder.

1
docs/internal/overview.md
View File

@@ -29,6 +29,7 @@ contributor workflow and validation.
| `internal/artifact` | Resolves ordinary inline and unrestricted caller-selected file references into copied artifacts with metadata and hashes. | [Internal sources and validation](sources.md) |
| `internal/validate` | Validates basic, JSON, and JSON Schema output using operating-system filesystem or `fs.FS` schema sources and creates operation-local validation plans with canonical contained schema resources. | [Framework formats](../formats.md#schemas), [internal sources and validation](sources.md) |
| `internal/llm` | Defines the internal generation boundary and implements outbound OpenAI-compatible chat requests from resolved execution targets, including bounded structured non-success response decoding, successful-response decoding, authentication, deadline handling, and ownership of the OpenAI-compatible reserved request-field policy. | [Internal model client](llm.md) |
| `internal/observation` | Owns invocation-local run IDs, presence-aware usage and phase accounting, and synchronous internal metadata event delivery; generation and runner instrumentation are not yet connected. | [Internal execution observation](observation.md) |
| `internal/usecase` | Resolves prompt definitions and hashes, profiles, backends, and targets for exact inspection and request settings for preparation, and coordinates ordinary execution and one-attempt prepared execution across internal sources, rendering, artifact loading, operation-local validation plans, generation, capacity, and bounded repair. | [Internal runner](runner.md), [prepared-execution implementation](../../internal/usecase/prepared_execution.go) |
The internal engine assembles these internal components without exposing their

2
docs/roadmap/implementation.md
View File

@@ -334,6 +334,8 @@ injected-client single dispatch. Run full tests and focused race tests; commit.
## Stage 6 — Build The Internal Observation Recorder
**Completed:** 2026-09-23.
Create `internal/observation` for invocation-local counters, presence-aware
aggregation, and synchronous dispatch. Internal event/value counterparts and
the callback function type belong in domain so domain requests/payloads can