Collect PromptKit execution observations with logical attribution
This commit is contained in:
@@ -30,5 +30,9 @@ PromptKit's per-backend admission applies to both roles and every generation,
|
||||
retry, and repair. The runner still processes module instances in order,
|
||||
collects concurrent section work deterministically, and cancels remaining
|
||||
section work on a runtime failure. `llm.DiagnosticsWriter` remains the protected
|
||||
request/response artifact writer. Execution observations and report summaries
|
||||
are completed by the following roadmap work.
|
||||
request/response artifact writer. `llm.ObservationCollector` captures metadata
|
||||
events under a short mutex and freezes them after runner work joins. Its logical
|
||||
records include preparation failures; summary usage comes from terminal events
|
||||
once per call and retains knownness and completeness for each field. Runner wall
|
||||
and validator timings remain separate from provider attempts. The report and
|
||||
artifact mapping is completed by the following roadmap work.
|
||||
|
||||
@@ -429,7 +429,16 @@ or add placeholders merely to make this intermediate commit build.
|
||||
|
||||
## Stage 9: Collect Execution Observations Safely
|
||||
|
||||
**Status:** Pending. **Depends on:** Stage 8.
|
||||
**Status:** Complete (2026-09-24). Added Audita-owned logical-call and execution
|
||||
event records with call IDs, diagnostic attribution, short synchronized capture,
|
||||
copying snapshots, and terminal-only usage aggregation. Preserved runner wall
|
||||
and per-validator timings. Tests cover preparation failure, undispatched
|
||||
admission, known zero versus unknown usage, partial failed repairs, no double
|
||||
counting, concurrent attribution, and the real role-client hook. `go test
|
||||
-race ./internal/framework/llm ./internal/framework/runner`, focused package
|
||||
tests, and `go build -o /tmp/audita-stage9-build ./cmd/audita` passed;
|
||||
`git diff --check` passed. Report/artifact serialization follows in Stage 10.
|
||||
**Depends on:** Stage 8.
|
||||
**Owns:** Audita-owned execution record types and collector in the LLM boundary,
|
||||
runner attribution, and concurrency/observation tests.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user