Collect PromptKit execution observations with logical attribution

This commit is contained in:
2026-09-24 02:23:57 +00:00
parent 4812032c9e
commit 4bfc5f3456
11 changed files with 457 additions and 14 deletions

View File

@@ -30,5 +30,9 @@ PromptKit's per-backend admission applies to both roles and every generation,
retry, and repair. The runner still processes module instances in order,
collects concurrent section work deterministically, and cancels remaining
section work on a runtime failure. `llm.DiagnosticsWriter` remains the protected
request/response artifact writer. Execution observations and report summaries
are completed by the following roadmap work.
request/response artifact writer. `llm.ObservationCollector` captures metadata
events under a short mutex and freezes them after runner work joins. Its logical
records include preparation failures; summary usage comes from terminal events
once per call and retains knownness and completeness for each field. Runner wall
and validator timings remain separate from provider attempts. The report and
artifact mapping is completed by the following roadmap work.

View File

@@ -429,7 +429,16 @@ or add placeholders merely to make this intermediate commit build.
## Stage 9: Collect Execution Observations Safely
**Status:** Pending. **Depends on:** Stage 8.
**Status:** Complete (2026-09-24). Added Audita-owned logical-call and execution
event records with call IDs, diagnostic attribution, short synchronized capture,
copying snapshots, and terminal-only usage aggregation. Preserved runner wall
and per-validator timings. Tests cover preparation failure, undispatched
admission, known zero versus unknown usage, partial failed repairs, no double
counting, concurrent attribution, and the real role-client hook. `go test
-race ./internal/framework/llm ./internal/framework/runner`, focused package
tests, and `go build -o /tmp/audita-stage9-build ./cmd/audita` passed;
`git diff --check` passed. Report/artifact serialization follows in Stage 10.
**Depends on:** Stage 8.
**Owns:** Audita-owned execution record types and collector in the LLM boundary,
runner attribution, and concurrency/observation tests.