eric/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Isolate execution observations and preserve injected error categories

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-23 17:56:15 +00:00
parent 48378a3269
commit 0c08038b4f
8 changed files with 291 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files

43
docs/roadmap/remediation.md
View File

@@ -8,15 +8,15 @@ The intended architecture is substantially implemented: the public facade
adapts contracts, the runner owns execution and repair, one generation
coordinator owns retries and active permits, the HTTP adapter freezes requests
and decodes responses, and an invocation-local recorder produces observations.
The following two correctness gaps need remediation before release. This is a
review record, not an implementation change or a new public contract.
The two correctness gaps below have been remediated. This document preserves
the review evidence and resolution, not a new public contract.
## 1. Isolate Observation State Across Public Invocations (P2)
**Status:** Open.
**Status:** Resolved.
**Location:** [Engine.Run](../../engine.go), lines 535–547;
[Runner.Run](../../internal/usecase/runner.go), lines 138–145;
**Location:** [Engine.Run](../../engine.go);
[Runner.Run](../../internal/usecase/runner.go);
[recorder context propagation](../../internal/observation/recorder.go).
The observer is request-scoped, but its recorder is attached to the context
@@ -48,13 +48,21 @@ child with its own observer must have its own lifecycle. Include focused
prepared/preparation coverage where the same context plumbing is used, and
race-enabled coverage for concurrent delegation without sharing recorder state.
**Resolution:** Preparation and execution facade entry points mask inherited
recorder/claim state before installing any invocation-owned state. The context
boundary preserves caller values, deadlines, cancellation, and cancellation
causes. Public regression tests cover observed and unobserved delegated ordinary
and prepared runs, preparation isolation, independent accounting/identities,
and concurrent delegation. A focused recorder test checks context preservation
and claim-sink isolation.
## 2. Preserve Injected Failure Categories In Attempt Events (P2)
**Status:** Open.
**Status:** Resolved.
**Location:** [public client adapter](../../llm_adapter.go), lines 14–20;
[attempt classification](../../internal/generation/coordinator.go), lines
170–172 and 235–247; [terminal classification](../../observations.go).
**Location:** [public client adapter](../../llm_adapter.go);
[attempt classification](../../internal/generation/coordinator.go);
[terminal classification](../../observations.go).
The public adapter returns public error sentinels, including
`ErrMalformedResponse` for a successful nil response. The coordinator recognizes
@@ -81,15 +89,22 @@ error should assert both attempt and terminal classifications, preserved error
identity, and single dispatch. Retain context-first classification for wrapped
cancellation/deadline causes.
**Resolution:** The facade adapter adds internal classification identities for
explicit public transport/malformed errors, retaining the original diagnostic
and error chain. Nil success uses the same normalization path. Public tests
check attempt/terminal agreement, `errors.Is`/`errors.As`, single dispatch even
with a positive retry budget, arbitrary errors, and context-first classification.
## Validation And Review Outcome
- `./scripts/release-check.sh` passed on the committed implementation, including
- `./scripts/release-check.sh` passed on the review baseline and remediated tree, including
race-enabled tests, vet, the three offline examples, formatting, documentation
links, and repository hygiene checks.
- Temporary public-API probes reproduced both findings. They were removed after
review; no implementation or permanent test changes are included here.
- Temporary public-API probes reproduced both findings during review. They were
removed and replaced by focused permanent regression coverage during remediation.
- The existing tests provide useful coverage of retries, repair interaction,
capacity, prepared claims, usage presence, and redaction, but do not protect
these two observation-boundary cases.
capacity, prepared claims, usage presence, and redaction, but did not protect
these two observation-boundary cases before remediation. The new
[boundary tests](../../observation_boundary_contract_test.go) protect them.
- No other must-fix issue was identified in this review. This does not establish
live-provider behavior; all verification was offline.