diff --git a/docs/roadmap/remediation.md b/docs/roadmap/remediation.md new file mode 100644 index 0000000..0d74e9e --- /dev/null +++ b/docs/roadmap/remediation.md @@ -0,0 +1,115 @@ +# Configuration outcome review remediation + +Status — two open findings from the stage-by-stage review of +[the implementation plan](implementation.md) at commit +`8a22dd6`. Review findings concern the accepted +[feature target](invalid-configuration-outcomes.md). + +## R1 — Present ConfigKit diagnostics returned by request builders + +Priority — P2. Status — open. Affects Stages 3 and 4. + +### Relevant implementation + +- `WithRequestValidation` returns an empty acquisition when its builder fails in + [adapters.go](../../integrations/ckprompt/adapters.go) at line 196. +- `WithRequestPreparation` does the same in + [preparation.go](../../integrations/ckprompt/preparation.go) at line 43. +- `WithSmokeTesting` does the same in + [smoke.go](../../integrations/ckprompt/smoke.go) at line 51. +- The builder cases in + [configuration_outcomes_test.go](../../integrations/ckprompt/configuration_outcomes_test.go) + discard stderr and assert status and cause identity. The native engine-loader + cases also assert diagnostic presentation. + +### Circumstances and observed behavior + +An application request builder can load configuration while deriving a concrete +validation, preparation, or smoke request. If that load returns a wrapped +`*configkit.LoadError`, the native adapter returns the original cause with an +empty `Acquired` value. The handler correctly classifies the error, but receives +an empty diagnostic/explanation acquisition. Core renders the handler's generic +failure explanation and usage guidance. + +A temporary public-API reproduction used a ConfigKit loading error for malformed +`profiles: [` YAML with source label `operator-prompts.yml`. The following +commands returned status `2` and retained the typed loading cause, while stderr +omitted both `yaml.invalid` and the source label. + +- `promptkit requirements validate example` +- `promptkit show prepared example` +- `promptkit test reachability` +- `promptkit test all` + +Returning the same error from the engine loader presented the diagnostic and +source context in each command. Builder-failure cases invoked the engine loader +zero times. The early builder returns predate this sprint, but they leave a gap +in its accepted target of preserving actionable configuration diagnostics across +the affected paths. Stage 3 explicitly includes diagnostics in the native +loader/builder verification. + +### Expected behavior and remediation + +Adapt available ConfigKit source evidence, warnings, and terminal diagnostics +from builder-returned loading errors through the same presentation logic used +for engine-loader failures. Keep the original returned error as the cause and +apply the shared classifier and handler cancellation precedence. A failing +builder completes through its existing failure path. + +Reuse `promptKitDiagnostics` and the error adaptation already performed by +`loadPromptKitEngine`. A small private error-to-acquisition helper can make that +adaptation available to all three builder-failure branches. Ordinary application +errors follow their existing explanation and operational fallback behavior. + +Extend the public-command builder cases to capture stderr and assert the +diagnostic code, source context, usage status, and typed/wrapped cause. Cover +validation, preparation, individual smoke, and aggregate smoke through their +native adapters. Retain cancellation coverage and verify that the engine loader +is skipped following a builder error. These assertions concern CommandKit's +adaptation of supplied error evidence. + +## R2 — Reconcile feature-roadmap status and baseline wording + +Priority — P3. Status — open. Affects Stage 5 documentation consistency. + +The [feature roadmap](invalid-configuration-outcomes.md) still says +"pending implementation" at its opening. Its baseline section also describes +the current CLI contract as assigning loading failures status `1` and presents +the status change as proposed. Meanwhile, the implementation plan records all +stages complete, and the current +[CLI reference](../cli.md#configuration-failure-classification) documents the +implemented status `2` classification and operational fallback. + +Update the feature status to reflect implemented work, outstanding remediation, +and the separately tracked release coordination. Describe the old status `1` +contract explicitly as behavior at the reviewed baseline. Keep the historical +baseline evidence and accepted decisions, with links to the current CLI contract +and [release handoff](future.md#configuration-outcome-release-handoff). + +## Review verification + +Reviewed the code mapping and error-tree traversal in Stage 1, configuration +failure handling in Stage 2, inspection and request paths in Stage 3, smoke +aggregation and composed scopes in Stage 4, and documentation and release +coordination in Stage 5. The classifier's usage switch matches all 48 constants +in the plan. The implementation maintains the accepted ownership boundaries and +operational fallback. + +Using Go 1.27.0, offline dependencies, read-only module metadata, and a temporary +writable build cache, the following suites passed. + +- Root module in the aligned workspace, including the core example and scripts. +- ckconfig independently against its declared published dependencies, including + the simple example. +- ckconfig in the aligned workspace, including the simple example. +- ckprompt in the aligned workspace, including the composed example. + +Checked 133 local Markdown links/fragments across the six Markdown files changed +by the sprint and checked the sprint diff for whitespace errors. The temporary +builder reproduction described in R1 exercised the public command boundary +with deterministic, local failure fixtures. + +ckprompt's independent published-dependency validation follows the recorded +release handoff because its manifest still selects ckconfig v0.7.0. This is the +planned release coordination state. The review identified no additional +strengthened contracts or restrictions requiring remediation.