From 9d3ffff69f4356a1a5707c4c47a6bf75a4929ace Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Mon, 16 Nov 2026 17:56:34 +0000 Subject: [PATCH] Plan configuration outcome implementation --- docs/roadmap/implementation.md | 339 ++++++++++++++++++ .../roadmap/invalid-configuration-outcomes.md | 137 ++----- 2 files changed, 361 insertions(+), 115 deletions(-) create mode 100644 docs/roadmap/implementation.md diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md new file mode 100644 index 0000000..79f191f --- /dev/null +++ b/docs/roadmap/implementation.md @@ -0,0 +1,339 @@ +# Invalid configuration outcome implementation + +Status — ready for implementation. Stages are pending. + +## Sprint scope and baseline + +Implement the accepted [feature roadmap](invalid-configuration-outcomes.md). +Its target and accepted decisions govern the work. This plan supplies the +specific API, diagnostic mapping, integration edits, and validation. + +The planning baseline is commit `a53bf9f916ffb44d816f3929fb5297a68ad118c0`. +Production sources still match the reviewed v0.7.0 integration baseline. Read +[development guidance](../development.md), the [agent policy](../policy/agents.md), +and the applicable architecture, testing, documentation, and release policies. + +Execute Stage 1 through Stage 5 in order as one sprint. Intermediate stages may +leave dependent work or test expectation updates for a later stage. Mark each +stage complete with its actual verification evidence and commit completed units +under the repository's local commit policy. Continue through the remaining +stages using the decisions in this plan. + +Use Go 1.27.0 and the ignored workspace containing core, ckconfig, and ckprompt. +The new shared API will first exist in local ckconfig. ckprompt currently selects +published ckconfig v0.7.0, which lacks it. The aligned workspace supplies local +ckconfig throughout this sprint. Published dependency alignment is release +coordination work described in Stage 5. Record that pending state explicitly +when reporting validation. + +## Stage 1 — Implement the shared classifier + +Status — pending. + +Add `integrations/ckconfig/errors.go` with the following exported function and +GoDoc describing its CLI classification purpose, accepted evidence, fallback, +nil behavior, and precedence. + +```go +func ClassifyConfigurationError(err error) commandkit.OutcomeKind +``` + +The function returns an outcome kind and reads error evidence. Handlers own +outcome construction, explanations, causes, and invocation context. + +### Exact behavior + +1. A nil error returns `commandkit.OutcomeSuccess`. +2. A nonnil error matching `context.Canceled` or `context.DeadlineExceeded` + through `errors.Is` returns `commandkit.OutcomeOperational`. +3. Recognize a nonnil `*configkit.LoadError`, including ordinary wrapping, and + inspect its complete `Diagnostics()` result. Ignore warning diagnostics. + Require at least one error-severity diagnostic and classify as usage only + when every error diagnostic has a usage code in the table below. An unknown + severity, an operational or unknown error code, or insufficient error + evidence selects operational. +4. Traverse ordinary `Unwrap() error` wrappers and `Unwrap() []error` branches. + A join selects usage only when every nonnil failing branch selects usage. + An operational branch wins in either ordering. A nonnil error exposing an + empty child list is an unclassified failure and selects operational. +5. Other nonnil errors select `commandkit.OutcomeOperational`. This includes an + empty or typed-nil `*configkit.LoadError`, opaque application errors, and + native PromptKit errors carrying their existing domain classifications. + +Use the public ConfigKit constants in an explicit switch. The following table is +the complete usage-code list for the selected ConfigKit v1.2.0 baseline. All +error codes outside it select operational. + +| Evidence | ConfigKit constants selecting usage | +| --- | --- | +| YAML content and fixed parser input bounds | `CodeYAMLInvalid`, `CodeYAMLEmpty`, `CodeYAMLMultipleDocuments`, `CodeYAMLRootType`, `CodeYAMLDuplicateKey`, `CodeYAMLTag`, `CodeYAMLAnchor`, `CodeYAMLAlias`, `CodeYAMLMergeKey`, `CodeYAMLScalar`, `CodeYAMLDepthLimit`, `CodeYAMLNodeLimit` | +| Typed decoding | `CodeDecodeType`, `CodeDecodeNull`, `CodeDecodeRange`, `CodeDecodeDuration`, `CodeDecodeText`, `CodeDecodeUnknownField`, `CodeDecodeArrayLength` | +| Excessive individual source input | `CodeSourceLimitExceeded` | +| Unsupported import directives and invalid graph relationships | `CodeImportDisabled`, `CodeImportCycle`, `CodeImportRepeated`, `CodeImportVersion` | +| Sequence declarations and peer composition | `CodeSequenceInvalid`, `CodeSequenceConflict`, `CodeComposeInvalid`, `CodeComposeConflict` | +| Protected version contributions | `CodeUnderlayVersion`, `CodeOverlayVersion` | +| Invalid overlay relationships and directives | `CodeOverlayRepeated`, `CodeOverlayDirective` | +| Version and explicit semantic rejection | `CodeVersionType`, `CodeVersionError`, `CodeResolveError`, `CodeValidateError` | +| Inheritance declarations and references | `CodeInheritanceInvalid`, `CodeInheritanceMissing`, `CodeInheritanceCycle`, `CodeInheritanceVersion` | +| Invalid override paths and conflicting assignments | `CodeOverrideInvalidPath`, `CodeOverrideDuplicate`, `CodeOverrideOverlap`, `CodeOverrideDirective`, `CodeOverrideVersion` | +| Excessive override input | `CodeOverrideLimitExceeded` | +| Invalid or absent typed secret references | `CodeSecretRequired`, `CodeSecretInvalidName` | + +The mapping follows the public code meanings and ConfigKit v1.2.0's +`docs/config.md` sections on input limits, YAML documents, composition, +inheritance, overrides, secret references, and safe failures. Review the pinned +module source and documentation in the Go module cache when implementing it. + +The fallback intentionally includes these reviewed cases. + +- Acquisition failures, including missing or unreadable sources and imports, + permission failures, non-regular sources, and unavailable overlays. +- `CodeImportInvalid` and `CodeOverlayInvalid`, which cover both invalid content + and acquisition/context failures. Their codes alone provide ambiguous evidence. +- `CodeImportOutsideRoot` and `CodeOverlayOutsideRoot`, where containment can + reflect source resolution and filesystem conditions. +- `CodeImportLimitExceeded` and `CodeOverlayLimitExceeded`, whose acquisition + budgets can encompass source graph and filesystem traversal work. +- Schema and options errors, source-description and reused-reader errors, + `CodeHistoryLimitExceeded`, and callback failures or invalid callback actions. +- `CodeOverrideInvalidLabel` and `CodeOverrideInvalidValue`, whose meanings also + cover malformed application-created records and unsupported Go values. +- `CodeSecretUnset` and `CodeSecretEmpty`, which describe credential acquisition + from the runtime environment. Secrets-directory bootstrap failures also follow + the operational fallback. + +The fixed source/parser bounds and override input budgets in the usage list +identify excessive supplied input. The acquisition and retained-evidence budgets +listed above retain operational classification under the accepted fallback. +These choices classify returned evidence and leave upstream limits and +validation rules with ConfigKit. + +Implement straightforward error-tree traversal. Recognize the current typed +loading error before following its children. Inspect join branches separately +so an `errors.As` search over a whole join cannot hide later failures. Perform +the cancellation `errors.Is` checks once at the public entry point, then walk +the tree and inspect each encountered loading error's diagnostics once. A small +private diagnostics classifier and recursive walker are appropriate here. + +Add focused tests in `integrations/ckconfig/errors_test.go`. + +- Exercise the mapping through the private diagnostics classifier using ordinary + diagnostic values, including warnings, mixed evidence, unknown codes, and + missing error evidence. Include each selected usage constant and representative + operational families. +- Exercise the exported function with real public loading-error fixtures, + wrappers, all-input joins, mixed joins in both orders, nested joins, + cancellation, deadlines, nil, and empty loading errors. Assert classification + and preservation of the original error's supported identity. + +Generate representative fixtures through public ConfigKit loading APIs. Assertions +own CommandKit's mapping and traversal. Upstream domain validation remains owned +by ConfigKit under the [testing policy](../policy/testing.md#upstream-integration-testing-ownership). + +Run the focused classifier tests. Stage 5 owns the final independent module suite. + +## Stage 2 — Apply classification and cancellation precedence in ckconfig + +Status — pending. + +Update `configFailure` in `integrations/ckconfig/operations.go` to receive +`context.Context` as its first argument. Update its callers in `operations.go`, +`show.go`, `sources.go`, `explain.go`, `validate.go`, and `diff.go` to pass the +invocation context. Add context to `configAcquisitionFailure` and pass it through +from `runConfigShow`. + +At this failure boundary, sample `ctx.Err()` when context is available. If it is +nonnull, use it as the cause when the operation cause is nil, or join it with the +operation cause when `errors.Is` does not already identify it. Preserve the +original cause directly when additional cancellation evidence is absent. + +Call `ClassifyConfigurationError` with that combined cause. Construct +`commandkit.Usage` for usage classification and `commandkit.Failure` otherwise. +Keep the latter default for intentionally reported failures with a nil cause. +Retain the existing explanation selection, details, and cause attachment. + +Apply the result to all acquisition errors reaching the five configuration +operations. Classification uses the returned error's complete evidence. Detailed +validation filters and projected `View.Findings` are report inputs. Successful +callbacks continue through their existing result flow. + +Follow the existing operation-specific reporting lifecycle. Sources can emit +their inventory after failed loading, validation can emit its reached-scope +report, and comparisons retain side-specific diagnostics. The existing +`config diff --check` domain outcome and core presentation rules apply. + +Extend the existing command and native-adapter tests to cover these outcomes. + +1. A wrapped malformed-YAML loading error returned through `FromConfigKit` + produces usage status `2` for show, sources, validate, explain discovery, and + diff. Assert the actionable diagnostic, typed cause, and appropriate report + evidence for each operation. +2. Diff handles a failure on either loading side and preserves the acquired + diagnostic context. A detailed-validation filter hiding an error has the same + classification as the unfiltered failure. +3. Ordinary acquisition failures and a representative operational loading error + produce status `1`. +4. Already-canceled invocations produce status `1`, retain cancellation identity, + and leave stdout empty before acquisition. A callback that cancels context + and returns a loading error retains both causes and selects status `1`. +5. An output failure while delivering a source or validation report after an + input failure follows core's operational presentation-failure rule while + retaining the configuration cause. + +Use existing coverage in `detailed_validation_test.go`, `diff_execution_test.go`, +the source/report tests, and `contracts_adapter_test.go` where suitable. Update +the invalid-fixture expectations in `examples/simple/main_test.go`, which +currently assert operational outcomes for native loading errors. Check related +expectations against their actual fixture codes before changing them. + +Run the changed command tests and the affected simple-example tests. Stage 5 +owns the final independent ckconfig suite. + +## Stage 3 — Apply the shared function to PromptKit inspection and requests + +Status — pending. + +Update `definitionFailure` in `integrations/ckprompt/command.go` to receive the +invocation context and combine observed cancellation with the operation cause +using the same handler-level approach as Stage 2. Call +`ckconfig.ClassifyConfigurationError` and choose Usage or Failure while retaining +the existing explanation and cause. A nil cause for a missing provider result +still yields an operational failure. + +Update all callers, including the catalog fallback in `newCatalogListCommand`, +the handlers in `command.go`, `validation_command.go`, `preparation.go`, and the +smoke handlers that Stage 4 will verify. + +Cover every installed catalog and named prompt/profile/backend inspection, +requirements discovery, concrete validation, and prepared-request preview. +Configuration loading errors may come from engine loaders or request builders. +The returned ConfigKit evidence selects the classification in each path. +Native lookup, preparation, validation, and provider errors with insufficient +configuration evidence take the operational fallback. Existing preflight and +application input-check contracts continue to govern their own failures. + +Retain the original upstream error and diagnostic adaptation in `adapters.go`. +Use the newly context-aware failure boundary to observe cancellation when a +callback returns an error before the handler's later success checkpoint. + +Extend tests around `command_test.go`, `requirements_command_test.go`, +`validation_command_test.go`, and `preparation_test.go`. + +- Supply a shared real ConfigKit loading-error fixture through each relevant + suite callback and representative native loader/builder paths. Assert usage + status `2`, diagnostics, and `errors.As`/`errors.Is` preservation. +- Cover an ordinary PromptKit lookup error, an ordinary loader or builder + failure, and missing-result handling as operational outcomes. +- Cover pre-canceled input and cancellation occurring with a returned loading + error, checking status `1` and retained cancellation/loading identity. + +Use table-driven cases for handlers with equivalent failure behavior. Reuse +existing tests for forwarding and lifecycle semantics. Verify the affected +packages using the aligned workspace. + +## Stage 4 — Verify smoke aggregation and configuration composition + +Status — pending. + +In `integrations/ckprompt/smoke_command.go`, complete the context argument updates +and apply the shared classification to individual failures and the final joined +aggregate cause. Follow the existing sequential four-kind execution, cancellation +stop behavior, report delivery, explanation combination, and acquired diagnostic +flow. The classifier supplies the new aggregate outcome. + +Extend `smoke_outcome_test.go`, `smoke_aggregate_test.go`, and related command +tests to verify a single input failure, an all-input aggregate, a mixed +input/operational aggregate, and cancellation joined with an input failure. +Assert the available reports and all supplied error identities. Existing +report-with-error behavior remains part of those assertions. + +Exercise `ConfigScope` through the public ckconfig family in ckprompt tests. +Use a loader returning a wrapped malformed-YAML loading error and cover show, +sources, validate, explain discovery, and diff, including the right comparison +side. This establishes that composed PromptKit configuration commands use the +shared ckconfig policy. Check the accompanying source and validation report +evidence and the already-canceled behavior. + +Update affected assertions in `examples/composed/main_test.go` against the +fixture's actual evidence. Add a focused fixture only where it establishes a +CommandKit-specific composition or outcome boundary. Reuse the existing example +and adapter tests for ordinary upstream capability forwarding. + +Run the focused smoke/composition tests. Stage 5 owns the final aligned-workspace +ckprompt suite. Review native lookup and operational error tests for accidental +classification broadening. + +## Stage 5 — Update documentation, validate, and record release coordination + +Status — pending. + +Update the implemented CLI contract in `docs/cli.md`, including the outcome +table and affected load-failure wording in configuration comparison, inspection, +requirements discovery, concrete validation, catalogs, preparation, and smoke +testing. Document input classification, operational fallback, mixed-error +precedence, and the existing output-failure rules. Add the new exported API's +GoDoc and concise consumer guidance linked to it. + +Prepare `integrations/ckconfig/docs/releases/v0.8.0.md` explaining the status +contract change and shared classification API, using the module's existing +release-note conventions and immutable licensing footer. Record the planned +ckprompt v0.8.0 note and published-dependency update as release follow-up under +`docs/roadmap/future.md`. The ckprompt candidate is prepared after ckconfig +publication under the [release procedure](../release.md#scope-and-release-order). + +### Final sprint verification + +Format changed Go files. Run the commands below from the indicated module +directories, with Go 1.27.0 on PATH. The default suites are deterministic and +offline once selected dependencies are present in the module cache. + +From `integrations/ckconfig`, verify its published dependencies independently. + +```sh +GOWORK=off GOTOOLCHAIN=local GOPROXY=off go test -mod=readonly ./... +``` + +From the repository root, verify all three local modules together. + +```sh +GOTOOLCHAIN=local GOPROXY=off go test -mod=readonly ./... +GOTOOLCHAIN=local GOPROXY=off go test -mod=readonly ./integrations/ckconfig/... +GOTOOLCHAIN=local GOPROXY=off go test -mod=readonly ./integrations/ckprompt/... +``` + +The integration commands include their maintained example packages. Review +the final diff, Markdown links/fragments, and `git diff --check`. Confirm that +the shared code mapping, error trees, cancellation, reporting, both scope +adapters, and command families have the planned regression evidence. Report +actual checks and any remaining failures. + +ckprompt's independent published-dependency build is pending the ckconfig release +because its current v0.7.0 selection lacks the new function. Keep that limitation +visible in the development/consumer guidance and release handoff. Retain the +existing published requirements through the implementation sprint. The ignored +aligned workspace is the local build setup for the new source combination. + +### Release handoff + +Record these follow-up actions together in `future.md` so they survive later +retirement of the implementation documents. + +1. Publish ckconfig v0.8.0 from its independently validated candidate using the + documented release process when publication is authorized. +2. Prepare ckprompt with actual published ckconfig v0.8.0 using `go get` and + `go mod download` with `GOWORK=off GOTOOLCHAIN=local`. Retain real checksums, + then finish its v0.8.0 release note, consumer compatibility guidance, and + release preparation. The other dependency selections follow their existing + release preparation rules. +3. Validate ckprompt independently against its updated published dependencies + through its release candidate suite and publish ckprompt v0.8.0 under the + same authorization and procedure. Track hosted release delivery under the + runbook's separate delivery process. + +These actions follow the existing dependency-first release procedure. The sprint +can complete with verified local behavior and explicit pending release alignment. +The release handoff identifies the remaining independent-build evidence. + +Mark each completed stage with its implementation and verification evidence. +Commit the completed sprint work under the repository policy and report its +commits, checks, and pending release coordination. diff --git a/docs/roadmap/invalid-configuration-outcomes.md b/docs/roadmap/invalid-configuration-outcomes.md index 9d5c01e..0284b55 100644 --- a/docs/roadmap/invalid-configuration-outcomes.md +++ b/docs/roadmap/invalid-configuration-outcomes.md @@ -1,10 +1,9 @@ # Invalid configuration outcomes -Status — accepted roadmap, pending implementation planning. The user has -accepted the targeted integration behavior, the built-in diagnostic mapping -with operational fallback, and a shared exported classification function in -ckconfig. The implementation plan will enumerate the mapped codes and select -the function's exact name and signature. +Status — accepted feature roadmap, pending implementation. The user has +accepted the targeted behavior and both design decisions below. The +[implementation plan](implementation.md) owns the ordered work, exact code +mapping, shared API signature, and verification. ## Purpose @@ -89,110 +88,15 @@ execution, and application-specific error semantics retain their owning contracts. The work concerns interpreting returned failure evidence at the CLI boundary. -## Work needed +## Release targets -### 1. Define the configuration-failure classification - -- Inventory ConfigKit v1.2.0's public diagnostic codes and their documented - meaning. Record the selected input-error cases explicitly using upstream - constants. Review all code families, including composition, imports, - inheritance, versions, overrides, secrets, and configured limits. -- Use error-severity diagnostics as failure evidence. Warning records can - accompany a failure or a successful result. -- Handle wrapped `LoadError` values and standard joined error branches. A failure - selects usage only when its failing evidence is classified as invalid input. - An operational or unclassified failing branch makes the combined result - operational. Preserve the original cause tree. -- Account for cancellation and deadlines before choosing usage. When an existing - checkpoint observes cancellation alongside an acquisition failure, preserve - both causes using the established error-joining approach. -- Implement the built-in mapping and error-chain classification in one exported - ckconfig function, shared by both integrations. Existing core outcome - constructors and reporting provide the required mechanics. Invocation-specific - cancellation checks belong in the handlers. - -### 2. Apply it throughout ckconfig - -Wire the classification into acquisition failure handling for `config show`, -`config sources`, `config validate`, `config explain`, and `config diff`. - -Follow each operation's current preparation, aggregation, and reporting flow. -In particular, source and validation reports can accompany a failed load. -Detailed validation filters control displayed findings, while the complete -underlying failure supplies classification evidence. Comparison must classify a -failure on either loading side and retain the side-specific diagnostic context. - -Include native ConfigKit scopes, PromptKit configuration scopes, and custom -scope callbacks returning the supported evidence. Scope selection and input -preflight continue through their existing contracts. - -### 3. Apply it throughout ckprompt - -Cover engine-acquisition failures in catalogs, named prompt/profile/backend -inspection, requirements discovery, concrete request validation, prepared-request -preview, individual smoke operations, and aggregate smoke testing. - -Acquisition evidence may arrive through loaders or application request builders. -Use the same classification policy wherever it reaches the standard command's -failure boundary. For joined aggregate smoke errors, an operational or -unclassified failure takes precedence over input failures. Follow the existing -aggregate execution and report lifecycle when selecting the final outcome. - -### 4. Verify observable integration behavior - -Extend focused public-command tests where they provide regression protection. -Use upstream errors as fixtures to establish CommandKit's classification and -preservation behavior, following the -[integration testing policy](../policy/testing.md#upstream-integration-testing-ownership). - -Cover these distinct risks with a proportionate set of cases. - -- Malformed main and companion configuration produce status `2` with - `yaml.invalid` and the typed ConfigKit cause available through wrapping. -- Representative decoding and selected semantic-validation errors receive the - intended classification. Operational and unrecognized loading diagnostics - receive status `1`, including mixed failing evidence and warnings alongside - errors. -- The five configuration operations and affected PromptKit command families use - the policy, including comparison-side failures and aggregate smoke outcomes. -- Cancellation and deadlines receive status `1` and preserve their supported - identity. Already-canceled invocations retain the reviewed empty-output - behavior. A failed acquisition combined with cancellation preserves both causes. -- Source and validation reports retain their operation-specific failure evidence. - Detailed display filters do not determine failure classification. -- Output failures continue to follow the shared reporting contract when the - command has selected usage status `2`. - -Run the complete deterministic suites independently for each changed integration -and with the aligned local workspace as described in the -[developer guide](../development.md#documentation-validation). Exercise the -maintained examples where their existing assertions cover affected outcomes. -The Audita reproduction supplies supporting consumer evidence. CommandKit-owned -tests provide the maintained regression coverage. - -### 5. Document and prepare the integration releases - -Update the CLI status table and affected operation sections with the implemented -classification. Update any affected exported GoDoc and consumer guidance under -their canonical owners. Explain the classification boundary and fallback through -the chosen API's documentation. - -Prepare ckconfig v0.8.0 followed by ckprompt v0.8.0, with ckprompt selecting the -published ckconfig version. The status-contract change warrants minor releases -under the [compatibility policy](../policy/release.md#versioning-and-compatibility). -The release notes should explain that identifiable invalid configuration now -selects status `2`, alongside the operational classification and preserved error -evidence. Follow the [release procedure](../release.md) when publication is -authorized. Implementation completion and publication are tracked separately. - -## Completion evidence - -Implementation is complete when the accepted classification is exercised through -the public integration commands, the focused regression cases and selected -integration suites pass, and the canonical documentation describes the -implemented behavior. Review the final diff for the agreed scope and retire this -roadmap under the normal documentation lifecycle after recording any unfinished -release work. +Target ckconfig v0.8.0 followed by ckprompt v0.8.0, with released ckprompt +selecting the published ckconfig version containing the shared API. This is a +published status-contract change under the +[compatibility policy](../policy/release.md#versioning-and-compatibility). +Implementation and source publication have their own completion evidence under +the [release procedure](../release.md). Pending downstream dependency alignment +belongs in the release handoff. ## Accepted decisions @@ -204,9 +108,10 @@ code semantics. Include YAML-content and decoding errors, explicit whose documented meaning clearly identifies invalid configuration content. Classify schema/options defects, acquisition failures, hook execution failures, invalid callback paths or records, and validation mutation as operational. -Keep ambiguous limit, secret, and source-selection cases operational until their -individual meaning supports input classification. Review and record those cases -when preparing the implementation plan. +Use operational classification for ambiguous limit, secret, and source-selection +evidence. The implementation plan records the individual codes selected under +this rule, including limits that explicitly identify excessive input and secret +reference errors that explicitly identify invalid configuration. This uses the evidence upstream already supplies and distinguishes reported configuration rejection from callback machinery failing. Source evidence @@ -220,12 +125,14 @@ Place one narrowly documented, exported classification function in ckconfig. Both integrations use it, with ckprompt accessing it through its existing dependency. The function inspects the returned error and its ConfigKit diagnostics and returns the appropriate outcome classification. Its -responsibilities include the -diagnostic mapping, operational fallback, and joined-error precedence. +responsibilities include the diagnostic mapping, operational fallback, and +joined-error precedence. Keep command explanation construction, cause attachment, reporting, and -invocation-specific cancellation checks in their owning handlers. Select the -function's exact name and signature during implementation planning. +invocation-specific cancellation checks in their owning handlers. The +implementation plan specifies the shared function as +`ClassifyConfigurationError(error) commandkit.OutcomeKind`. It classifies nil as +success, identifiable input failures as usage, and other failures as operational. A shared implementation reduces drift across the two standard families and fits ckconfig's ownership of ConfigKit adaptation. The narrowly scoped public