Plan ConfigKit adoption in five stages

docs/roadmap/configkit.md

@@ -4,7 +4,8 @@
 
 **Status:** Planned; implementation has not begun. The dependency-only review
 passed Audita's existing offline test suite in a disposable checkout. That is
-baseline compatibility evidence, not completion of the feature work below.
+baseline compatibility evidence, not completion of the feature. The
+[implementation plan](implementation.md) owns the ordered work and verification.
 
 ## Purpose and Baseline
 
@@ -37,10 +38,10 @@ and release target do not govern this incremental upgrade.
 
 - Audita uses the released ConfigKit v0.5.0 module, with no local replacement.
   PromptKit stays at v0.11.2 throughout this adoption unit.
-- Existing configurations continue through the current file-selection,
+- Configuration retains the current file-selection, overlay, explicit-override,
-  overlay, explicit-override, resolution, and validation flow. Retain the
+  resolution, and validation flow, subject to the reserved-key compatibility
-  application schema version, CLI interface, and main-file-relative resource
+  impact described below. Retain the application schema version, CLI interface,
-  rules defined in the current contracts.
+  and main-file-relative resource rules defined in the current contracts.
 - Application configuration can use ConfigKit's native sibling `extends`
   relationships. Audita does not implement a second inheritance resolver,
   special backend-only syntax, or an opt-in switch around the library feature.
@@ -64,7 +65,7 @@ value still has to satisfy Audita's field-specific rules. Parent definitions
 also undergo ordinary application validation; inheritance does not introduce
 abstract, validation-exempt backend entries.
 
-## Scope and Sequencing
+## Scope and Adoption Boundary
 
 Keep the existing `Load` integration unless implementation reveals a concrete
 need to change it. Source-backed defaults, additional operator underlay tiers,
@@ -78,96 +79,28 @@ profile formats, profile inheritance, engine construction, embedded prompts,
 and backend catalogs remain governed by the existing PromptKit integration.
 There is no application-owned translation of profile documents in this work.
 
-Complete and validate ConfigKit adoption before migrating PromptKit. PromptKit
+This is an independent ConfigKit adoption ahead of the planned PromptKit
-v0.12.0 itself requires ConfigKit v0.5.0; this sequence isolates the generic
+migration. A separate public release between the two units is optional.
-configuration change from the later API and profile-format migration. Each
-completed adoption unit should build and pass its applicable checks. A separate
-public release between the two units is optional, not a prerequisite.
 
-## Work Plan
+## Acceptance Criteria
 
-### 1. Upgrade the Dependency
+- The released ConfigKit v0.5.0 dependency and exact source attribution are
-
+  adopted; PromptKit remains at v0.11.2 with its existing catalogs.
-Update `go.mod` and `go.sum` to the released ConfigKit version. Review the
+- Native inheritance preserves effective endpoints, capacity, explicit zero
-resolved dependency diff to keep PromptKit and its catalogs at their current
+  queue capacity, child overrides, and surviving credential references through
-versions. Retain the existing Go minimum and other YAML dependencies used by
+  Audita's loading, resolution, and validation boundary.
-Audita.
+- Selected overlays and explicit overrides retain their precedence; omitted
-
+  validation-role selection still derives from the final proposal selection.
-Check the production loader against the new presence and inheritance semantics.
+- Null can remove a child's credential while surviving references on parents
-Make only changes needed to preserve the target behavior; do not replace the
+  and other unselected backends still follow the existing credential contract.
-loader simply to exercise the new layered API.
+- Selected roles use inherited backend settings and share its configured
-
+  capacity under the existing PromptKit runtime contract.
-### 2. Verify Inheritance at Audita's Boundary
+- Effective settings remain redacted, and both CLI explanations and debug
-
+  configuration origins expose compact, value-free inheritance evidence while
-Review existing configuration tests and add focused coverage for integration
+  retaining existing fields and application-derived explanations.
-risks not already protected:
+- A maintained complete example and canonical documentation describe native
-
+  inheritance, its field-validation implications, and the reserved-key
-- A child backend inherits endpoint, capacity, and a credential reference while
+  compatibility impact. Existing workflows and applicable offline checks pass.
-  retaining its own explicit overrides. Derived defaults respect inherited
-  presence, including an explicitly inherited zero queue capacity.
-- Selected overlays and explicit flags reach the final configuration under
-  ConfigKit's precedence rules. Omitted validation-role selection still derives
-  from the final proposal-role selection, while explicit invalid values fail
-  Audita's validation.
-- Surviving inherited credentials resolve and redact correctly. Null removes a
-  child's reference according to the existing credential contract; a surviving
-  reference on the parent remains subject to validation even if that backend is
-  not selected for a role.
-- A representative invalid relationship produces a configuration failure at
-  the CLI boundary. Reuse upstream coverage for the full generic inheritance
-  grammar rather than duplicating its cycle, nesting, and expansion test suite.
-
-Use synthetic credentials and real ConfigKit loading. Keep domain validation
-in Audita; inherited invalid settings must not become valid solely because the
-library successfully assembled them.
-
-### 3. Preserve Inspection and Runtime Wiring
-
-Check effective settings, explanation output, and diagnostic configuration
-snapshots with inherited values. Audita's current `ExplanationYAML` exposes
-source/layer information and application-derived defaults but does not expose
-ConfigKit's `Origin.Inheritance` evidence. Extend that projection with a compact,
-value-free explanation of inheritance, retaining existing fields and secret
-redaction. This is an explicit adoption deliverable; use upstream metadata
-without reconstructing ancestry from YAML or adding an unrelated inspection API.
-
-Use a representative controlled-provider workflow to verify that selected roles
-receive the inherited backend endpoint and credential and share its configured
-capacity. Reuse existing integration coverage where it already proves these
-properties; do not duplicate the full PromptKit runtime test matrix.
-
-### 4. Update Documentation, Examples, and Attribution
-
-When implementation lands, update [configuration](../config.md) with the
-reserved keyword, inheritance behavior, presence/default interactions, and
-credential implications. Link to upstream for the generic grammar. Update
-[configuration change recipes](../internal/cli-config.md) where contributor
-steps change, and the [CLI reference](../cli.md) only if its documented
-inspection behavior needs clarification.
-
-Add a small, complete inheritance example under [examples](../../examples/)
-using valid parent and child backends and existing PromptKit profile syntax.
-Validate it offline, alongside the maintained local, service, and testing-overlay
-examples, following the existing example-validation recipe.
-
-Check the selected release's license and notice files and update the ConfigKit
-entry in [attribution](../oss/attribution.md), including its commit-pinned source
-and notice links, under the [OSS policy](../policy/oss.md). The review found no
-change to ConfigKit's license or dependency declarations; verify the selected
-artifacts when making the dependency change.
-
-### 5. Validate and Close the Adoption Unit
-
-Run targeted configuration and CLI checks, then the normal build and complete
-offline test suite described in [development](../development.md). Validate
-changed examples and documentation links and run `git diff --check`. No live
-provider, paid model call, or new permanent release gate is needed.
-
-Record validation against the actual implementation revision and mark this
-roadmap complete only after the criteria below are met. The earlier disposable
-checkout result establishes that the unmodified suite passed with ConfigKit
-v0.5.0 and PromptKit v0.11.2; it does not cover newly documented inheritance
-workflows or substitute for final validation.
 
 ## Completion and Release Treatment
 

docs/roadmap/implementation.md

@@ -0,0 +1,236 @@
+# Implementation Plan: ConfigKit v0.5.0 Adoption
+
+**Last updated:** 2026-10-03
+**Status:** Ready for implementation.
+**Audience:** Luna coding agent.
+
+## Scope and Execution
+
+Implement the target state in the [ConfigKit roadmap](configkit.md), which owns
+feature intent, scope, and compatibility considerations. Execute the numbered
+stages below in order. Each stage is a bounded implementation prompt; the
+complete feature comprises all five stages. Commit each completed unit with a
+short, plain English message under the [agent policy](../policy/agents.md).
+No approval gate is needed between stages.
+
+Begin with [development guidance](../development.md) and the applicable agent,
+[architecture](../policy/architecture.md), [testing](../policy/testing.md),
+[documentation](../policy/documentation.md), and [OSS](../policy/oss.md)
+policies. Use graph-first discovery and read the relevant current contracts.
+The planning baseline is Audita commit
+`23e1df4cfb844d3fdeb1fb64483c1038cc4f2e21`; application code remains at the
+roadmap's reviewed revision. Target released ConfigKit v0.5.0, source commit
+`dc09959cca4e105f49ec316c6d4ba585bd2340bd`.
+
+### Selected Implementation Decisions
+
+- Keep `internal/core/config.Load` using ordinary `ck.Load`. Native inheritance
+  already runs before decoding and application resolution; another resolver or
+  `LoadLayered` conversion is unnecessary.
+- Keep PromptKit at v0.11.2 and its `/v2` catalogs at their existing versions.
+  Retain Go 1.25.5, configuration `version: 2`, existing profile syntax, role
+  selection, CLI flags, file discovery, overlays, and main-file-relative paths.
+- Preserve absence-based defaults. Inherited input is explicit input, including
+  zero, false, empty, and null; existing field rules still decide validity.
+  Every backend, including inheritance parents and unselected backends, remains
+  subject to ordinary validation and surviving credential resolution.
+- Preserve existing explanation fields and add an optional `inheritance`
+  sequence to each field record. Each step has `from`, `to`, and `reference`
+  string paths copied from `Origin.Inheritance` through `Path.String()`.
+  This additive, value-free projection serves both `--view explain` and debug
+  configuration origins through their existing shared function. It is not a
+  new inspection command or a complete upstream metadata serialization.
+- The `extends` keyword is reserved by the library throughout application input
+  mappings. Document its compatibility impact, including legacy backend IDs
+  named `extends`; do not add a compatibility translator or a second naming
+  restriction to application validation.
+
+Follow existing ownership boundaries. Production changes beyond the dependency
+and explanation projection are warranted only if focused integration checks
+expose a defect in the agreed behavior. Do not change report schemas, provider
+policy, debug enablement, or application validation to accommodate test fixtures.
+Do not add live-provider checks or permanent release gates. This plan does not
+select a release version, modify release policy, publish, tag, or push.
+
+## Stage 1 — Upgrade the Dependency and Its Attribution
+
+**Primary files:** `go.mod`, `go.sum`, `docs/oss/attribution.md`.
+
+1. Review the current dependency declarations and upgrade only ConfigKit:
+   `go get gitea.maximumdirect.net/eric/ConfigKit@v0.5.0`. Review the resolved
+   module diff; preserve PromptKit, both catalog modules, the Go minimum, and
+   unrelated dependency selections. Do not add a local replacement.
+2. Check the selected release's `COPYRIGHT`, `LICENSE`, and `go.mod` against the
+   pinned upstream revision. Update the ConfigKit attribution entry's version
+   and commit-pinned source and notice links. Preserve its verified copyright,
+   LGPL-3.0-or-later expression, and existing LGPL/GPL text links when unchanged.
+   Follow the existing House Format; no additional licensing machinery is needed.
+3. Leave the loader and profile integration intact. If the dependency introduces
+   a compilation failure, make only the API adjustment needed for this upgrade,
+   recording the actual difference rather than broadening scope.
+
+**Verification and completion:** Run
+`go test ./internal/core/config ./internal/cli ./internal/framework/llm` and
+`git diff --check`. Review the dependency and attribution diff, then commit.
+This establishes the upgraded baseline; it does not establish feature completion
+or replace the final suite in Stage 5.
+
+## Stage 2 — Verify Inherited Configuration, Defaults, and Credentials
+
+**Primary files:** `internal/core/config/config_test.go`; change
+`internal/core/config/config.go` only if these checks expose an integration defect.
+
+1. Read `Load`, its resolver, and domain validation. Existing backend capacity
+   defaults use `Resolution.IsExplicit`; omitted validation-role selection uses
+   `ck.Default` against the final proposal selection. Keep those mechanisms if
+   they already handle inherited presence correctly.
+2. Add a focused real-loader fixture with valid parent and child backends. The
+   child inherits an endpoint, concurrency, zero queue capacity, and a synthetic
+   credential reference. Check effective values and inherited explicitness;
+   in another case, let the child override concurrency. This protects against
+   replacing inherited capacity with Audita's absent-field defaults. Reuse
+   current tests for ordinary omitted defaults.
+3. Exercise precedence with a selected overlay that changes a parent's setting
+   and a child override that wins over its resolved parent. Preserve coverage
+   that an explicit proposal-profile override wins over the overlay and omitted
+   validation-profile selection derives from that final value. Use the existing
+   `TestOverlayOverrideAndInheritance` fixture where useful; its current
+   "inheritance" check covers role derivation, not native sibling inheritance.
+4. Extend credential coverage using controlled environment variables. Check that
+   an inherited reference resolves and that effective YAML excludes its resolved
+   value. A child's `credential: null` removes only that child's reference;
+   the parent's surviving reference still must resolve, even if the parent is
+   unselected. Check that an unset surviving reference fails and a missing
+   reference removed by final precedence is not resolved. Reuse
+   `TestSurvivingCredentials` rather than duplicating all secret cases.
+5. Include a representative inherited setting that violates existing Audita
+   validation, such as zero backend concurrency. Inheritance must not bypass
+   validation of either the parent or child. Existing explicit-empty/null role
+   validation cases remain applicable; extend them only if inherited presence
+   exposes an uncovered interaction.
+6. If a regression appears, correct the owning default or resolution integration
+   using ConfigKit presence/path APIs. Do not parse raw YAML, exempt parent
+   backends, or introduce abstract backend templates. Do not reproduce upstream
+   cycle, nesting, or expansion-limit tests.
+
+**Verification and completion:** Run `go test ./internal/core/config` and
+`git diff --check`. Review assertions for new validation rules or duplicated
+library tests, then commit the focused configuration unit.
+
+## Stage 3 — Expose Compact Inheritance Evidence in Safe Inspection
+
+**Primary files:** `internal/framework/llm/safe_view.go`,
+`internal/framework/llm/runtime_inspection_test.go`, `docs/cli.md`, and the
+relevant explanation description in `docs/internal/llm-runtime.md` if needed.
+
+1. Extend `explanationView` with `inheritance,omitempty` and a small step view
+   containing `from`, `to`, and `reference`. For effective field records, project
+   `info.Origin.Inheritance` in its upstream order. Preserve existing `path`,
+   `source`, `layer`, and application `derivations` behavior. Do not serialize
+   field values, whole `Explanation` objects, resolved credentials, or arbitrary
+   configuration content into this view.
+2. Continue to use `Metadata.Lookup`, `HistoricalPaths`, and existing derivation
+   mapping. Do not infer ancestry from labels or YAML, flatten a chain to one
+   guessed parent, or reinterpret inheritance as an application-derived default.
+   This addition describes effective inheritance; complete displaced ancestry
+   and upstream source-coordinate reporting are not new requirements.
+3. Test decoded YAML for one inherited field's source/destination/reference paths
+   and a child field that directly overrides its parent. Include an inherited
+   credential reference with a distinctive synthetic resolved value and prove
+   that value is absent. Retain existing application-derived role explanations
+   and safe profile-view coverage. Assert meaningful records, not complete
+   YAML snapshots or upstream-generated identifiers.
+4. Check the existing shared callers: CLI explain uses `ExplanationYAML`, and
+   `processState.startDebug` uses it for configuration origins. Keep this shared
+   path; no separate debug projection is needed. Update the CLI contract with
+   the optional inheritance step fields and their value-free meaning once they
+   are implemented. Link internal documentation to that contract for exact shape.
+
+**Verification and completion:** Run
+`go test ./internal/framework/llm ./internal/cli` and `git diff --check`.
+Review emitted fields and affected documentation, then commit the inspection unit.
+
+## Stage 4 — Verify CLI Failures and Runtime Use of Inherited Backends
+
+**Primary files:** `internal/framework/llm/adapter_capacity_test.go`,
+`internal/cli/config_input_test.go`, and
+`cmd/audita/main_integration_test.go`. Touch production wiring only to fix a
+proven integration defect.
+
+1. Reuse `TestRolesShareBackendCapacityAndReleaseAfterCancellation` or its
+   existing fixture to load a valid parent and a selected child backend through
+   `config.Load`. Both existing-format role profiles select that child. Inherit
+   the controlled server endpoint, a test-selected positive capacity, zero
+   queue capacity, and a synthetic credential; keep the parent valid too.
+2. Verify actual requests reach the inherited endpoint and carry the inherited
+   credential. Existing overlapping role-call coverage proves that the two
+   roles share the child's capacity. Preserve that check with inherited input
+   and its release/cancellation coverage; do not build another scheduler or
+   duplicate the provider retry/repair matrix. Use synchronization already in
+   the test, not sleeps to infer admission behavior.
+3. Add one representative invalid relationship at the CLI boundary: a missing
+   sibling parent should yield configuration exit code `2` and no provider
+   dispatch. Use the existing subprocess fixture and semantic diagnostic
+   assertions; do not freeze full error text or invent early-failure reports.
+4. Extend one representative subprocess success/inspection case to use inherited
+   backend input. With debug explicitly enabled, check that effective settings
+   and configuration-origin artifacts retain inherited settings and the new
+   explanation evidence without the resolved synthetic secret. Check the
+   explain view through the existing CLI route. Use existing coverage for
+   unchanged report placement, ordinary processing, and debug-disabled behavior.
+   Keep assertions on the affected fields rather than the entire artifact set.
+
+**Verification and completion:** Run
+`go test ./internal/framework/llm ./internal/cli ./cmd/audita` and
+`git diff --check`. Review whether each added test covers a distinct integration
+risk, then commit the runtime/CLI verification unit.
+
+## Stage 5 — Document Native Inheritance and Close the Adoption Unit
+
+**Primary files:** `docs/config.md`, `examples/configkit-inheritance.yml`,
+`docs/internal/cli-config.md`, and `docs/roadmap/configkit.md`. Update another
+canonical document only where an implemented statement actually changes.
+
+1. Add `examples/configkit-inheritance.yml` beside the existing examples, with
+   `version: 2`, a valid `local-base` backend, and a valid `local` backend that
+   extends it. Inherit a local endpoint and zero queue capacity; use a positive
+   parent concurrency and a child concurrency override. Select the existing
+   `local-proposal` and `local-validation` profiles under `profiles/` and the
+   grammar module. The example needs no credential or new profile format and
+   can be validated offline without a running local server.
+2. Update the configuration contract to describe native sibling inheritance,
+   precedence before inheritance, inherited explicit presence, null credential
+   removal, and normal parent validation. Link the maintained example and the
+   version-pinned upstream generic grammar. Retain current file/overlay/override
+   behavior and main-file-relative path rules. Do not imply that ordinary
+   PromptKit v0.11.2 profile files now accept ConfigKit `extends` syntax.
+3. Document that `extends` is reserved across input mappings. Explain the legacy
+   backend-name conflict and migration: rename the backend to a non-conflicting
+   ID and update every profile selecting it. Keep application `version: 2`;
+   do not claim all previously valid configurations are compatible. Record the
+   compatibility finding in the feature roadmap for later release preparation.
+   Release classification remains outside this implementation unit, governed by
+   the existing release policy; no policy exception is selected here.
+4. Add the new example's offline validation command to the existing example
+   recipe. Validate it and the maintained minimal, production, and selected
+   testing-overlay examples, with a controlled synthetic service credential.
+   Inspect the new example's settings and explanation output. No live model
+   request is needed to check configuration or profiles.
+5. Run `go build -o ./bin/audita ./cmd/audita`, `go test ./...`, affected-link
+   checks, and `git diff --check`. Review the final dependency graph and diff:
+   ConfigKit is the released v0.5.0, PromptKit/catalogs retain their selections,
+   source attribution is pinned, and no unrelated settings or limits were added.
+   Keep generated binaries out of the commit. Resolve actual failures rather
+   than adding gates unrelated to this feature.
+6. Mark the feature roadmap and this plan complete only after all stage outcomes
+   pass. Record the tested source revision and checks accurately; if a final
+   documentation-only completion commit follows validation, distinguish it from
+   the tested implementation revision. Review and commit the final documentation
+   unit, leaving a clean workspace. Reuse successful checks for unchanged content.
+
+**End state:** Native ConfigKit inheritance reaches typed settings, safe
+inspection, selected runtime backends, and maintained operator examples;
+PromptKit migration remains a separate unit. No implementation-blocking design
+questions remain. The eventual release version and treatment of the known
+compatibility impact remain release-preparation decisions, not prerequisites
+for these stages.