Plan ConfigKit inheritance support

docs/roadmap/configkit.md

@@ -0,0 +1,227 @@
+# ConfigKit 0.5 And Configuration Inheritance
+
+## Purpose
+
+Adopt ConfigKit v0.5.0 and make its sibling inheritance useful and supported in
+Notarius configuration, particularly for deriving development or testing
+pipelines from a complete D&D pipeline without duplicating the pipeline body.
+
+ConfigKit owns source composition and inheritance. Notarius continues to own
+typed configuration, domain defaults and validation, pipeline resolution, and
+execution. This work does not introduce a second configuration-merging engine.
+
+This is a feature roadmap, not an implementation sequence or release approval.
+
+## Reviewed Baseline
+
+Reviewed on 2026-10-03. Notarius pins ConfigKit v0.4.2 and uses its ordinary
+`Load` API in `internal/core/config/load.go`. Configuration version 5 supports
+imports, selected configuration overlays, explicit command overrides, recorded
+normalization, and redacted effective/explain output. Runtime configuration is
+detached from the ConfigKit inspection snapshot.
+
+Named pipelines and artifact lanes are mappings; pipeline `steps` and validator
+chains are sequences. The complete example imports
+`examples/pipelines/dnd-session.yml`, while the existing development overlay
+changes operational settings. Relative configured paths are resolved from the
+file supplying their surviving declaration. The current path resolver uses
+ConfigKit origin/source metadata and host filesystem paths.
+
+Target ConfigKit v0.5.0 at full source commit
+`dc09959cca4e105f49ec316c6d4ba585bd2340bd`. Consult the pinned
+[release guide](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/docs/releases/v0.5.0.md)
+and [configuration contract](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/docs/config.md).
+The release adds inheritance, layered loading, filesystem-backed sources,
+mapping sources, and passive resource contexts. Its Go requirement and YAML
+dependency are unchanged. The upgrade has not yet been trialed against
+Notarius's test suite at this review baseline.
+
+## Target End State
+
+### Native Sibling Inheritance
+
+Operators can define a named pipeline with `extends: parent-pipeline` and
+override only the settings that differ. The parent remains an ordinary named,
+valid pipeline; no abstract-template flag, template registry, or special
+validation exemption is introduced. Derived pipelines use the existing run,
+listing, validation, effective-configuration, and explanation commands.
+
+Use ConfigKit's native semantics without restricting inheritance to pipelines
+or implementing a Notarius-specific subset:
+
+- `extends` selects one exact key among the mapping's immediate siblings.
+  It is not a path, cross-scope lookup, or multiple-parent expression.
+- Source precedence is assembled before inheritance, and inheritance runs
+  before typed decoding and application resolution. Inherited input is explicit
+  for presence-sensitive defaults, including zero, false, empty, and null.
+- Child mappings recursively overlay parent mappings; sequences, scalars,
+  and null replace inherited values. Existing typed/domain validation still
+  determines whether the resulting value is valid.
+- Omitting `extends` retains a lower-priority relationship; `extends: null`
+  cancels the relationship, not arbitrary inherited fields independently.
+- Root and sequence-element mappings have no sibling namespace for their own
+  string relationship. Nested mappings may have one. In particular, individual
+  entries of the `steps` sequence cannot extend other step entries.
+- ConfigKit rejects missing/non-mapping parents, invalid directive types, and
+  cycles, and resolves effective relationships even in unselected entries.
+- Parent definitions receive normal typed decoding and Notarius validation.
+  Inheritance is not a way to hide invalid partial pipeline definitions.
+- Inherited nested relationships resolve at their destination according to
+  ConfigKit's rules; do not pre-expand or reinterpret them in Notarius.
+
+Names are matched by ConfigKit before Notarius's existing identifier trimming.
+Document exact-key lookup rather than introducing pre-inheritance name rewriting.
+
+The principal documented workflow is whole-pipeline reuse. Inheritance does not
+append steps, merge sequences by `id`, or allow a child to override one list
+element without supplying a replacement list. The same applies to validator
+chains. Do not disguise this boundary through custom list-merging code.
+
+### Paths, Provenance, And Runtime Boundaries
+
+An inherited relative reference or resource path remains relative to its
+original declaring file, not the child's file or the invocation directory.
+Imported parents, overlays, and nested inheritance must retain that behavior.
+Direct child declarations use their own source context. Built-in relative
+destinations and CLI-supplied paths keep their existing bases.
+
+Review `documentPathBase` and related original-path/normalization handling
+against the new origin metadata. Make narrowly required corrections if needed;
+do not assume that successful typed decoding proves correct path resolution.
+Effective output must agree with the runtime paths, and explanations must retain
+the upstream derivation evidence through recorded Notarius normalization.
+
+Keep the ConfigKit boundary private: no upstream inheritance objects or
+unresolved directives reach runtime pipeline/module types. ConfigKit expands
+relationships once during loading. Notarius's existing binding-profile,
+validation-policy, and module-default resolution still operate on the resulting
+configuration; they are not replaced with generic sibling inheritance.
+
+Configuration overlays, pipeline inheritance, and PromptKit `base_profile`
+inheritance remain distinct mechanisms. The latter still belongs to PromptKit.
+
+### Compatibility And Upgrade Policy
+
+Keep Notarius configuration `version: 5`. The typed runtime shape is unchanged;
+the new capability belongs to configuration composition. No new CLI flag or
+inheritance opt-in setting is needed or available upstream.
+
+The significant compatibility change is that `extends` becomes a reserved key
+in every input mapping, including dynamic module-option maps and named maps.
+It is consumed before typed decoding, so it can no longer be ordinary data or
+an ordinary entry name there. Audit maintained configuration/fixtures for such
+usage and document that affected operator configurations must rename the key
+or identifier and any associated consumer field. Do not silently strip, alias,
+or bypass the directive to preserve an incompatible interpretation.
+
+Configurations without that collision should retain their existing meaning.
+Preserve discovery, import rules, overlay selection, explicit CLI override
+precedence, strict decoding, redaction, defaults, and validation. Existing
+configuration-stage inheritance does not alter the fixed execution pipeline,
+artifact contracts, warning policy, or checkpoint identity design.
+
+The eventual Notarius release should be a new minor version, explaining both
+the feature and the newly reserved input key. Release preparation/publication
+is separate from this work; no exact release number is selected here.
+
+## Required Work
+
+### Dependency And Configuration Integration
+
+Upgrade `go.mod` and `go.sum` to the exact target. Keep PromptKit and unrelated
+dependencies unchanged unless dependency resolution demonstrably requires it.
+Continue using ordinary `configkit.Load`, the existing typed defaults factory,
+and source-aware resolution/validation callbacks.
+
+Inspect the load, operational-document, path, conversion, inspection, and CLI
+boundaries. Add only integration adjustments demonstrated necessary by the
+upstream change. There should be no `Extends` field added to Notarius runtime
+types and no homegrown inheritance algorithm.
+
+Update ConfigKit's factual attribution version and COPYRIGHT/LICENSE/source
+links to the target commit after inspecting its actual notices. Preserve the
+selected project grant and reuse unchanged local license texts.
+
+### Maintained Example
+
+Extend the complete example's imported pipeline definitions with a small
+`dnd-session-development` sibling deriving from `dnd-session`, demonstrating a
+pipeline-level setting override such as a smaller structural-repair budget.
+Use already supported values and existing profile sources so the example
+validates without an invented model/profile dependency. Explain that
+`llm_profile` is another useful override when the operator supplies that profile.
+
+Retain the runnable parent, minimal example, and existing operational overlay.
+Do not duplicate the complete steps list or add a third standalone configuration
+merely to demonstrate inheritance. Update example-loading/listing expectations
+for the additional pipeline, checking inherited behavior rather than snapshots
+of incidental formatting or total line counts.
+
+### Canonical Documentation
+
+- `docs/config.md`: own syntax, composition order, exact sibling lookup,
+  replacement/null semantics, path bases, reserved-key migration, and limits
+  on step/validator-list composition. Link to the maintained example and pinned
+  upstream rules rather than duplicating the entire upstream manual.
+- `docs/internal/configuration.md`: describe expansion before typed decoding,
+  presence and source-aware normalization, and the unchanged runtime boundary.
+- `docs/integrations/pkg-configkit.md`: update the supported version, public
+  contract links, and integration scope.
+- `docs/oss/attribution.md`: update the dependency's verified source/notice links.
+- `docs/cli.md` and consumer guides: update only where navigation or examples
+  benefit from showing selection of the derived pipeline. Do not invent new
+  invocation syntax or duplicate the configuration reference.
+
+The existing ConfigKit architecture decision remains applicable. No new ADR is
+required solely for consuming the library's composition feature; do not rewrite
+an accepted ADR or turn new private implementation details into invariants.
+
+## Verification And Acceptance
+
+Use focused Notarius boundary tests, not a duplicate of ConfigKit's full suite:
+
+- Existing maintained configurations continue loading and resolving.
+- A derived pipeline retains its own identifier and inherits the complete
+  parent pipeline while applying its explicit overrides.
+- Imports and selected overlays participate in composition before inheritance;
+  explicit CLI overrides keep their documented precedence.
+- Representative zero/false/null inputs exercise existing presence-sensitive
+  defaulting or validation correctly; null is not universally valid merely
+  because inheritance can represent it.
+- Replacing a steps or validator sequence does not concatenate or merge it.
+- References inherited from a parent in another file retain their declaring
+  directory, while child-supplied replacements use the child's directory.
+- Effective and explanation output remain redacted and expose useful source
+  and inheritance information after application normalization.
+- Representative invalid inheritance produces ConfigKit load diagnostics through
+  the normal Notarius CLI error path, without a parallel validation engine.
+- Maintained examples and their imported fragments validate offline without
+  credentials or provider calls.
+
+Start with configuration and CLI tests, including document/path/operational,
+effective/explanation, and maintained-example coverage. Add framework-resolution
+coverage only where a distinct handoff risk requires it. Before handoff, run
+the offline default suite, vet, and build with `GOWORK=off`, plus dependency
+tidiness and documentation diff/link checks appropriate to the changes.
+These are upgrade validation activities, not additional permanent release gates.
+
+## Deferred Capabilities And Non-Goals
+
+- Do not adopt `LoadLayered` or add a new underlay precedence tier now. Existing
+  imports, overlays, and typed defaults remain the loading model. Bundled
+  pipeline presets or site-wide default layers need a separate operator-facing
+  design if selected later.
+- Do not move Go defaults into embedded YAML or mapping sources solely because
+  v0.5.0 provides those APIs.
+- Do not generalize references to arbitrary `fs.FS` resources in this work.
+  `SourceContext` may be used if it materially helps the required path behavior,
+  but its filesystem access rules must not silently replace Notarius's existing
+  host-path contract or introduce new path restrictions.
+- No abstract templates, multiple inheritance, list-element composition,
+  new template schema, new pipeline scheduler, or extra configuration knobs.
+- No opt-in gate or Notarius-only prohibition on other valid native inheritance
+  locations; use upstream semantics and existing domain validation.
+
+There are no significant open design decisions at this baseline. Implementation
+should verify the new metadata behavior and correct concrete integration gaps
+without broadening the agreed feature scope.