From 6e5e277426c0ebbc4b41d961d6f12f99d53e5402 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sat, 3 Oct 2026 17:16:41 +0000 Subject: [PATCH] Plan ConfigKit inheritance support --- docs/roadmap/configkit.md | 227 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 227 insertions(+) create mode 100644 docs/roadmap/configkit.md diff --git a/docs/roadmap/configkit.md b/docs/roadmap/configkit.md new file mode 100644 index 00000000..eced84a8 --- /dev/null +++ b/docs/roadmap/configkit.md @@ -0,0 +1,227 @@ +# ConfigKit 0.5 And Configuration Inheritance + +## Purpose + +Adopt ConfigKit v0.5.0 and make its sibling inheritance useful and supported in +Notarius configuration, particularly for deriving development or testing +pipelines from a complete D&D pipeline without duplicating the pipeline body. + +ConfigKit owns source composition and inheritance. Notarius continues to own +typed configuration, domain defaults and validation, pipeline resolution, and +execution. This work does not introduce a second configuration-merging engine. + +This is a feature roadmap, not an implementation sequence or release approval. + +## Reviewed Baseline + +Reviewed on 2026-10-03. Notarius pins ConfigKit v0.4.2 and uses its ordinary +`Load` API in `internal/core/config/load.go`. Configuration version 5 supports +imports, selected configuration overlays, explicit command overrides, recorded +normalization, and redacted effective/explain output. Runtime configuration is +detached from the ConfigKit inspection snapshot. + +Named pipelines and artifact lanes are mappings; pipeline `steps` and validator +chains are sequences. The complete example imports +`examples/pipelines/dnd-session.yml`, while the existing development overlay +changes operational settings. Relative configured paths are resolved from the +file supplying their surviving declaration. The current path resolver uses +ConfigKit origin/source metadata and host filesystem paths. + +Target ConfigKit v0.5.0 at full source commit +`dc09959cca4e105f49ec316c6d4ba585bd2340bd`. Consult the pinned +[release guide](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/docs/releases/v0.5.0.md) +and [configuration contract](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/docs/config.md). +The release adds inheritance, layered loading, filesystem-backed sources, +mapping sources, and passive resource contexts. Its Go requirement and YAML +dependency are unchanged. The upgrade has not yet been trialed against +Notarius's test suite at this review baseline. + +## Target End State + +### Native Sibling Inheritance + +Operators can define a named pipeline with `extends: parent-pipeline` and +override only the settings that differ. The parent remains an ordinary named, +valid pipeline; no abstract-template flag, template registry, or special +validation exemption is introduced. Derived pipelines use the existing run, +listing, validation, effective-configuration, and explanation commands. + +Use ConfigKit's native semantics without restricting inheritance to pipelines +or implementing a Notarius-specific subset: + +- `extends` selects one exact key among the mapping's immediate siblings. + It is not a path, cross-scope lookup, or multiple-parent expression. +- Source precedence is assembled before inheritance, and inheritance runs + before typed decoding and application resolution. Inherited input is explicit + for presence-sensitive defaults, including zero, false, empty, and null. +- Child mappings recursively overlay parent mappings; sequences, scalars, + and null replace inherited values. Existing typed/domain validation still + determines whether the resulting value is valid. +- Omitting `extends` retains a lower-priority relationship; `extends: null` + cancels the relationship, not arbitrary inherited fields independently. +- Root and sequence-element mappings have no sibling namespace for their own + string relationship. Nested mappings may have one. In particular, individual + entries of the `steps` sequence cannot extend other step entries. +- ConfigKit rejects missing/non-mapping parents, invalid directive types, and + cycles, and resolves effective relationships even in unselected entries. +- Parent definitions receive normal typed decoding and Notarius validation. + Inheritance is not a way to hide invalid partial pipeline definitions. +- Inherited nested relationships resolve at their destination according to + ConfigKit's rules; do not pre-expand or reinterpret them in Notarius. + +Names are matched by ConfigKit before Notarius's existing identifier trimming. +Document exact-key lookup rather than introducing pre-inheritance name rewriting. + +The principal documented workflow is whole-pipeline reuse. Inheritance does not +append steps, merge sequences by `id`, or allow a child to override one list +element without supplying a replacement list. The same applies to validator +chains. Do not disguise this boundary through custom list-merging code. + +### Paths, Provenance, And Runtime Boundaries + +An inherited relative reference or resource path remains relative to its +original declaring file, not the child's file or the invocation directory. +Imported parents, overlays, and nested inheritance must retain that behavior. +Direct child declarations use their own source context. Built-in relative +destinations and CLI-supplied paths keep their existing bases. + +Review `documentPathBase` and related original-path/normalization handling +against the new origin metadata. Make narrowly required corrections if needed; +do not assume that successful typed decoding proves correct path resolution. +Effective output must agree with the runtime paths, and explanations must retain +the upstream derivation evidence through recorded Notarius normalization. + +Keep the ConfigKit boundary private: no upstream inheritance objects or +unresolved directives reach runtime pipeline/module types. ConfigKit expands +relationships once during loading. Notarius's existing binding-profile, +validation-policy, and module-default resolution still operate on the resulting +configuration; they are not replaced with generic sibling inheritance. + +Configuration overlays, pipeline inheritance, and PromptKit `base_profile` +inheritance remain distinct mechanisms. The latter still belongs to PromptKit. + +### Compatibility And Upgrade Policy + +Keep Notarius configuration `version: 5`. The typed runtime shape is unchanged; +the new capability belongs to configuration composition. No new CLI flag or +inheritance opt-in setting is needed or available upstream. + +The significant compatibility change is that `extends` becomes a reserved key +in every input mapping, including dynamic module-option maps and named maps. +It is consumed before typed decoding, so it can no longer be ordinary data or +an ordinary entry name there. Audit maintained configuration/fixtures for such +usage and document that affected operator configurations must rename the key +or identifier and any associated consumer field. Do not silently strip, alias, +or bypass the directive to preserve an incompatible interpretation. + +Configurations without that collision should retain their existing meaning. +Preserve discovery, import rules, overlay selection, explicit CLI override +precedence, strict decoding, redaction, defaults, and validation. Existing +configuration-stage inheritance does not alter the fixed execution pipeline, +artifact contracts, warning policy, or checkpoint identity design. + +The eventual Notarius release should be a new minor version, explaining both +the feature and the newly reserved input key. Release preparation/publication +is separate from this work; no exact release number is selected here. + +## Required Work + +### Dependency And Configuration Integration + +Upgrade `go.mod` and `go.sum` to the exact target. Keep PromptKit and unrelated +dependencies unchanged unless dependency resolution demonstrably requires it. +Continue using ordinary `configkit.Load`, the existing typed defaults factory, +and source-aware resolution/validation callbacks. + +Inspect the load, operational-document, path, conversion, inspection, and CLI +boundaries. Add only integration adjustments demonstrated necessary by the +upstream change. There should be no `Extends` field added to Notarius runtime +types and no homegrown inheritance algorithm. + +Update ConfigKit's factual attribution version and COPYRIGHT/LICENSE/source +links to the target commit after inspecting its actual notices. Preserve the +selected project grant and reuse unchanged local license texts. + +### Maintained Example + +Extend the complete example's imported pipeline definitions with a small +`dnd-session-development` sibling deriving from `dnd-session`, demonstrating a +pipeline-level setting override such as a smaller structural-repair budget. +Use already supported values and existing profile sources so the example +validates without an invented model/profile dependency. Explain that +`llm_profile` is another useful override when the operator supplies that profile. + +Retain the runnable parent, minimal example, and existing operational overlay. +Do not duplicate the complete steps list or add a third standalone configuration +merely to demonstrate inheritance. Update example-loading/listing expectations +for the additional pipeline, checking inherited behavior rather than snapshots +of incidental formatting or total line counts. + +### Canonical Documentation + +- `docs/config.md`: own syntax, composition order, exact sibling lookup, + replacement/null semantics, path bases, reserved-key migration, and limits + on step/validator-list composition. Link to the maintained example and pinned + upstream rules rather than duplicating the entire upstream manual. +- `docs/internal/configuration.md`: describe expansion before typed decoding, + presence and source-aware normalization, and the unchanged runtime boundary. +- `docs/integrations/pkg-configkit.md`: update the supported version, public + contract links, and integration scope. +- `docs/oss/attribution.md`: update the dependency's verified source/notice links. +- `docs/cli.md` and consumer guides: update only where navigation or examples + benefit from showing selection of the derived pipeline. Do not invent new + invocation syntax or duplicate the configuration reference. + +The existing ConfigKit architecture decision remains applicable. No new ADR is +required solely for consuming the library's composition feature; do not rewrite +an accepted ADR or turn new private implementation details into invariants. + +## Verification And Acceptance + +Use focused Notarius boundary tests, not a duplicate of ConfigKit's full suite: + +- Existing maintained configurations continue loading and resolving. +- A derived pipeline retains its own identifier and inherits the complete + parent pipeline while applying its explicit overrides. +- Imports and selected overlays participate in composition before inheritance; + explicit CLI overrides keep their documented precedence. +- Representative zero/false/null inputs exercise existing presence-sensitive + defaulting or validation correctly; null is not universally valid merely + because inheritance can represent it. +- Replacing a steps or validator sequence does not concatenate or merge it. +- References inherited from a parent in another file retain their declaring + directory, while child-supplied replacements use the child's directory. +- Effective and explanation output remain redacted and expose useful source + and inheritance information after application normalization. +- Representative invalid inheritance produces ConfigKit load diagnostics through + the normal Notarius CLI error path, without a parallel validation engine. +- Maintained examples and their imported fragments validate offline without + credentials or provider calls. + +Start with configuration and CLI tests, including document/path/operational, +effective/explanation, and maintained-example coverage. Add framework-resolution +coverage only where a distinct handoff risk requires it. Before handoff, run +the offline default suite, vet, and build with `GOWORK=off`, plus dependency +tidiness and documentation diff/link checks appropriate to the changes. +These are upgrade validation activities, not additional permanent release gates. + +## Deferred Capabilities And Non-Goals + +- Do not adopt `LoadLayered` or add a new underlay precedence tier now. Existing + imports, overlays, and typed defaults remain the loading model. Bundled + pipeline presets or site-wide default layers need a separate operator-facing + design if selected later. +- Do not move Go defaults into embedded YAML or mapping sources solely because + v0.5.0 provides those APIs. +- Do not generalize references to arbitrary `fs.FS` resources in this work. + `SourceContext` may be used if it materially helps the required path behavior, + but its filesystem access rules must not silently replace Notarius's existing + host-path contract or introduce new path restrictions. +- No abstract templates, multiple inheritance, list-element composition, + new template schema, new pipeline scheduler, or extra configuration knobs. +- No opt-in gate or Notarius-only prohibition on other valid native inheritance + locations; use upstream semantics and existing domain validation. + +There are no significant open design decisions at this baseline. Implementation +should verify the new metadata behavior and correct concrete integration gaps +without broadening the agreed feature scope.