Retire completed plans and outline named inheritance

docs/roadmap/named-inheritance.md

@@ -0,0 +1,281 @@
+# Opt-In Named Configuration Inheritance
+
+Status: proposed feature design for agreement; not implemented. This document
+defines the intended end state and the remaining architectural decisions. A
+separate implementation plan will define execution steps after design agreement.
+
+## Purpose and priorities
+
+Allow an application to designate a collection of named configuration entries
+whose members can inherit settings from one parent in that collection. ConfigKit
+resolves the relationships during loading and returns final typed configuration
+with inspectable provenance. Applications retain ownership of field meanings,
+domain validation, and execution.
+
+The [downstream request](feature-request-promptkit-inheritance.md) supplies a
+concrete need for reusable named defaults and chains of specialization. ConfigKit
+adopts a shared structural model, using its existing recursive combination and
+presence conventions. The design serves named presets, connection settings, or
+similar configuration entries without interpreting their domain.
+
+Priorities are consistency and predictability, accurate presence/provenance, and
+reduced consumer configuration infrastructure. Computational efficiency matters
+within those goals: resolve shared ancestors once and account for the size of
+materialized results and retained history. Avoid an unconditional O(n) promise.
+
+## Current baseline
+
+Baseline: `b604aa167b684525f0d5d806d9c4267824151ee4`, after the filesystem-source
+and underlay implementation and its remediation.
+
+- The [configuration contract](../config.md) supports host/native filesystem,
+  byte, reader, and snapshotted mapping sources through a shared engine.
+- Independent contributions compose strictly within a tier; explicitly ordered
+  tiers combine using recursive overlay. Typed defaults sit below source input;
+  selected overlays and explicit CLI overrides retain their documented roles.
+- Named application entries can already be represented as string-keyed mappings.
+  ConfigKit currently treats relationships between their keys as application data.
+- Recorded application resolution runs after typed decoding and preserves
+  dependencies and provenance. Consumers currently own relationship resolution;
+  there is no built-in pre-decode inheritance phase.
+- Source contexts support later application-owned resource access. Loading is
+  synchronous and complete on return, with library-owned loading handles closed.
+
+This feature is a deliberate extension of structural input processing. It follows
+the [architecture goals](../policy/architecture.md#success-criteria), the
+composition distinction in [ADR 0001](../adr/0001-separate-composition-and-overlay.md),
+the presence/provenance principles in
+[ADR 0003](../adr/0003-preserve-presence-and-resolution-history.md), and the
+complete-on-return design in [draft ADR 0008](../adr/0008-complete-on-return-loading.md).
+
+## Target configuration model
+
+### Explicit collection declarations
+
+Applications opt in particular string-keyed collections through their reusable
+configuration definition, consistently for ordinary and layered loading. Each
+mapping key is the entry's identity. References use exact names within the
+declared collection, independent of filesystem namespaces or declaration order.
+One entry can name one parent; chains and shared parents are supported.
+
+The declaration identifies structural metadata rather than enabling callbacks
+that choose arbitrary lookup or merge strategies. Ordinary fields remain ordinary
+data outside declared collections. The declaration syntax and treatment of the
+parent-reference field are addressed in Open Decisions below.
+
+Mappings with no parent remain independent entries. Explicit null or scalar
+entries retain their ordinary data meaning and are subject to normal final
+schema validation; an entry used as a parent must resolve to a mapping so its
+settings can combine with the child. Missing collections contribute no entries.
+A present collection must have the mapping shape required by its declaration.
+
+Identity comes from the collection key, not a copied field in the parent payload.
+The child keeps its key. Application fields such as `name` or `id` have ordinary
+combination semantics unless consumed as explicitly declared structural metadata.
+
+### Source precedence first, inheritance second
+
+First assemble the effective input under the existing source rules: strict
+composition within tiers and recursive overlay across underlays, main input,
+selected overlay, and explicit CLI overrides. Strict conflicts remain errors.
+Then resolve inheritance against the resulting effective collection.
+
+Every parent reference sees that parent's effective definition, including source
+overrides. For each entry, resolve the parent's chain, then recursively overlay
+the child's own effective payload. Each child obtains independent effective
+settings; resolving a child leaves its parent and siblings unchanged.
+
+Source precedence and ancestry answer different questions. Source precedence
+selects each entry's own settings and parent reference. Ancestry supplies settings
+that entry omits. For example, if a child's own timeout is `10`, changing its
+parent's timeout to `30` through higher-priority source input leaves the child's
+explicit `10` intact. To change that explicit child value, override the child.
+This applies equally to CLI overrides at those respective paths.
+
+### One combination contract
+
+Parent-to-child combination uses the existing recursive mapping overlay:
+
+- Omitted children retain inherited values.
+- Present zero, false, and empty strings replace inherited values.
+- Mappings combine recursively; an empty mapping retains inherited children.
+- Sequences replace as a unit, including empty sequences.
+- Explicit null replaces the inherited value, subject to final schema acceptance.
+- Cross-kind replacement follows the same overlay rules and final schema checks.
+
+The parent's inheritance-control metadata is not inherited as a payload setting.
+The child supplies its own relationship. Inherited payload values can themselves
+contain ordinary strings or fields with similar names without acquiring reference
+semantics outside the declared position.
+
+These rules intentionally differ from the request's suggested null-as-unspecified
+and whole-parameter-map replacement. Consumers adopting the feature adopt
+ConfigKit's shared rules. Multiple inheritance, sequence merging, per-field
+strategies, deletion operators, cross-collection references, and named-overlay
+file inheritance are outside this feature's initial scope.
+
+## Loading and validation
+
+The proposed lifecycle is: existing acquisition/composition and main-owned version
+checking; finish source precedence assembly; resolve named inheritance; apply
+typed defaults and decode; recorded application resolution; secret lookup;
+semantic validation; publish the final configuration and metadata.
+
+Resolve inheritance in the private input representation while omission is still
+explicit. Ordinary and layered loading use the same implementation. Typed defaults
+remain the final decoder's baseline and are not converted into parent declarations
+or recursively copied from parent Go values. Domain resolution receives already
+expanded settings and can apply recorded normalization/defaulting under its
+existing contract.
+
+Preserve the existing version owner's declaration and checking point. Inheritance
+must not supply, replace, or remove the protected version path or bypass ancestor
+protections. Identify attempted protected-path changes explicitly during design
+of declarations rather than moving the checker to a different lifecycle point.
+
+All declared relationships resolve during the load, including entries the
+application might not select for execution. Missing parents, non-mapping parents,
+invalid reference values, self-reference, and longer cycles fail the load with
+safe structured diagnostics and relevant source locations. Traverse deterministically;
+diagnostics should identify the relationship and chain rather than expose payload
+values. Synthetic inputs retain labels and semantic paths without fabricated file
+coordinates. Normal syntax and strict-composition errors remain unmaskable.
+
+Partial base entries may omit domain-required settings. Omission is not by itself
+a structural error. All returned entries still undergo ordinary typed decoding;
+application validation determines which entries need to be independently executable
+and which are reusable partial definitions. This adds no alternate schema,
+automatic abstract-entry marker, or general exemption from validation.
+
+Successful loading yields the completed configuration and metadata. Inheritance
+performs no background resolution and retains no filesystem handles. Applications
+may load again explicitly using the existing source-reuse contracts.
+
+## Presence, provenance, and confidentiality
+
+Consumers must distinguish direct child input, inherited input, typed defaults,
+and recorded application-derived values. Explicit zero/false/empty/null settings
+remain distinguishable from omission. The representation of inherited presence
+in the existing query surface is an open decision below; Go zero-value inference
+is not an acceptable substitute.
+
+Each inherited value retains its originating source and source path, together
+with the relationship that placed it at the child's destination path. Chains
+remain explainable, including a child's explicit replacement of an inherited
+value. Source-precedence history remains inspectable alongside inheritance
+derivation; inheritance must not masquerade as another acquired file or source
+tier. Effective metadata, resolver-entry queries, history, explanation, and
+redacted rendering must agree.
+
+A relative resource inherited from a parent keeps the context of the source
+that supplied that resource value. It does not acquire the child's directory.
+Composite origins remain composite and derived dependencies remain explicit.
+
+Inherit secret references as unresolved configuration input; resolve typed
+credentials only at the existing lifecycle point. Preserve source and destination
+sensitivity through inheritance dependencies so copying a classified value cannot
+make it public. A child's independent replacement has its own provenance; inherited
+or displaced historical secrets remain safely explainable and redacted. No raw
+credential values belong in inheritance trees, metadata, or diagnostics.
+
+## Efficiency and ownership
+
+Model each collection as entries and single-parent edges. Use indexed name lookup,
+cycle detection, and memoized parent resolution to avoid repeated ancestor work.
+Materialize independent public results while allowing carefully owned immutable
+private structures to share work. Do not introduce global caches or reload sources
+for individual entries.
+
+Assess work against input size, relationship edges, materialized effective values,
+and retained provenance/history. Many children can legitimately produce much more
+output than the source declarations. Use representative chains, shared-base trees,
+wide collections, and replacement cases to evaluate growth. Performance evidence
+is not a new release timing gate.
+
+Existing acquired-input accounting remains in force. The eventual implementation
+plan must define how expanded output and retained derivation fit existing decoded
+tree/history limits, including where expansion is checked. Avoid silently charging
+one inherited value as a newly acquired source or inventing arbitrary entry/chain
+caps. Any necessary new public limit requires an explicit design decision.
+
+## Work needed from the baseline
+
+The feature requires root-owned opt-in declarations, a private relationship graph
+and expansion phase, and lifecycle integration before typed decoding. Presence,
+derivation history, diagnostics, sensitivity propagation, and source-relative
+resource origins must extend together. The public parser-independent facade
+remains the only consumer boundary.
+
+Representative examples and behavioral tests should establish source/inheritance
+ordering, chains and shared parents, failure diagnostics, partial bases, explicit
+values versus omission, unchanged sequence/null behavior, origin/resource
+preservation, result independence, and safe secret inspection. Both loading entry
+points and supported source representations should share those semantics.
+
+A new ADR should record the selected inheritance architecture after the open
+decisions are resolved. Architecture/deferred-scope documentation must recognize
+the new structural responsibility; public API, configuration, consumer, and
+internal documentation should describe implemented behavior when it lands.
+Implementation sequencing belongs in a separate future plan. Existing callers
+that do not opt in retain their behavior; an additive minor release is the target,
+with release preparation and publication handled separately.
+
+## Completion criteria
+
+A consumer can declare a named collection, load mixed-source configuration once,
+and receive resolved independent entries with one set of combination rules.
+It can inspect whether each setting was direct, inherited, defaulted, or derived,
+follow inherited resource origins, and validate final domain meaning. Missing or
+cyclic relationships fail before a usable result is returned. The consumer needs
+no separate graph resolver, generic merge pass, or provenance reconstruction.
+
+## Open Decisions
+
+### 1. Declaring collections and parent references
+
+**Recommendation:** a root-owned declaration in `Definition` identifies a
+collection path and one direct parent-reference field name within each entry.
+Consume that field as structural metadata during expansion and retain its source
+location for inspection. An omitted reference means no parent; explicit null
+removes a lower-tier relationship. A string names an exact key in the effective
+collection, including an empty string if such an entry exists. Other shapes are
+reference errors. Outside the opted-in position, the field remains ordinary data.
+This integrates naturally with existing source precedence and avoids a globally
+reserved `extends` field or duplicated entry identifiers.
+
+**Alternative:** declare a separate name-to-parent table at a designated input
+path. Payload fields never overlap control metadata and application structs need
+no inheritance field, but settings and relationships become separated and must
+be kept consistent. It also introduces a second keyed collection into diagnostics
+and source composition for each inheritance relationship set.
+
+### 2. Meaning of inherited presence
+
+**Recommendation:** an inherited value counts as present input for effective
+`IsExplicit`/defaulting behavior, with separate derivation metadata distinguishing
+it from direct child input. This treats inheritance as input expansion, keeps
+typed and resolver defaults below inherited settings, and avoids consumers
+reimplementing presence logic. The API must clearly explain that effective
+presence differs from direct declaration at the destination path.
+
+**Alternative:** inherited values remain non-explicit at the child path, with
+a new effective-input-presence query/defaulting rule. This preserves a narrow
+direct-declaration meaning of explicitness, but requires two presence concepts
+through the resolver lifecycle and risks defaults overwriting inherited settings
+unless callers adopt the new query consistently.
+
+### 3. Nested or overlapping collection declarations
+
+**Recommendation:** initially support multiple independent, non-overlapping
+declared collection paths. Report overlapping declarations as an options error.
+An outer inheritance expansion can copy or replace an inner collection, so
+overlap otherwise needs a selected rule for whether inner inheritance resolves
+before copying or after arrival at each destination. This restriction makes that
+unresolved interaction explicit rather than allowing declaration order to choose
+semantics. It does not restrict ordinary nested payload mappings.
+
+**Alternative:** define destination-based outer-to-inner expansion, resolving each
+inner collection after the enclosing entry is materialized. This supports nested
+presets, but creates additional namespace, dependency, and provenance semantics
+and can multiply graph instances. It is viable if a concrete consumer need
+justifies that additional model; it should not emerge accidentally from traversal.