From dc09959cca4e105f49ec316c6d4ba585bd2340bd Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Wed, 30 Sep 2026 00:18:40 +0000 Subject: [PATCH] Prepare ConfigKit v0.5.0 release --- docs/adr/0009-sibling-inheritance.md | 4 +- docs/development.md | 5 +- docs/releases/v0.5.0.md | 53 +++ .../feature-request-promptkit-inheritance.md | 134 ------- docs/roadmap/implementation.md | 353 ------------------ docs/roadmap/named-inheritance.md | 319 ---------------- docs/roadmap/remediation.md | 139 ------- 7 files changed, 58 insertions(+), 949 deletions(-) create mode 100644 docs/releases/v0.5.0.md delete mode 100644 docs/roadmap/feature-request-promptkit-inheritance.md delete mode 100644 docs/roadmap/implementation.md delete mode 100644 docs/roadmap/named-inheritance.md delete mode 100644 docs/roadmap/remediation.md diff --git a/docs/adr/0009-sibling-inheritance.md b/docs/adr/0009-sibling-inheritance.md index 8f3240f..c4457c4 100644 --- a/docs/adr/0009-sibling-inheritance.md +++ b/docs/adr/0009-sibling-inheritance.md @@ -3,8 +3,8 @@ ## Status Accepted and implemented. The [configuration contract](../config.md#sibling-inheritance) -owns current behavior; the [implementation plan](../roadmap/implementation.md) -records the delivery sequence. +owns current behavior. The completed implementation plan is preserved in Git +history at `83c59ea`. ## Date diff --git a/docs/development.md b/docs/development.md index 4e0a678..87ce364 100644 --- a/docs/development.md +++ b/docs/development.md @@ -84,8 +84,9 @@ The completed filesystem-source and underlay roadmap, original feature request, implementation plan, and resolved remediation record are preserved in Git history at `b604aa1` under `docs/roadmap/`. Current behavior belongs to the [configuration contract](config.md#layered-loading). Built-in sibling -inheritance is now part of the [configuration contract](config.md#sibling-inheritance); -its [roadmap](roadmap/named-inheritance.md) preserves the design background. +inheritance is now part of the [configuration contract](config.md#sibling-inheritance). +The completed request, roadmap, plan, and resolved review findings are preserved +in Git history at `83c59ea` under `docs/roadmap/`. The completed overlay rename plan and release-workflow simplification record are preserved in Git history at `73ce6e9` under `docs/roadmap/`. Current overlay diff --git a/docs/releases/v0.5.0.md b/docs/releases/v0.5.0.md new file mode 100644 index 0000000..16eefd6 --- /dev/null +++ b/docs/releases/v0.5.0.md @@ -0,0 +1,53 @@ +# ConfigKit v0.5.0 + +This minor release adds source-backed layered configuration and built-in sibling +inheritance, so applications can load one complete configuration while retaining +presence, source context, and explainable derivation. + +## Summary + +- `LoadLayered` combines ordered underlay groups beneath the main input, selected + overlay, and explicit CLI overrides. Inputs within a group still compose + strictly; precedence tiers combine mappings recursively. The main input + remains the owner of the optional schema-version check. +- `NewFilesystem(...).File(...)` accepts caller-owned `fs.FS` sources, and + `Mapping(...)` snapshots supported built-in values as a labeled source. + `Result.SourceContext` and `Resolution.SourceContext` expose passive resource + contexts for values from file-backed sources. +- `extends` now resolves a mapping against one named sibling after source + precedence and before typed decoding. Inherited values count as explicit + input. Origins, history, explanations, and redacted rendering retain the + source and the structural path by which a value reached its destination. + Review corrections preserve immutable nested-history evidence and bounded + expansion work. + +See the [configuration contract](../config.md#layered-loading), +[inheritance rules](../config.md#sibling-inheritance), and +[consumer examples](../consumers/README.md) for current behavior. + +## Compatibility and upgrade + +This is an intentional pre-v1 incompatible minor release for configurations +that use `extends` as ordinary data. That key is now reserved wherever an input +mapping occurs; it is consumed before decoding. Rename any such YAML key and +corresponding Go field, then rebuild. No collection registration or opt-in flag +is available. Configurations without `extends` retain their existing meaning. + +Applications adopting the new features may move source-backed defaults into +explicit `Underlay` groups, use `SourceContext` for relative resources, or +replace application-owned sibling merging with one `extends` relationship per +mapping. Adoption is optional. The maintained +[named-preset example](../consumers/README.md#named-sibling-inheritance) +shows inherited `false` and destination-local nested resolution. + +Select the new module version: + +```sh +go get gitea.maximumdirect.net/eric/ConfigKit@v0.5.0 +``` + +The minimum Go line remains 1.25. Linux amd64 and arm64 remain the +release-validated targets; macOS remains best-effort. The YAML dependency is +unchanged at `go.yaml.in/yaml/v4 v4.0.0-rc.6`. + +[COPYRIGHT](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/COPYRIGHT) | [LICENSE](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/LICENSE) | [ATTRIBUTION](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/docs/oss/attribution.md) diff --git a/docs/roadmap/feature-request-promptkit-inheritance.md b/docs/roadmap/feature-request-promptkit-inheritance.md deleted file mode 100644 index c643ca2..0000000 --- a/docs/roadmap/feature-request-promptkit-inheritance.md +++ /dev/null @@ -1,134 +0,0 @@ -# PromptKit Feature Request: Opt-In Named Configuration Inheritance - -Status: downstream proposal for ConfigKit consideration, not an approved scope -change, API design, or implementation plan. - -## Consumer Need - -PromptKit defines named execution profiles. A consumer can give an application -profile a stable name, reference another profile as its base, and override only -the settings it wants to change. Changing that base later does not require -updating every prompt or application setting that selects the application name. - -For example, an application-owned `weather-light` profile might derive from a -maintained standard profile and explicitly set a reasoning level or timeout. -Another profile can derive from `weather-light`, creating a chain. Multiple -children can share the same base. The selected child retains its own identity -while obtaining effective settings from its ancestors. - -PromptKit requests consideration of a reusable, opt-in named-inheritance -capability in ConfigKit. The goal is shared semantics for this structural -relationship, not knowledge of LLM models, endpoints, or execution policy. -ConfigKit should decide whether this belongs within its scope and what public -contract, if any, would be appropriate. - -## Distinction From Layered Loading - -The completed filesystem and layered-loading request is preserved in Git history -at `b604aa1:docs/roadmap/feature-request-promptkit.md`. The implemented -[layered-loading contract](../config.md#layered-loading) covers acquiring -configuration from different sources and combining it under explicit composition -or precedence rules. The proposed ConfigKit design for this remaining request -is in the [named-inheritance roadmap](named-inheritance.md). - -Named inheritance concerns relationships between configuration entries: a child -identifies a base, which may itself identify another base. The relationship -determines which entries contribute to the child's effective settings. Source -precedence and inheritance are different: choosing which definition owns an ID -does not itself resolve that definition's relationship to another ID. - -The earlier request assumes PromptKit retains relationship interpretation. This -proposal asks ConfigKit to reconsider that particular boundary; it does not -replace or expand the two already-requested prerequisites by implication. -Named inheritance remains optional for PromptKit's adoption of ConfigKit. - -## Desired Consumer Behavior - -- A caller can explicitly opt a collection of named entries into inheritance. - Ordinary configuration strings or similarly named fields must not acquire - reference semantics accidentally. -- An entry can reference one base by name, inherit its effective settings, and - override selected values. Single inheritance is sufficient for PromptKit; - this request does not seek multiple-parent conflict resolution. -- Chains and shared bases behave consistently regardless of declaration order. - A child's resolution does not mutate its base or change sibling results. -- Missing bases, self-reference, and longer cycles produce actionable errors - identifying the relevant entries and source locations where available. -- Omitted settings remain distinguishable from explicit zero, false, empty, - and null values. The chosen inheritance contract must explain which of these - inherit, replace, or clear rather than relying on Go zero-value heuristics. -- Consumers can identify the origins of inherited and overridden settings. - Sources may be embedded, file-backed, or programmatic; a host file path is - not always available. -- Callers can perform their domain validation on resolved settings without - teaching ConfigKit what those settings mean. - -These are behavior requests, not a proposed reserved `extends` key or API -signature. How a caller declares the collection, identity, and relationship is -for ConfigKit to decide. No automatic interpretation of arbitrary application -relationships is requested. - -## PromptKit Semantics To Consider - -PromptKit's proposed refactor uses a single effective definition for each profile -ID after source precedence has been applied. A base reference resolves within -that effective collection, including application overrides of maintained profiles. -Partial base profiles are useful even when they are not independently executable. - -Its current design direction is omission-based inheritance: explicit zero, false, -and empty values replace inherited settings where meaningful, while null means -unspecified for optional settings. Provider-parameter maps are atomic replacements, -including an explicitly empty map, rather than recursive combinations of keys. - -These details expose questions a shared contract must address; they are not a -request to build PromptKit-specific merge policies into ConfigKit. In particular, -generic recursive mapping overlay may not have the same meaning as a complete -provider-parameter replacement. PromptKit is pre-1.0 and can reconsider syntax -and semantics when a coherent shared design warrants it. Any remaining domain -exceptions should be explicit, not disguised as universal configuration rules. - -## Standardization Benefit - -The main benefit is consistency across consumers, not a large reduction in -PromptKit's code size. Applications can each write a small resolver, but differ -subtly in missing-reference handling, cycles, declaration-order dependence, -precedence, empty/null behavior, collection replacement, or inherited origins. -Those differences make configuration harder to transfer, explain, and maintain. - -A shared opt-in contract could let users learn these rules once and give -libraries a common diagnostic and provenance model. Similar needs could arise -in named processing presets, connection settings, or reusable job definitions; -these are potential use cases, not claims of existing consumer demand. - -The benefit depends on consumers adopting a meaningful common contract. If the -result requires arbitrary per-consumer merge and lookup strategies, much of that -standardization disappears. ConfigKit may reasonably conclude that keeping -relationship interpretation downstream is the better boundary. - -## Efficiency And Scope - -PromptKit expects to resolve definitions during construction and reuse the -results for ordinary executions. Large collections with shared ancestors should -not require repeated source loading, decoding, or avoidable repeated ancestor -work. Evaluate processing cost relative to entries, relationships, materialized -values, and retained provenance; this is not a fixed runtime or unconditional -linear-memory guarantee. No internal algorithm or storage representation is -prescribed here. - -The request excludes automatic hot reload, multiple inheritance, provider -availability checks, model selection policy, credential lookup timing, and -prompt execution. PromptKit continues to own LLM-specific validation and behavior -whether or not ConfigKit adopts structural inheritance. - -## Decision Requested From ConfigKit - -Please evaluate whether explicit, domain-neutral named inheritance fits -ConfigKit's scope, considering its [current configuration contract](../config.md) -and [deliberate exclusions](deferred.md). This request does not assert that an -earlier rejection exists or that any existing decision has been superseded. - -If accepted, define the shared consumer semantics and their relationship to -layered loading, including presence, collection handling, and origins. If deferred -or declined, PromptKit can retain its named-profile resolver while still adopting -ConfigKit's generic filesystem and layered-loading capabilities. Neither outcome -requires ConfigKit to understand the domain meaning of profile settings. diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md deleted file mode 100644 index f293677..0000000 --- a/docs/roadmap/implementation.md +++ /dev/null @@ -1,353 +0,0 @@ -# Sibling Inheritance Implementation Plan - -## Scope and working rules - -Implement the agreed [named-inheritance roadmap](named-inheritance.md), starting -from `01e38b5`. The roadmap owns feature policy; this document owns execution and -the concrete implementation choices below. Follow stages in order. Each stage is -a focused task for a gpt-5.6-terra coding agent and should end with relevant -passing checks and a plain-English commit. Do not publish or tag a release during -this sprint. - -Read `docs/development.md`, `docs/policy/agents.md`, and task-relevant policies. -Preserve unrelated work. The root owns public types; the private engine owns -processing. Reuse the parser-independent document representation and existing -helpers. Do not add dependencies, configurable keywords, registration APIs, -merge strategies, background work, or consumer-specific semantics. - -Stages 1–6 develop private machinery without activating inheritance in public -loading. Stage 7 connects the complete pipeline. Intermediate commits must not -publish resolved values with incomplete provenance or redaction. Private names -and file splits below are guidance, not new public contracts. - -## Decisions carried into implementation - -- `extends` is reserved input syntax. A string names an exact sibling mapping; - null cancels a relationship; omission retains lower contributions. Other types, - missing/non-mapping parents, self-reference, and cycles are errors. Empty strings - are exact names. Root and direct sequence-element mappings cannot use string - references; null needs no lookup. -- Strict composition still rejects overlapping non-mapping contributions, - including duplicate `extends` definitions. Complete all source-tier and CLI - precedence before expanding the effective input. Overwritten earlier references - are historical evidence, not active relationships to validate. -- Expand enclosing parent chains before resolving nested relationships in each - destination. Reusable parent templates retain unresolved nested instructions. - Sort mapping keys lexicographically, follow parent dependencies as needed, and - visit sequences by index. Sorting never determines value precedence. -- Parent-to-child combination uses recursive overlay, including explicit - zero/false/empty/null replacement and whole-sequence replacement. Consume the - current mapping's own relationship during enclosing expansion; nested - relationships travel with nested payload until their scope is resolved. -- Inherited values count as explicit input for presence/defaulting queries. - Provenance distinguishes direct input from inherited input. -- Version checking stays at its existing main-input point. Expansion follows - source assembly and precedes defaults/decoding, recorded resolution, secret - lookup, validation, and publication. Invalid effective relationships fail even - in entries the application does not select for execution. -- Preserve source records and passive resource contexts. Inheritance is neither - a new acquired source nor another precedence layer. - -### Concrete metadata representation - -Extend engine and root `Origin` with `Inheritance []InheritanceStep`. Define -root-owned and engine counterpart records with `From Path`, `To Path`, -`Reference Path`, and `Locations []Location`. A step describes a payload-path -transfer. `Reference` is the destination's controlling `extends` path; `Locations` -identifies that directive's contributing source coordinates. Paths are concrete -semantic paths, not names requiring another lookup. Records contain no values. - -Keep `Origin.Layer` and `Origin.Locations` for actual value contributors. Order -steps by derivation; composite origins retain contributing branches rather than -inventing a single chain. Project concrete descendant paths when queried. Keep -original contributor paths privately alongside locations, so synthetic sources -do not depend on file coordinates to identify original semantic paths. Direct -replacement keeps its own effective origin; displaced inheritance stays in history. - -Add inspection-only `ChangeInheritance = "inheritance"`, like `ChangeDecoding`; -do not accept it through `Resolution.Record`. Inheritance events identify -parent/control dependencies through existing `Change`, `ValueChange`, and -`Explanation` surfaces. No separate public query family is needed. Effective -inherited origins retain source layers and transfer evidence; expansion history -events use the new operation kind. - -### Budgets and diagnostics - -Keep acquired-input accounting unchanged. Use `defaultProjectionLimits()` for -expanded tree depth/nodes, counted by logical materialized nodes rather than -unique pointers; report `CodeSchemaProjectionLimit`. Check during materialization -before uncontrolled allocation and typed decoding. Do not charge copies as -acquired input or add a chain-length cap. - -Extend `historyNodesWithin` to retained inheritance templates and snapshots, -counting shared immutable document nodes once under the existing `historyNodeLimit`; -report `CodeExplainLimitExceeded`. Compact relationship records refer to shared -evidence instead of copying a full chain/tree per field. Do not redefine node -limits as byte limits or introduce a new public limit. Check retained growth as -it is accumulated and release temporary templates. Use private small test budgets. - -Add mirrored engine/root codes: `CodeInheritanceInvalid` (`inheritance.invalid`) -for invalid type/position or non-mapping parent; `CodeInheritanceMissing` -(`inheritance.missing`); `CodeInheritanceCycle` (`inheritance.cycle`); and -`CodeInheritanceVersion` (`inheritance.version`). Use directive paths/locations -as primary evidence and related definition locations for dependencies/cycles. -Messages must not interpolate raw reference strings or payload. Existing syntax, -composition, decode, and budget codes retain their meanings. - -## Stage 1 — Prepare shared recursive combination - -**Completed 2026-09-29.** Recursive mapping overlay now uses a private -value-combination walk with path-aware add, combine, and replacement callbacks; -the existing overlay adapter continues to own source origins and history, while -strict composition remains separate. A focused table covers recursive and -atomic replacements, empty mappings, event paths, and detached inputs. Focused -composition/overlay/override/underlay tests and `./scripts/check.sh` passed. - -Inspect `internal/engine/overlay.go`, `compose.go`, `mapping_index.go`, -`history.go`, and `internal/document/document.go`. `applyOverlay` already shares -`mergeOverlayMappings` across underlays, main-over-underlay assembly, and selected -overlays. Strict composition uses `mergeDisjointMappings` after claim validation; -history replay has a separate value-only traversal. - -Extract or adapt the smallest private recursive-overlay primitive needed for -arbitrary mapping roots, preserving ordered entries and indexed lookup. Keep -source-tier bookkeeping in its adapter. Provide path/replacement information for -inheritance's later adapter; do not model inheritance as a fake layer. Reuse -`document.Clone` and `mappingIndex`. Keep strict composition distinct. Consolidate -value-only replay only if doing so preserves its behavior without extra machinery; -do not build a public or general-purpose merge framework. - -Validation: existing compose/overlay/override and underlay tests, plus a compact -table covering recursion, atomic sequence/scalar/null replacement, empty mappings, -and detached ownership. Existing behavioral expectations should not change. -Run `./scripts/check.sh` before committing. - -## Stage 2 — Expand one sibling scope - -**Completed 2026-09-29.** A private, non-activated resolver indexes sibling -definitions, follows parent dependencies with an explicit stack, memoizes -unresolved enclosing templates, consumes each mapping's own relationship, and -retains transfer/control evidence. Safe diagnostics cover missing or non-mapping -parents, invalid reference types, and cycles. Focused tests include forward and -backward references, shared chains, exact unusual names, null cancellation, -independent results, and a deep cycle. `./scripts/check.sh` passed. - -Add a private inheritance resolver, callable only from engine tests initially. -Index effective sibling definitions. Resolve single-parent dependencies with -visiting/resolved states and memoized enclosing-chain templates. Use an explicit -work stack so graph chain length is not tied to the Go call stack. Overlay parent -payload with child payload using Stage 1. Consume the current mapping's own -`extends`, leaving nested instructions intact. Handle null before lookup. - -Implement internal diagnostic codes and deterministic traversal. Preserve -locations and collect transfer/control evidence for Stage 4. Keep original input -unchanged and templates private; a child must not mutate its parent. Failed -expansion returns diagnostics and no usable expanded result. - -Validation: forward/backward references, chains, shared parents, child replacement, -null cancellation, exact unusual/empty names, missing/non-mapping parents, invalid -types, self/long cycles, and input independence. Assert results and diagnostic -fields, not stack layout or exact prose. Run focused tests and the everyday checker. - -## Stage 3 — Resolve nested destination scopes and bound expansion - -**Completed 2026-09-29.** Private expansion now walks from enclosing mappings -into destination-local nested scopes, including mappings within sequences. -Root and direct sequence-element string references fail; null cancellation is -consumed. Logical materialization uses an early non-directive lower bound and -final node/depth checks under projection limits, with no global cache. Focused -tests cover destination-specific nested values, three-generation chains, -relationship replacement/cancellation, a destination-only cycle, sequence -scopes, invalid positions, and budget boundaries. `./scripts/check.sh` passed. - -Extend to the complete effective tree. Distinguish enclosing-chain templates from -fully resolved destination subtrees. After materializing an enclosing mapping, -resolve its nested sibling scopes independently. Visit sequence-contained mappings; -reject string references on root/direct sequence elements without ancestor lookup. - -Scope memoization to each effective destination namespace. Copying a template must -not copy its already-resolved nested children. Preserve inherited nested control -evidence and support cancellation through a child's nested null directive. Whole- -container replacement removes those active relationships at that destination; -the parent's own surviving relationships still require validation. - -Apply the selected expanded-tree budgets, counting logical output from shared -templates and releasing transient state. No global caches or unrelated-scope lookup. - -Validation: the roadmap's nested `primary`/`secondary` example; three-generation -enclosing chains; nested reference replacement/cancellation; destination-only -cycles; scopes inside lists; root/list-element errors; and parent names sorting -before or after children. Exercise small node/depth budgets at and beyond their -boundaries. Run the everyday checker. - -## Stage 4 — Preserve derivation, presence, and resource origins - -**Completed 2026-09-29.** Private expansion now returns the effective tree, -per-path origins, shared linked transfer traces, control records, and displaced -replacement evidence. Contributor records retain original semantic paths and -source locations through chains, nested destination scopes, composite mappings, -and sequence descendants; direct child replacements have independent effective -origins. Clones detach mutable slices and paths, and origin containment checks -include derivation. Focused source/context, provenance, replacement, and -detachment tests and `./scripts/check.sh` passed. Public loading remains unchanged. - -Implement the selected metadata representation privately in the engine. Carry -original semantic contributor paths, locations/layers, transfer edges, and control -evidence through copies and recursive combination. Preserve composite origins and -sequence-descendant origins, including atomic sequence contributions. Do not infer -ownership from the last matching layer: multiple sources can share a layer. - -Return an expanded tree, origins, and inheritance evidence for the decoded metadata -pipeline. Presence comes from expanded input. Direct replacement removes effective -inheritance at replaced paths but retains displaced evidence. Extend cloning, -comparison/deduplication, and private carriers; share immutable internal evidence -while detaching query outputs. - -Validation: inherited zero/false/null presence, chains and nested transfers, -composites, same-layer multiple sources, and replacement evidence. Demonstrate -that an inherited relative resource points to its original source context rather -than the child's directory. Inheritance does not open referenced resources. -Run the everyday checker. - -## Stage 5 — Integrate historical inspection - -**Completed 2026-09-29.** Source replay stays at original paths, followed by -projected structural control/transfer events before typed reconciliation and -application resolution. History and explanations preserve consumed/cancelled -directives, equal-valued child replacements, displaced descendants, nested -destination results, and parent source updates. Shared retained evidence is -counted with a bounded incremental check; focused tests and `./scripts/check.sh` -passed. Public loading remains unchanged. - -Extend `history.go`, `load_metadata.go`, `explain.go`, and explanation rendering. -Retain original source applications; represent structural expansion separately, -between source replay and canonical decoding, followed by recorded application -resolution. Do not attribute inherited values or consumed directives to decoding. - -Destination history must show parent contribution and child replacement when both -exist, even if expansion leaves the effective child value unchanged. Include -derivation dependencies, preserve source history at original paths, and make -chains navigable through concrete paths. Do not fabricate repeated chronological -source loading at every child. Explain consumed directives as historical control -input absent from effective metadata/configuration. Preserve cancellation and -superseded directive evidence. - -Extend historical path enumeration, ancestor replacement, origin comparison, and -retained-node accounting. Keep typed reconciliation separate. Share scope evidence -and project relevant changes instead of storing a full snapshot per leaf. - -Validation: inherited values, equal child replacements, displaced inherited -subtrees, parent source changes, consumed/canceled directives, destination-specific -nested results, and subsequent application-derived replacement. Check agreement -among final values, `History`, `Explain`, and historical paths. Exercise small -history budgets. Run the everyday checker. - -## Stage 6 — Preserve sensitivity through inheritance - -**Completed 2026-09-29.** Sensitivity classification now follows corresponding -source/destination paths through structural transfers and controls to a fixed -point, while leaving neighboring fields unclassified. Historical replacement -evidence remains redacted, and inherited secret references remain unresolved -until the existing typed lookup point. Focused secret/history tests and -`./scripts/check.sh` passed. Public loading remains unchanged. - -Connect transfer dependencies to existing secret classification and historical -redaction. Classify both source and destination paths under schema and -`Definition.Secret` rules; propagate sensitivity through chains/nested copies. -Include displaced evidence and control history. Preserve existing conservative -historical redaction without declaring unrelated siblings secret. - -Copy unresolved secret references only; lookup remains at the existing lifecycle -point. Credentials never enter expansion trees/history. Independent replacements -retain their own effective provenance while previous secrets remain redacted. -Queries and renderers must agree. - -Validation: source-only and destination-only classification, shared parents, -transitive inheritance, replacement/removal, and lookup after expansion. Use -disposable canaries across affected inspection/failure surfaces. Avoid real -credentials and parallel environment mutation. Run focused secret/history tests -and the everyday checker. - -## Stage 7 — Activate the common lifecycle and facade metadata - -**Completed 2026-09-29.** Ordinary and layered loads now expand assembled -input before decoding while retaining original source applications, provenance, -history, sensitivity, and resource contexts. Main-owned version paths reject -structural changes without rejecting unrelated inheritance. Root records and -conversions expose detached transfer evidence; public tests cover precedence, -presence, resource context, version timing, cancellation, imports, and secrets. -`./scripts/check.sh` and `go test -race ./...` passed. - -Insert expansion in `executeLoadRequest` after ordinary `applyInputLayers` or -layered `assembleLayeredInput`, before `decodeLoadInput` invokes defaults/decoding. -Include plain single-source loads, creating the minimal composition carrier needed -to preserve original history. Update `load_result.go` and inspection carriers with -Stages 4–6 evidence. Preserve no-input behavior and use a no-directive fast path -where practical. Do not repeat acquisition, version callbacks, or user resolution. - -Keep the version path main-owned. Detect inherited contributions, replacement, or -directive removal affecting it, including protected ancestor replacement; report -`CodeInheritanceVersion`. Do not reject an entire enclosing mapping merely because -it contains an unchanged version field. A version rule targeting an `extends` -field cannot silently lose its checked declaration when the directive is consumed. -Keep the checker at its original point and retain accepted version representations. - -Mirror records/codes in root `records.go`, conversions, clones, and API comments. -Ensure root `Origin.Inheritance` and `InheritanceStep` expose only root types and -detached slices/paths. Keep `ChangeInheritance` inspection-only. Update root/engine -`IsExplicit` comments, including resolver-entry queries. Preserve `SourceContext`. - -Do not treat `extends` as the acquisition-time `configkit` directive. Imports, -selected overlays, Mapping input, and CLI overrides carry it through combination -until expansion. Keep existing `configkit` validation and CLI path restrictions. -Typed defaults and values added by application resolution do not initiate another -structural pass; inheritance processes assembled source input only. - -Validation: public ordinary/layered loads; imports and strict conflicts; multiple -underlays; selected overlay and CLI changes to parent/child/reference; resolver -defaulting using inherited presence; version ownership and callback timing; -typed unknown-field behavior after consumption; no later hooks on expansion -failure; partial bases with normal validation; and no-directive regressions. -Include a filesystem resource-origin example and mapping/bytes equivalence case, -not a full duplication of every mechanism test. Run `./scripts/check.sh` and -`go test -race ./...` for ownership changes. - -## Stage 8 — Validate scaling and complete documentation - -**Completed 2026-09-29.** Benchmarks cover chains, shared-parent width, nested -destinations, replacements, and full inspected loads. Skipping repeated scope -assembly when no immediate relationship exists reduced measured work while a -single boundary clone preserves caller-input ownership. The maintained preset -example, configuration contract, consumer guidance, internal navigation, -architecture/deferred scope, and ADR 0009 now describe the implemented feature. -`./scripts/check.sh`, `go test -race ./...`, and the focused benchmarks passed. -No release was tagged or published. - -Add focused benchmarks beside assembly/underlay benchmarks: long chains, wide -shared-parent scopes, nested destination expansion, and replacements. Evaluate -allocations/work against input, materialized output, and retained history, -including sorting costs. Fix repeated ancestor work or whole-tree copying without -changing semantics. No unconditional O(n) promise or timing release gate. - -Add a maintained consumer example showing one-load inheritance, explicit false, -nested destination behavior, and source/derivation inspection. Update -`docs/config.md` as behavior owner, root comments as API owners, README/consumer -guidance as summaries, and internal overview/development test map as navigation. -Update architecture/deferred-scope descriptions for implemented inheritance. -Explain reserved-key migration, null cancellation, no registration, sibling lookup, -source-first ordering, presence versus provenance, and unchanged version/lifecycle -policy. No release-policy or platform-gate changes are part of this feature. - -Write the next available ADR (currently 0009) recording ConfigKit's chosen -architecture and rationale. Keep downstream-specific discussion out; rejected -choices belong only in Alternatives Considered. Do not edit accepted ADR decision -bodies or silently accept earlier drafts. Mark the feature roadmap's implementation -status accurately and link the implemented contract. Release notes/tagging remain -separate release-preparation work. - -Run the everyday checker, reuse race evidence if ownership code is unchanged, and -run benchmarks for evidence without pass/fail thresholds. Review the final diff -against the roadmap, source equivalence, confidentiality, and public boundary. -Confirm no independent merge rule, registration API, extra release gate, or -background/resource lifecycle was introduced. Commit and report validation and -any remaining limitations. diff --git a/docs/roadmap/named-inheritance.md b/docs/roadmap/named-inheritance.md deleted file mode 100644 index 53f7ad3..0000000 --- a/docs/roadmap/named-inheritance.md +++ /dev/null @@ -1,319 +0,0 @@ -# Built-In Sibling Configuration Inheritance - -Status: implemented. The [configuration contract](../config.md#sibling-inheritance) -owns current behavior. This document preserves the agreed design and its -baseline; the [implementation plan](implementation.md) records completed work. - -## Purpose and priorities - -Allow a configuration mapping to inherit settings from one sibling mapping by -using the built-in `extends` keyword. No collection registration, configurable -reference-field name, or inheritance field in application Go types is needed. -ConfigKit resolves the relationships during loading and returns final typed configuration -with inspectable provenance. Applications retain ownership of field meanings, -domain validation, and execution. - -The [downstream request](feature-request-promptkit-inheritance.md) supplies a -concrete need for reusable named defaults and chains of specialization. ConfigKit -adopts a shared structural model, using its existing recursive combination and -presence conventions. The design serves named presets, connection settings, or -similar configuration entries without interpreting their domain. - -Priorities are consistency and predictability, accurate presence/provenance, and -reduced consumer configuration infrastructure. Computational efficiency matters -within those goals: resolve shared ancestors once and account for the size of -materialized results and retained history. Avoid an unconditional O(n) promise. - -## Baseline at design time - -Baseline: `b604aa167b684525f0d5d806d9c4267824151ee4`, after the filesystem-source -and underlay implementation and its remediation. - -- The [configuration contract](../config.md) supports host/native filesystem, - byte, reader, and snapshotted mapping sources through a shared engine. -- Independent contributions compose strictly within a tier; explicitly ordered - tiers combine using recursive overlay. Typed defaults sit below source input; - selected overlays and explicit CLI overrides retain their documented roles. -- Named application entries can already be represented as string-keyed mappings. - ConfigKit currently treats relationships between their keys as application data. -- Recorded application resolution runs after typed decoding and preserves - dependencies and provenance. Consumers currently own relationship resolution; - there is no built-in pre-decode inheritance phase. -- Source contexts support later application-owned resource access. Loading is - synchronous and complete on return, with library-owned loading handles closed. - -This feature is a deliberate extension of structural input processing. It follows -the [architecture goals](../policy/architecture.md#success-criteria), the -composition distinction in [ADR 0001](../adr/0001-separate-composition-and-overlay.md), -the presence/provenance principles in -[ADR 0003](../adr/0003-preserve-presence-and-resolution-history.md), and the -complete-on-return design in [draft ADR 0008](../adr/0008-complete-on-return-loading.md). - -## Target configuration model - -### Built-in keyword and sibling lookup - -A mapping containing `extends: foo` inherits from the mapping named `foo` in -its immediately enclosing mapping. Lookup uses an exact sibling key, independent -of declaration order. It does not interpret names as paths, search enclosing -scopes, or search unrelated branches. One mapping can name one parent; chains -and shared parents are supported. - -```yaml -baz: - foo: - width: 10 - length: 20 - bar: - extends: foo - width: 20 -``` - -The resulting `bar` has `width: 20` and `length: 20`. ConfigKit consumes the -`extends` field as structural metadata, retaining its provenance for inspection -rather than passing it to the application's typed payload. - -`extends` becomes reserved configuration syntax throughout input mappings, -consistently for ordinary and layered loading and every supported source kind. -It is not a configurable keyword or an application-selected collection feature. -Mappings without a relationship retain their own payload, which can still contain -nested inheritance. Explicit null or scalar siblings retain ordinary data meaning -and final schema validation; a referenced parent must be a mapping. - -Identity comes from the sibling key, not a copied field in the parent payload. -The child keeps its key. Application fields such as `name` or `id` follow ordinary -combination semantics. - -### Reference values and positions - -A string-valued `extends` names an exact sibling key, including an empty string -if that key exists. Explicit `extends: null` means no parent and cancels a -relationship supplied by a lower-precedence source or inherited nested payload. -Omitting `extends` contributes no change to an existing relationship during -combination; if none remains in the effective mapping, the mapping has no parent. -Other value shapes are reference errors. - -Cancellation retains the mapping's own assembled settings but adds no settings -from the canceled parent. Like a string reference, the null control field is -consumed as structural metadata before typed decoding. This follows existing -presence and replacement rules without introducing a deletion operator. - -A string-valued `extends` on the document root or directly on a sequence-element -mapping has no enclosing mapping of named siblings and produces a location-bearing -reference error. ConfigKit does not search farther up for a namespace. Mappings -nested within those structures can still use their own sibling namespaces normally. - -### Source precedence first, inheritance second - -First assemble the effective input under the existing source rules: strict -composition within tiers and recursive overlay across underlays, main input, -selected overlay, and explicit CLI overrides. Strict conflicts remain errors. -Then resolve inheritance against the resulting effective tree. Inheritance is -the last structural composition operation, not the last loading lifecycle phase. - -Every parent reference sees that parent's effective definition, including source -overrides. For each entry, resolve the parent's chain, then recursively overlay -the child's own effective payload. Each child obtains independent effective -settings; resolving a child leaves its parent and siblings unchanged. - -Source precedence and ancestry answer different questions. Source precedence -selects each entry's own settings and parent reference. Ancestry supplies settings -that entry omits. For example, if a child's own timeout is `10`, changing its -parent's timeout to `30` through higher-priority source input leaves the child's -explicit `10` intact. To change that explicit child value, override the child. -This applies equally to CLI overrides at those respective paths. - -### One combination contract - -Parent-to-child combination uses the existing recursive mapping overlay: - -- Omitted children retain inherited values. -- Present zero, false, and empty strings replace inherited values. -- Mappings combine recursively; an empty mapping retains inherited children. -- Sequences replace as a unit, including empty sequences. -- Explicit null replaces the inherited value, subject to final schema acceptance. -- Cross-kind replacement follows the same overlay rules and final schema checks. - -The parent's own `extends` relationship is consumed when its parent chain is -expanded; it is not copied as the child's own relationship. Nested `extends` -fields remain structural instructions and travel with the nested payload until -that destination scope is resolved, as described below. - -These rules intentionally differ from the request's suggested null-as-unspecified -and whole-parameter-map replacement. Consumers adopting the feature adopt -ConfigKit's shared rules. Multiple inheritance, sequence merging, per-field -strategies, deletion operators, cross-scope references, and named-overlay -file inheritance are outside this feature's initial scope. - -### Enclosing-first, dependency-aware resolution - -Resolve inheritance from the root outward. Within each mapping scope, parent -dependencies determine resolution order: expand a sibling's parent chain before -combining that sibling's own payload. Then resolve nested relationships within -the resulting destination mappings. Nested relationships inherited from a parent -are evaluated in the destination context, after enclosing combination. - -```yaml -presets: - base: - settings: - primary: - width: 10 - secondary: - extends: primary - custom: - extends: base - settings: - primary: - width: 20 -``` - -Here `base.settings.secondary.width` becomes `10`, and -`custom.settings.secondary.width` becomes `20`. The nested relationship is -resolved independently in each resulting `settings` mapping. Copying the parent's -already fully resolved nested settings would produce the wrong result for `custom`. - -The engine therefore distinguishes expansion of a mapping's own parent chain -from resolution of its nested payload. It retains unresolved nested instructions -while combining enclosing mappings. Memoized work must respect that distinction -and the destination namespace; a fully resolved parent subtree is not a reusable -template for every child. - -Visit mapping keys in lexicographic order for deterministic traversal and -diagnostics, following dependencies as needed. Dependencies and nesting determine -the result; alphabetical order never determines which values win. A parent may -sort before or after its child without changing the meaning. Breadth-first versus -recursive traversal is an implementation choice subject to these semantics. - -Nested mappings inside sequences are also visited, while sequence combination -continues to replace whole sequences. A relationship on a sequence element itself -follows the reference-position rules above. - -## Loading and validation - -The proposed lifecycle is: existing acquisition/composition and main-owned version -checking; finish source precedence assembly; resolve named inheritance; apply -typed defaults and decode; recorded application resolution; secret lookup; -semantic validation; publish the final configuration and metadata. - -Resolve inheritance in the private input representation while omission is still -explicit. Ordinary and layered loading use the same implementation. Typed defaults -remain the final decoder's baseline and are not converted into parent declarations -or recursively copied from parent Go values. Domain resolution receives already -expanded settings and can apply recorded normalization/defaulting under its -existing contract. - -Preserve the existing version owner's declaration and checking point. Inheritance -must not supply, replace, or remove the protected version path or bypass ancestor -protections. Account for attempted protected-path changes in the expansion -design rather than moving the checker to a different lifecycle point. - -All relationships in the effective configuration resolve during the load, including -entries the application might not select for execution. Missing parents, non-mapping parents, -invalid reference values, self-reference, and longer cycles fail the load with -safe structured diagnostics and relevant source locations. Traverse deterministically; -diagnostics should identify the relationship and chain rather than expose payload -values. Synthetic inputs retain labels and semantic paths without fabricated file -coordinates. Normal syntax and strict-composition errors remain unmaskable. - -Partial base entries may omit domain-required settings. Omission is not by itself -a structural error. All returned entries still undergo ordinary typed decoding; -application validation determines which entries need to be independently executable -and which are reusable partial definitions. This adds no alternate schema, -automatic abstract-entry marker, or general exemption from validation. - -Successful loading yields the completed configuration and metadata. Inheritance -performs no background resolution and retains no filesystem handles. Applications -may load again explicitly using the existing source-reuse contracts. - -## Presence, provenance, and confidentiality - -Consumers must distinguish direct child input, inherited input, typed defaults, -and recorded application-derived values. Explicit zero/false/empty/null settings -remain distinguishable from omission. Go zero-value inference is not an acceptable -substitute. - -An inherited value counts as present configuration input: `IsExplicit` returns -true at its destination, while derivation metadata distinguishes inherited input -from a direct child declaration. Presence answers whether configuration supplied -the value; provenance explains where it came from and how it reached the destination. -Typed defaults and application resolver defaults based on absence therefore remain -below inherited settings, including inherited zero, false, empty, and null values. -The API documentation must explain this effective-input meaning of explicitness; -it does not mean the value was written directly at the destination path. - -Each inherited value retains its originating source and source path, together -with the relationship that placed it at the child's destination path. Chains -remain explainable, including a child's explicit replacement of an inherited -value. Source-precedence history remains inspectable alongside inheritance -derivation; inheritance must not masquerade as another acquired file or source -tier. Effective metadata, resolver-entry queries, history, explanation, and -redacted rendering must agree. - -A relative resource inherited from a parent keeps the context of the source -that supplied that resource value. It does not acquire the child's directory. -Composite origins remain composite and derived dependencies remain explicit. - -Inherit secret references as unresolved configuration input; resolve typed -credentials only at the existing lifecycle point. Preserve source and destination -sensitivity through inheritance dependencies so copying a classified value cannot -make it public. A child's independent replacement has its own provenance; inherited -or displaced historical secrets remain safely explainable and redacted. No raw -credential values belong in inheritance trees, metadata, or diagnostics. - -## Efficiency and ownership - -Model each sibling scope as entries and single-parent edges. Use indexed name lookup, -cycle detection, and memoized parent resolution to avoid repeated ancestor work. -Materialize independent public results while allowing carefully owned immutable -private structures to share work. Do not introduce global caches or reload sources -for individual entries. Cache enclosing expansion separately from destination-local -nested resolution, so shared work does not change reference meaning. - -Assess work against input size, relationship edges, materialized effective values, -and retained provenance/history. Many children can legitimately produce much more -output than the source declarations. Use representative chains, shared-base trees, -wide collections, and replacement cases to evaluate growth. Performance evidence -is not a new release timing gate. - -Existing acquired-input accounting remains in force. Expanded output and retained -derivation respect existing decoded tree/history limits; the implementation plan -defines accounting and check placement. Avoid silently charging -one inherited value as a newly acquired source or inventing arbitrary entry/chain -caps. Any necessary new public limit requires an explicit design decision. - -## Work needed from the baseline - -The feature requires built-in recognition of `extends`, a private dependency-aware -expansion phase with destination-local nested resolution, and lifecycle integration -before typed decoding. No collection-registration API is needed. Presence, -derivation history, diagnostics, sensitivity propagation, and source-relative -resource origins must extend together. The public parser-independent facade -remains the only consumer boundary. - -Representative examples and behavioral tests should establish source/inheritance -ordering, chains and shared parents, enclosing-first nested resolution, independence -from parent/child name sort order, failure diagnostics, partial bases, explicit -values versus omission, cancellation of relationships with null, invalid reference -positions and types, inherited presence during defaulting, unchanged sequence/null behavior, origin/resource -preservation, result independence, and safe secret inspection. Both loading entry -points and supported source representations should share those semantics. - -A new ADR should record the selected inheritance architecture. -Architecture/deferred-scope documentation must recognize -the new structural responsibility; public API, configuration, consumer, and -internal documentation should describe implemented behavior when it lands. -Implementation sequencing belongs in the implementation plan. Reserving `extends` -changes behavior for applications that currently use it as ordinary data. Document -that compatibility change and the required migration rather than describing this -as a purely additive opt-in feature. Configurations without `extends` retain their -existing behavior. Release preparation and publication are separate work. - -## Completion criteria - -A consumer can use `extends` between sibling mappings, load mixed-source -configuration once, and receive resolved independent entries with one set of combination rules. -It can inspect whether each setting was direct, inherited, defaulted, or derived, -follow inherited resource origins, and validate final domain meaning. Missing or -cyclic relationships fail before a usable result is returned. The consumer needs -no separate graph resolver, generic merge pass, or provenance reconstruction. diff --git a/docs/roadmap/remediation.md b/docs/roadmap/remediation.md deleted file mode 100644 index 12dd5e2..0000000 --- a/docs/roadmap/remediation.md +++ /dev/null @@ -1,139 +0,0 @@ -# Sibling Inheritance Review Findings - -Reviewed implementation commits `78d4b59` through `2ca1f80` against the -[feature roadmap](named-inheritance.md), [implementation plan](implementation.md), -and the user's agreed sibling lookup, enclosing-first expansion, presence, and -provenance decisions. Both findings below are now resolved. The original review -evidence is retained below; source line references identify the reviewed baseline. - -## Resolution — 2026-09-30 - -- The destination walker now detaches each mapping/sequence container before - replacing its child pointers. Retained enclosing templates remain immutable; - untouched subtrees remain shared, with existing retained-history accounting. - Regressions check value introduction, inherited control arrival/consumption, - container explanations, a three-generation chain, and sequence-contained scopes. - The nested-history regression failed before the fix and passes afterward. -- Budget accounting now maintains one logical document total and updates it from - old/new scope counts. Each scope receives the budget remaining after other - branches are reserved. Early expansion checks and final node/depth validation - remain in place. Boundary tests include independent scopes inside sequences, - unchanged sibling data, and consumed null directives. -- The maintained `BenchmarkInheritanceIndependentScopes` covers 256, 512, and - 1,024 independent scopes. Allocations measured approximately 8.95, 17.87, and - 36.12 MB/op respectively: roughly proportional to scope count. This benchmark - has a larger per-scope payload than the original review probe, so absolute - values are not a before/after comparison. No timing threshold was introduced. -- Validation after the fixes: `./scripts/check.sh`, `go test -race ./...`, - the new independent-scope benchmark, and the existing expansion/inspected-load - benchmarks passed on Linux. Existing redaction, ownership, and history-budget - tests remain passing; no public contract or limit was changed. - -## 1. Preserve immutable enclosing snapshots during nested resolution - -Priority: P2 — incorrect historical explanation despite correct final values. - -Locations: [retained operation templates](../../internal/engine/inheritance.go) -(lines 230–235), [nested mutation](../../internal/engine/inheritance_walk.go) -(lines 107–109), and [explanation replay](../../internal/engine/inheritance_explain.go) -(lines 26–38). - -`inheritanceOperation.parent` and `.effective` refer to the same node trees that -the destination walker later modifies. Replacing nested `entry.Value` pointers -therefore changes already-recorded enclosing snapshots. Their origin indexes -still describe the earlier template, so historical values and origins can disagree. -Replaying an enclosing transfer prematurely installs already-resolved nested values -and loses the nested control input that should explain those values. - -Reproduction: load this configuration through the public loading path (a -`map[string]any` configuration suffices): - -```yaml -presets: - base: - settings: - primary: {width: 10} - secondary: {extends: primary} - custom: - extends: base - settings: - primary: {width: 20} -``` - -The final child width is correctly `20`. However: - -- `Explain(/presets/custom/settings/secondary/width)` reports its only - inheritance value transition as `20` to `20`, without showing its introduction. -- `Explain(/presets/custom/settings/secondary/extends)` returns two structural - events with both before and after absent. It never shows the inherited - `primary` reference before consumption. -- Explaining the enclosing child shows fully resolved nested values in its - enclosing transfer, before the nested operations that supposedly produce them. - -These observations were reproduced with temporary engine tests invoking `Load` -and `Inspection.Explain`; the temporary tests were removed after the review. -This violates the agreed inspectable derivation and enclosing-before-nested -history, not a request for additional historical guarantees. - -Required correction: - -- Keep retained parent/effective templates immutable while destination-local - traversal proceeds. Use detached working structure or copy-on-write ownership - as appropriate, preserving shared evidence without a snapshot per leaf. -- Keep each operation's values and origins at the same structural point. -- Replay inherited control arrival/consumption and nested value introduction - explicitly, with metadata and explanation agreeing. -- Preserve existing retained-history accounting for any additional owned nodes. - -Acceptance: add regressions using this nested example that assert meaningful -before/after values and consumed control history, not only the final result or -absence of `ChangeDecoding`. Cover a further enclosing chain and compare leaf -and container explanations. Existing ownership, redaction, and budget tests must -continue to pass. - -## 2. Remove whole-document budget rescans per sibling scope - -Priority: P2 — avoidable superlinear expansion work. - -Location: [per-scope global lower-bound check](../../internal/engine/inheritance_walk.go) -(line 95). - -Every resolved scope invokes `inheritanceLowerBoundWithin(*w.root, ...)`, traversing -the complete current document again. With N independent constant-sized sibling -scopes, input, output, and retained relationship evidence are all O(N), but this -check alone performs O(N²) visits. Its temporary traversal stacks also create -unnecessary allocation growth. This is independent of legitimate large inherited -output and of key-sorting costs. - -A temporary benchmark generated N independent root entries, each containing -`a: {x: v}` and `b: {extends: a}`, and invoked `expandInheritanceTree`. Allocation -measurements were approximately 5.8 MB/op at 256 scopes, 14.1 MB/op at 512, and -49.5 MB/op at 1,024. Timing was noisy and is not proposed as a gate; the repeated -global traversal establishes the complexity defect directly. Existing benchmarks -concentrate on shared parents/chains rather than this independent-scope shape. - -Required correction: - -- Maintain a document-level logical node total using old/new scope counts, or - equivalent incremental accounting, rather than rescanning unrelated branches. -- Keep early bounded expansion and the existing final node/depth verification; - do not weaken the existing budgets or add a new cap to work around the cost. -- Include an independent-scope benchmark alongside current shapes and inspect - growth relative to output/evidence size. Do not add timing thresholds. - -Acceptance: show that independent-scope accounting no longer revisits the whole -document per scope; retain at/beyond-budget tests and the existing value/origin -semantics. Benchmark evidence should demonstrate the removed allocation/work -growth without asserting an unconditional O(N) public guarantee. - -## Validation and contract review - -- `./scripts/check.sh` passed on Linux. -- `go test -race ./...` passed on Linux. -- Temporary diagnostic tests and the independent-scope benchmark reproduced the - findings; no probe files remain in the repository. -- No clear additional public prohibition, strengthened security guarantee, or - release gate was found. Reserved `extends`, sibling-only lookup, null cancellation, - invalid root/list-element references, inherited explicitness, and preservation - of version ownership reflect the agreed design. The performance correction - should preserve existing limits rather than introduce stricter ones.