Align the assessment with ConfigKit v0.5.0

docs/roadmap/configkit-assessment.md

@@ -1,8 +1,9 @@
 # ConfigKit Adoption Assessment
 
-Status: assessment with agreed design direction; upstream API details remain
+Status: assessment aligned to released ConfigKit v0.5.0. Upstream capability
-to be settled before the feature roadmap and implementation plan. No runtime
+prerequisites are available; PromptKit integration and migration decisions remain
-changes are part of this assessment.
+for the feature roadmap and implementation plan. No runtime changes are part of
+this assessment.
 
 ## Goal And Recommendation
 
@@ -14,8 +15,8 @@ and their migration paths.
 
 Recommend adoption through an eager, immutable definition-loading boundary.
 ConfigKit should own YAML interpretation, source-tier combination, strict typed
-decoding, presence, and source diagnostics. PromptKit should own definition identity, named profile
+decoding, sibling inheritance, presence, and source diagnostics. PromptKit should
-relationships, LLM-specific rules, and execution. Do not retain the lazy loading
+own definition selection, LLM-specific rules, and execution. Do not retain the lazy loading
 architecture merely to minimize migration, or introduce a second generic
 configuration framework inside PromptKit.
 
@@ -25,69 +26,53 @@ lazy error arbitration, and repeated profile-chain resolution. Some of those
 savings come from the loading redesign rather than ConfigKit itself. ConfigKit
 adds ongoing value by owning the generic decoding behavior and its tests.
 
-## Prerequisite: Proposed Upstream ConfigKit Work
+## Released ConfigKit Capabilities
 
-Design, implement, and release these two ConfigKit capabilities before the
+ConfigKit v0.5.0 satisfies the two requested prerequisites and also implements
-dependent PromptKit refactor. The remaining recommendations assume this upstream
+sibling inheritance. Adopt the released public contracts rather than waiting for
-work; finalize their integration details against its agreed public contracts.
+the earlier proposals to be implemented:
-These are selected architectural prerequisites, not a claim that adoption is
-technically impossible using today's reader-based API.
 
-1. **Native `fs.FS` sources and source-relative context.** Support embedded and
+1. **Filesystem sources:** `NewFilesystem(label, fsys).File(name)` borrows an
-   caller-supplied filesystems as first-class configuration sources. Define source
+   `fs.FS` with reusable namespace identity. Imports and resource contexts retain
-   identity, relative path resolution, ownership, and how enabled imports and
+   that namespace. `Result.SourceContext` and `Resolution.SourceContext` expose
-   overlays operate within that filesystem. This is broadly useful for libraries
+   passive `ResourceContext` values with `FS` and `BaseDir`.
-   bundling default configuration. Passing an opened file to today's `Reader`
+2. **Source-backed underlays:** `LoadLayered` accepts a main `Input`
-   supplies bytes and a label, but does not make filesystem-relative loading
+   and ordered `Underlay` groups. Each group composes strictly; groups overlay
-   native to ConfigKit; that workaround is not the target architecture.
+   recursively from lowest to highest priority. `Input{Source: NoSource()}` omits
-2. **Source-backed underlays with origins and programmatic input.** Adopt
+   the main input. `Mapping(label, map[string]any)` snapshots supported built-in
-   ConfigKit's working design: strictly compose contributions within each tier,
+   scalar/list/map values with explicit presence; it does not ingest arbitrary
-   then recursively overlay explicitly ordered tiers. Support embedded, host-file,
+   typed structs. Generic programmatic input is not represented as CLI overrides.
-   byte/reader, and labeled programmatic sources without temporary files or YAML
+3. **Sibling inheritance:** the reserved `extends` key names one immediate sibling
-   serialization round trips. Prefer the proposed string-keyed programmatic
+   mapping. Relationships resolve after precedence assembly and before typed
-   mappings: omitted keys convey absence; present values, including nil/null,
+   decoding, with inherited origins, history, and resource context preserved.
-   convey explicit input. Do not require typed-value ingestion as a prerequisite.
+   This is built-in behavior, not the opt-in/registered-collection API originally
+   suggested by PromptKit.
 
 Adopt ConfigKit's recursive combination model rather than preserving PromptKit's
 whole-profile shadowing or atomic provider-map replacement across source tiers.
-No hidden replacement pass should run after ConfigKit loading. Named inheritance
+No hidden replacement or inheritance pass should run after ConfigKit loading.
-is a separate optional upstream request, not part of these prerequisites.
+Use released sibling inheritance for profiles, leaving domain validation in
-
+PromptKit. This assessment does not authorize code changes in either repository.
-The upstream design should show how the two capabilities work together, including
-source origins and efficient ownership/copying. Resolve these API questions before
-turning the assessment into a decision-complete PromptKit implementation plan.
-This assessment does not authorize code changes in either repository.
 
 ## Evidence And Scope
 
-Reviewed local baselines on 2026-09-29:
+PromptKit's runtime baseline remains v0.11.2; this assessment has since received
+documentation-only revisions. Updated upstream review on 2026-09-30:
 
 - PromptKit: `12d6d80024e80c552d122a076c22429bf6f2f6c2` (v0.11.2).
-- ConfigKit: `67c03690181c5af8dde6be325abac241dae6732f` (v0.4.2).
+- ConfigKit: `dc09959cca4e105f49ec316c6d4ba585bd2340bd` (v0.5.0), verified
-
+  against the published tag. Its clean checkout passed `go test ./...` during
-Design update: reviewed ConfigKit's working
+  the release review. This replaces the earlier v0.4.2 and uncommitted-roadmap
-`docs/roadmap/sources-and-layering.md` with checkout HEAD
+  comparison as the upstream basis for adoption.
-`36172ed20e832b371506d6aceb02dc6c8f50f71b`. The reviewed roadmap included
-uncommitted changes, so that commit alone does not identify the reviewed text.
-It remains a proposed target, not implemented behavior. The recommendations here
-now follow its underlay semantics and support its two open-decision recommendations:
-labeled programmatic mappings and direct source-context queries. Verify the
-final upstream contracts before preparing dependent implementation details.
 
 PromptKit evidence: [consumer workflows](../consumers/pkg-promptkit.md),
 [file formats](../formats.md), [source implementation](../internal/sources.md),
 [runner](../internal/runner.md), and the implementation owners identified below.
-ConfigKit evidence: its reviewed `docs/config.md`, `docs/roadmap/deferred.md`,
+ConfigKit evidence at that release: `docs/config.md`, `docs/releases/v0.5.0.md`,
-`lifecycle.go`, `source.go`, and ADR 0001 in the sibling checkout. That checkout
+`layered.go`, `source.go`, `resource_context.go`, and ADR 0009 in the sibling
-is review context, not a proposed build dependency or required directory layout.
+checkout. That checkout is review context, not a build dependency or required
-
+directory layout. The released configuration contract owns exact upstream behavior;
-ConfigKit currently exposes `Load[T]` over file, byte, reader, and defaults-only
+the summaries here identify its consequences for the proposed PromptKit design.
-sources. It supports strict decoding, presence/origin records, defaults,
-recorded resolution, semantic validation, explicit imports, one selected
-filesystem overlay, and secret handling. Its parser and generic conversion
-machinery are private. It has no public arbitrary layer-composition, generic
-inheritance, or `fs.FS` source API. Its deferred scope explicitly excludes several
-of these generalizations; an upstream enhancement requires a deliberate decision.
 
 This is a source-and-contract assessment, not a performance benchmark or full
 correctness audit. It does not claim measured allocation, latency, or line-count
@@ -132,7 +117,8 @@ provide configuration provenance.
 | Configuration filesystem sources, relative imports, and generic layer origins | Delegate to the prerequisite ConfigKit APIs; do not maintain parallel reader-based loading machinery |
 | Definition discovery and relative message-content references | Retain PromptKit's domain resource adapters, using ConfigKit source context and standard `io/fs` facilities where sufficient |
 | Profile-source priority and combination | PromptKit selects tiers; ConfigKit owns strict within-tier composition and recursive cross-tier overlay; eliminate repository fallback and whole-definition shadowing |
-| Named profile inheritance and backend selection | PromptKit owns domain relationships unless the separate inheritance proposal is adopted; integrate resolution with ConfigKit's recorded lifecycle |
+| Named profile inheritance | Delegate sibling graph resolution, combination, and derivation to ConfigKit's `extends`; remove the custom profile resolver |
+| Backend selection and domain normalization | Retain PromptKit domain rules; use recorded application resolution for configuration transformations not already handled by structural inheritance |
 | LLM setting bounds, role/session normalization, output rules, reserved provider fields | Retain shared PromptKit domain owners; ConfigKit validation callbacks may invoke them |
 | Definition defaults | Use ConfigKit defaults where they do not erase inheritance presence; apply effective execution defaults after inheritance |
 | Programmatic configuration | Prefer labeled mapping input with explicit presence; evaluate whether typed PromptKit convenience constructors justify conversion code, without YAML round trips or zero-value inference |
@@ -162,7 +148,8 @@ in the feature roadmap. Support OS directories, individual files, embedded/custo
 `fs.FS` sources, and programmatic input through the new source model; source selection
 and definition discovery remain PromptKit concerns, not a second composition engine.
 
-Perform domain-specific resolution through ConfigKit's recorded lifecycle before
+ConfigKit expands structural inheritance before typed defaults/decoding. Perform
+any remaining domain-specific resolution through its recorded lifecycle before
 semantic validation and result publication. Reuse its presence and provenance;
 do not add a generic merge/defaulting pass or reconstruct origins after loading.
 Runtime client assembly and resource interpretation remain outside typed decoding.
@@ -174,7 +161,7 @@ source-relative content resolution and containment without expanding the threat
 model. A snapshot is a set of captured reads, not an atomic filesystem transaction.
 
 Resolve each effective content reference using its contributing source's context,
-not the highest tier's directory. Support ConfigKit's proposed direct context
+not the highest tier's directory. Use ConfigKit's released direct context
 query rather than a consumer-maintained filesystem registry. Composite origins
 must not be assigned an arbitrary directory; ambiguous or absent context needs
 an actionable resource error or explicit consumer-supplied context. ConfigKit
@@ -212,7 +199,7 @@ not a reason to impose intermediate typed validation on ordinary inputs.
 Preserve profile templates that become usable through descendants or runtime
 overrides: eager loading must not require every consumer profile to be an
 independently runnable endpoint/model pair. Validate supplied fields and graph
-integrity eagerly (including missing bases and cycles), and check final execution
+integrity eagerly (ConfigKit handles missing bases and cycles), and check final execution
 requirements after runtime selection/overrides. Prompt default-profile references
 can be checked against the assembled collection; a prompt with no default remains
 valid for callers that supply a profile explicitly.
@@ -230,7 +217,7 @@ If a separate configuration schema version is later needed, respect ConfigKit's
 designated main-input ownership and pre-underlay check timing; do not assume
 last-wins version selection across catalogs.
 
-### Source Presence And Separate Named Inheritance
+### Source Presence And Released Sibling Inheritance
 
 Adopt ConfigKit source-layer semantics: absence retains lower input; explicit
 zero, false, and empty strings replace it; mappings combine recursively; empty
@@ -247,21 +234,40 @@ profile fields in ways that erase that distinction before relationship resolutio
 Provider `extra_params` maps combine recursively across source tiers too: `{}`
 does not clear lower keys. Retire the proposed atomic source-map replacement and
 null-as-unspecified conventions; do not emulate them after loading. An operator
-overriding only `model` also retains a lower `base_profile` declaration and other
+overriding only `model` also retains a lower `extends` declaration and other
 omitted settings. Diagnose and document these consequences explicitly.
 
-Named inheritance still selects relationships between effective entries; source
+Replace `base_profile` with `extends` on sibling profile mappings. A string names
-layering does not implement it. The optional upstream request remains pending.
+one exact immediate sibling; null cancels the relationship while preserving the
-Finalize named-inheritance null/map semantics separately, favoring consistency
+child's own settings. Omission retains a lower-tier directive. There is no ancestor
-with source layering rather than automatically retaining atomic maps or null-as-
+scope search or cross-collection reference. Superseded references are history,
-inherit. Any domain exception needs explicit justification. This also does not
+not active relationships to validate; every effective relationship resolves,
-silently change the existing request-level override contract, which is a separate
+including profiles not selected for execution.
-runtime boundary to review in the feature roadmap.
 
-Resolve shared ancestors once per snapshot. Use recorded domain resolution to
+Children recursively overlay their resolved parents with the same value rules:
-keep resulting values and origins consistent. Confirm the upstream lifecycle can
+zero/false/empty strings and sequences/null replace, mappings combine, and `{}`
-express required relationship provenance; do not claim generic layer history
+does not clear inherited keys. Inherited values count as explicit input for
-already explains named inheritance or invent a parallel audit framework.
+presence queries and absence-based defaults, even when false, zero, or null.
+Do not repeat inheritance in PromptKit's application-resolution callback. Use
+`Origin.Inheritance` and retained history for explicit operator inspection.
+Inherited resource references retain their declaring source's context.
+
+Enclosing inheritance resolves before nested relationships; nested inherited
+directives resolve at each destination against its siblings. Defaults and
+application-derived values do not trigger another inheritance pass. Model this
+deliberately in PromptKit configuration rather than promising arbitrary graph
+resolution over runtime values. Partial bases must fit the typed schema and
+PromptKit's validation; ConfigKit grants them no special validation exemption.
+
+`extends` is reserved in every input mapping, including provider-parameter maps,
+across all source kinds. There is no opt-in switch or ordinary-data escape in the
+released contract. Audit catalog and consumer payload needs before finalizing
+their format; a literal provider field with that name cannot silently pass through
+the configuration loader. Do not invent an encoding workaround in this assessment.
+
+The existing request-level override API remains a separate runtime boundary.
+The feature roadmap must review its interaction with configured values without
+silently subjecting runtime request data to configuration inheritance directives.
 
 ### Reloading, Errors, And Application Integration
 
@@ -335,7 +341,7 @@ cost, but explanation generation and provenance analysis stay off the run path.
 | `internal/profile/filesystem_repository.go`: metadata scans, selected strict decode, stream-end checks, malformed-file selection arbitration | Replace with source acquisition, ConfigKit assembly/decoding, and indexed effective values; remove the metadata prepass and per-lookup filesystem path |
 | `internal/promptdef/filesystem_repository.go`: metadata selection and repeated decoding/content opening | Replace with eager prompt loads and an ID/version index; keep domain normalization and contained content reading |
 | Profile filesystem, `fs.FS`, eager, memory, and overlay repository variants | Collapse loading into source adapters plus immutable collections; preserve only interfaces that still serve a real boundary |
-| `internal/profile/resolving_repository.go` | Replace per-operation traversal and zero-sentinel merging with once-per-snapshot presence-aware typed resolution; graph logic remains |
+| `internal/profile/resolving_repository.go` | Remove custom graph traversal, cycle detection, and zero-sentinel merging in favor of ConfigKit sibling inheritance; retain only PromptKit domain validation |
 | `internal/engine` raw repository-stack assembly | Replace with ConfigKit source tiers and one effective typed collection; engine capacity/client assembly remains |
 | `internal/catalog` profile metadata extraction | Obtain explicit field information from ConfigKit; retain manifests, ownership, namespace, connection-metadata and secret checks |
 | `internal/filecatalog` | Retain/reduce discovery and containment helpers actually needed by the eager loader; do not claim the entire package disappears |
@@ -377,10 +383,12 @@ input and produced-data size, not a blanket O(n) claim with an undefined n.
   captured content instead of reparsing it for metadata, selection, and validation.
 - Build identity indexes in one pass, with expected constant-time map lookup.
   Do not scan whole collections for every lookup or precedence decision.
-- Process inheritance graph structure in O(V + E), where V is the number of
+- Delegate inheritance expansion to ConfigKit once during loading rather than
-  profiles and E the inheritance edges. Resolve shared ancestors once. Account
+  rebuilding its graph in PromptKit. Assess scaling against input, materialized
-  separately for the size of materialized settings and retained origins; copying
+  inherited values, nested destination resolution, and retained history; do not
-  large inherited maps to many independent results necessarily costs more than V.
+  claim unconditional O(V + E) total work from the count of profiles and edges.
+  ConfigKit bounds expansion and retained evidence through its existing budgets;
+  representative catalog sizes must fit them without new per-run processing.
 - Avoid whole-collection copying per layer, repeated serialization, and repeated
   prefix/history traversal. Analyze layer processing against total input nodes
   and produced values/provenance, including intentionally retained history.
@@ -397,17 +405,13 @@ The implementation plan should include focused benchmarks for consequential
 repeated-work risks, such as shared-ancestor profiles and many layers, rather than
 machine-dependent timing gates or benchmarks for every helper.
 
-## 5. Additional Upstream Opportunities
+## 5. Upstream Readiness And Remaining Opportunities
 
-Native filesystem sources and public layered loading with origins are the
+Filesystem sources, underlays, programmatic mappings, source contexts, and sibling
-prerequisites described near the beginning, not optional follow-ups. The following
+inheritance are released in v0.5.0. No additional upstream feature is currently a
-remain optional and do not block adoption unless the upstream design establishes
+demonstrated prerequisite. Context-aware loading remains a possible future request
-a concrete dependency:
+if concrete consumers need it, not a reason to delay adoption. Existing loading
-
+is synchronous; do not simulate cancellation with detached background work.
-| Candidate | Recommendation and tradeoff |
-| --- | --- |
-| Generic named configuration inheritance | Requested separately in ConfigKit's `docs/roadmap/feature-request-promptkit-inheritance.md`, primarily for shared semantics across consumers. ConfigKit decides whether to adopt it; PromptKit can retain a small domain resolver if declined. Presence-aware source layering belongs to the prerequisite work, not this optional proposal. |
-| Context-aware loading | Optional for reusable cancellable construction if large-source consumers need it. Promise cooperative cancellation only; no detached reads or instant interruption of arbitrary readers. |
 
 Do not propose a public raw AST, permissive identity-only decoder, or dynamic
 subtree conversion solely to preserve PromptKit's current scanning design.
@@ -418,12 +422,12 @@ ConfigKit and PromptKit both declare LGPL-3.0-or-later. Adoption must still upda
 dependency notices under the OSS policy and inspect the exact selected release
 and its transitive notices. Their current Go minimums do not require a PromptKit
 toolchain increase. Use a released module dependency, not a sibling-directory
-replacement. This assessment proposes upstream work, but makes no upstream edits
+replacement. Select v0.5.0 as the assessed adoption baseline; any newer selection
-or licensing change.
+needs its own review. This assessment changes neither dependencies nor licenses.
 
 ## 6. Decisions And Migration
 
-The agreed direction now incorporates ConfigKit's working roadmap. This supersedes
+The agreed direction now incorporates ConfigKit's v0.5.0 contracts. This supersedes
 the earlier recommendations for independently typed documents, whole-definition
 shadowing, intermediate typed validation, and atomic maps/null-as-unspecified at
 the source boundary.
@@ -438,9 +442,9 @@ the source boundary.
    typed loads followed by custom combination are not the selected direction.
 3. **Presence and relationships.** Accept recursive source-tier overlay, retained
    children for empty mappings, explicit null replacement, and effective metadata.
-   Named inheritance remains separate pending the upstream decision; settle its
+   Adopt built-in sibling inheritance, including reserved `extends`, recursive
-   domain semantics before implementation rather than assuming either automatic
+   mapping combination, null cancellation, and inherited explicit presence. Do
-   equivalence or an exception to ConfigKit's rules.
+   not preserve a parallel PromptKit structural resolver.
 4. **Diagnostic boundary.** Keep wrapping ConfigKit's public loading errors rather
    than mirroring them. Retain actionable domain relationship and resource errors
    without duplicating ConfigKit's generic diagnostic/provenance system.
@@ -450,14 +454,17 @@ the source boundary.
    ordinary prompt execution. Finalize targeted query shapes and ownership in the
    feature roadmap, with presentation remaining application-owned.
 
-Support the upstream recommendations for labeled mapping-based programmatic input
+Use released mapping-based programmatic input and direct filesystem resource-context
-and direct filesystem resource-context queries. If real consumer examples show
+queries. If real consumer examples show
 mapping construction to be repetitive, consider small typed PromptKit conveniences
 with explicit presence rather than requiring an upstream typed-ingestion API.
-Final upstream signatures and ownership details remain open dependencies of the
+
-feature roadmap, not implemented guarantees. Resolve the named-inheritance
+Remaining PromptKit decisions are the exact collection/version format, catalog
-direction there as well; a PromptKit-owned resolver remains viable if ConfigKit
+migration, programmatic ergonomics, operator-inspection surface, and validation
-declines or defers that optional request.
+of partial base profiles. Also settle how operators discard unwanted provider
+settings or old relationships using the released null semantics without a hidden
+replacement pass. Check the reserved-key impact on arbitrary provider maps. These
+are integration decisions, not unresolved ConfigKit API proposals.
 
 Expected consumer migration:
 
@@ -467,11 +474,15 @@ Expected consumer migration:
   for historical YAML coercion.
 - Move standalone definitions into the agreed keyed contribution format. Review
   same-name contributions: disjoint fields within a tier can combine, and higher
-  tiers retain omitted lower fields, including `base_profile` references.
+  tiers retain omitted lower fields, including `extends` references. Convert
+  `base_profile` to sibling `extends`; use `extends: null` to cancel a lower-tier
+  relationship, not to erase all settings already supplied to the child.
 - Review zero/false/empty/null values explicitly. Provider maps merge across
   tiers, `{}` does not clear them, and null replaces rather than means omission.
   Use labeled programmatic mappings or approved presence-aware conveniences.
-  Document the separately selected inheritance semantics when settled.
+  Apply those same combination rules to inherited settings. Rename ordinary
+  configuration keys named `extends`; examine provider maps for cases where
+  renaming would change their substantive meaning and resolve those explicitly.
 - Handle effective-definition errors at construction, fix syntax/composition
   failures even in overridden inputs, and deliberately rebuild/swap engines
   after definition changes.
@@ -486,8 +497,9 @@ Expected consumer migration:
   custom backend, fallback, and embedded workflows in the migration guidance.
 
 The subsequent feature roadmap should carry forward these choices, resolve the
-upstream API details, and specify the target end state. The staged implementation
+remaining PromptKit integration decisions, and specify the target end state.
-plan must sequence the prerequisite ConfigKit releases before dependent PromptKit
+The staged implementation plan can now adopt released ConfigKit support without
-work, then cover deletions/replacements, contract tests, canonical documentation,
+waiting for upstream feature work, coordinating any catalog format releases before
+their consumption. Cover deletions/replacements, contract tests, canonical documentation,
 examples, attribution, and minor-release migration notes. Neither document should
 become a reason to preserve obsolete compatibility machinery.