diff --git a/docs/roadmap/configkit-assessment.md b/docs/roadmap/configkit-assessment.md index f8689d1..40ce790 100644 --- a/docs/roadmap/configkit-assessment.md +++ b/docs/roadmap/configkit-assessment.md @@ -1,8 +1,9 @@ # ConfigKit Adoption Assessment -Status: assessment with agreed design direction; upstream API details remain -to be settled before the feature roadmap and implementation plan. No runtime -changes are part of this assessment. +Status: assessment aligned to released ConfigKit v0.5.0. Upstream capability +prerequisites are available; PromptKit integration and migration decisions remain +for the feature roadmap and implementation plan. No runtime changes are part of +this assessment. ## Goal And Recommendation @@ -14,8 +15,8 @@ and their migration paths. Recommend adoption through an eager, immutable definition-loading boundary. ConfigKit should own YAML interpretation, source-tier combination, strict typed -decoding, presence, and source diagnostics. PromptKit should own definition identity, named profile -relationships, LLM-specific rules, and execution. Do not retain the lazy loading +decoding, sibling inheritance, presence, and source diagnostics. PromptKit should +own definition selection, LLM-specific rules, and execution. Do not retain the lazy loading architecture merely to minimize migration, or introduce a second generic configuration framework inside PromptKit. @@ -25,69 +26,53 @@ lazy error arbitration, and repeated profile-chain resolution. Some of those savings come from the loading redesign rather than ConfigKit itself. ConfigKit adds ongoing value by owning the generic decoding behavior and its tests. -## Prerequisite: Proposed Upstream ConfigKit Work +## Released ConfigKit Capabilities -Design, implement, and release these two ConfigKit capabilities before the -dependent PromptKit refactor. The remaining recommendations assume this upstream -work; finalize their integration details against its agreed public contracts. -These are selected architectural prerequisites, not a claim that adoption is -technically impossible using today's reader-based API. +ConfigKit v0.5.0 satisfies the two requested prerequisites and also implements +sibling inheritance. Adopt the released public contracts rather than waiting for +the earlier proposals to be implemented: -1. **Native `fs.FS` sources and source-relative context.** Support embedded and - caller-supplied filesystems as first-class configuration sources. Define source - identity, relative path resolution, ownership, and how enabled imports and - overlays operate within that filesystem. This is broadly useful for libraries - bundling default configuration. Passing an opened file to today's `Reader` - supplies bytes and a label, but does not make filesystem-relative loading - native to ConfigKit; that workaround is not the target architecture. -2. **Source-backed underlays with origins and programmatic input.** Adopt - ConfigKit's working design: strictly compose contributions within each tier, - then recursively overlay explicitly ordered tiers. Support embedded, host-file, - byte/reader, and labeled programmatic sources without temporary files or YAML - serialization round trips. Prefer the proposed string-keyed programmatic - mappings: omitted keys convey absence; present values, including nil/null, - convey explicit input. Do not require typed-value ingestion as a prerequisite. +1. **Filesystem sources:** `NewFilesystem(label, fsys).File(name)` borrows an + `fs.FS` with reusable namespace identity. Imports and resource contexts retain + that namespace. `Result.SourceContext` and `Resolution.SourceContext` expose + passive `ResourceContext` values with `FS` and `BaseDir`. +2. **Source-backed underlays:** `LoadLayered` accepts a main `Input` + and ordered `Underlay` groups. Each group composes strictly; groups overlay + recursively from lowest to highest priority. `Input{Source: NoSource()}` omits + the main input. `Mapping(label, map[string]any)` snapshots supported built-in + scalar/list/map values with explicit presence; it does not ingest arbitrary + typed structs. Generic programmatic input is not represented as CLI overrides. +3. **Sibling inheritance:** the reserved `extends` key names one immediate sibling + mapping. Relationships resolve after precedence assembly and before typed + decoding, with inherited origins, history, and resource context preserved. + This is built-in behavior, not the opt-in/registered-collection API originally + suggested by PromptKit. Adopt ConfigKit's recursive combination model rather than preserving PromptKit's whole-profile shadowing or atomic provider-map replacement across source tiers. -No hidden replacement pass should run after ConfigKit loading. Named inheritance -is a separate optional upstream request, not part of these prerequisites. - -The upstream design should show how the two capabilities work together, including -source origins and efficient ownership/copying. Resolve these API questions before -turning the assessment into a decision-complete PromptKit implementation plan. -This assessment does not authorize code changes in either repository. +No hidden replacement or inheritance pass should run after ConfigKit loading. +Use released sibling inheritance for profiles, leaving domain validation in +PromptKit. This assessment does not authorize code changes in either repository. ## Evidence And Scope -Reviewed local baselines on 2026-09-29: +PromptKit's runtime baseline remains v0.11.2; this assessment has since received +documentation-only revisions. Updated upstream review on 2026-09-30: - PromptKit: `12d6d80024e80c552d122a076c22429bf6f2f6c2` (v0.11.2). -- ConfigKit: `67c03690181c5af8dde6be325abac241dae6732f` (v0.4.2). - -Design update: reviewed ConfigKit's working -`docs/roadmap/sources-and-layering.md` with checkout HEAD -`36172ed20e832b371506d6aceb02dc6c8f50f71b`. The reviewed roadmap included -uncommitted changes, so that commit alone does not identify the reviewed text. -It remains a proposed target, not implemented behavior. The recommendations here -now follow its underlay semantics and support its two open-decision recommendations: -labeled programmatic mappings and direct source-context queries. Verify the -final upstream contracts before preparing dependent implementation details. +- ConfigKit: `dc09959cca4e105f49ec316c6d4ba585bd2340bd` (v0.5.0), verified + against the published tag. Its clean checkout passed `go test ./...` during + the release review. This replaces the earlier v0.4.2 and uncommitted-roadmap + comparison as the upstream basis for adoption. PromptKit evidence: [consumer workflows](../consumers/pkg-promptkit.md), [file formats](../formats.md), [source implementation](../internal/sources.md), [runner](../internal/runner.md), and the implementation owners identified below. -ConfigKit evidence: its reviewed `docs/config.md`, `docs/roadmap/deferred.md`, -`lifecycle.go`, `source.go`, and ADR 0001 in the sibling checkout. That checkout -is review context, not a proposed build dependency or required directory layout. - -ConfigKit currently exposes `Load[T]` over file, byte, reader, and defaults-only -sources. It supports strict decoding, presence/origin records, defaults, -recorded resolution, semantic validation, explicit imports, one selected -filesystem overlay, and secret handling. Its parser and generic conversion -machinery are private. It has no public arbitrary layer-composition, generic -inheritance, or `fs.FS` source API. Its deferred scope explicitly excludes several -of these generalizations; an upstream enhancement requires a deliberate decision. +ConfigKit evidence at that release: `docs/config.md`, `docs/releases/v0.5.0.md`, +`layered.go`, `source.go`, `resource_context.go`, and ADR 0009 in the sibling +checkout. That checkout is review context, not a build dependency or required +directory layout. The released configuration contract owns exact upstream behavior; +the summaries here identify its consequences for the proposed PromptKit design. This is a source-and-contract assessment, not a performance benchmark or full correctness audit. It does not claim measured allocation, latency, or line-count @@ -132,7 +117,8 @@ provide configuration provenance. | Configuration filesystem sources, relative imports, and generic layer origins | Delegate to the prerequisite ConfigKit APIs; do not maintain parallel reader-based loading machinery | | Definition discovery and relative message-content references | Retain PromptKit's domain resource adapters, using ConfigKit source context and standard `io/fs` facilities where sufficient | | Profile-source priority and combination | PromptKit selects tiers; ConfigKit owns strict within-tier composition and recursive cross-tier overlay; eliminate repository fallback and whole-definition shadowing | -| Named profile inheritance and backend selection | PromptKit owns domain relationships unless the separate inheritance proposal is adopted; integrate resolution with ConfigKit's recorded lifecycle | +| Named profile inheritance | Delegate sibling graph resolution, combination, and derivation to ConfigKit's `extends`; remove the custom profile resolver | +| Backend selection and domain normalization | Retain PromptKit domain rules; use recorded application resolution for configuration transformations not already handled by structural inheritance | | LLM setting bounds, role/session normalization, output rules, reserved provider fields | Retain shared PromptKit domain owners; ConfigKit validation callbacks may invoke them | | Definition defaults | Use ConfigKit defaults where they do not erase inheritance presence; apply effective execution defaults after inheritance | | Programmatic configuration | Prefer labeled mapping input with explicit presence; evaluate whether typed PromptKit convenience constructors justify conversion code, without YAML round trips or zero-value inference | @@ -162,7 +148,8 @@ in the feature roadmap. Support OS directories, individual files, embedded/custo `fs.FS` sources, and programmatic input through the new source model; source selection and definition discovery remain PromptKit concerns, not a second composition engine. -Perform domain-specific resolution through ConfigKit's recorded lifecycle before +ConfigKit expands structural inheritance before typed defaults/decoding. Perform +any remaining domain-specific resolution through its recorded lifecycle before semantic validation and result publication. Reuse its presence and provenance; do not add a generic merge/defaulting pass or reconstruct origins after loading. Runtime client assembly and resource interpretation remain outside typed decoding. @@ -174,7 +161,7 @@ source-relative content resolution and containment without expanding the threat model. A snapshot is a set of captured reads, not an atomic filesystem transaction. Resolve each effective content reference using its contributing source's context, -not the highest tier's directory. Support ConfigKit's proposed direct context +not the highest tier's directory. Use ConfigKit's released direct context query rather than a consumer-maintained filesystem registry. Composite origins must not be assigned an arbitrary directory; ambiguous or absent context needs an actionable resource error or explicit consumer-supplied context. ConfigKit @@ -212,7 +199,7 @@ not a reason to impose intermediate typed validation on ordinary inputs. Preserve profile templates that become usable through descendants or runtime overrides: eager loading must not require every consumer profile to be an independently runnable endpoint/model pair. Validate supplied fields and graph -integrity eagerly (including missing bases and cycles), and check final execution +integrity eagerly (ConfigKit handles missing bases and cycles), and check final execution requirements after runtime selection/overrides. Prompt default-profile references can be checked against the assembled collection; a prompt with no default remains valid for callers that supply a profile explicitly. @@ -230,7 +217,7 @@ If a separate configuration schema version is later needed, respect ConfigKit's designated main-input ownership and pre-underlay check timing; do not assume last-wins version selection across catalogs. -### Source Presence And Separate Named Inheritance +### Source Presence And Released Sibling Inheritance Adopt ConfigKit source-layer semantics: absence retains lower input; explicit zero, false, and empty strings replace it; mappings combine recursively; empty @@ -247,21 +234,40 @@ profile fields in ways that erase that distinction before relationship resolutio Provider `extra_params` maps combine recursively across source tiers too: `{}` does not clear lower keys. Retire the proposed atomic source-map replacement and null-as-unspecified conventions; do not emulate them after loading. An operator -overriding only `model` also retains a lower `base_profile` declaration and other +overriding only `model` also retains a lower `extends` declaration and other omitted settings. Diagnose and document these consequences explicitly. -Named inheritance still selects relationships between effective entries; source -layering does not implement it. The optional upstream request remains pending. -Finalize named-inheritance null/map semantics separately, favoring consistency -with source layering rather than automatically retaining atomic maps or null-as- -inherit. Any domain exception needs explicit justification. This also does not -silently change the existing request-level override contract, which is a separate -runtime boundary to review in the feature roadmap. +Replace `base_profile` with `extends` on sibling profile mappings. A string names +one exact immediate sibling; null cancels the relationship while preserving the +child's own settings. Omission retains a lower-tier directive. There is no ancestor +scope search or cross-collection reference. Superseded references are history, +not active relationships to validate; every effective relationship resolves, +including profiles not selected for execution. -Resolve shared ancestors once per snapshot. Use recorded domain resolution to -keep resulting values and origins consistent. Confirm the upstream lifecycle can -express required relationship provenance; do not claim generic layer history -already explains named inheritance or invent a parallel audit framework. +Children recursively overlay their resolved parents with the same value rules: +zero/false/empty strings and sequences/null replace, mappings combine, and `{}` +does not clear inherited keys. Inherited values count as explicit input for +presence queries and absence-based defaults, even when false, zero, or null. +Do not repeat inheritance in PromptKit's application-resolution callback. Use +`Origin.Inheritance` and retained history for explicit operator inspection. +Inherited resource references retain their declaring source's context. + +Enclosing inheritance resolves before nested relationships; nested inherited +directives resolve at each destination against its siblings. Defaults and +application-derived values do not trigger another inheritance pass. Model this +deliberately in PromptKit configuration rather than promising arbitrary graph +resolution over runtime values. Partial bases must fit the typed schema and +PromptKit's validation; ConfigKit grants them no special validation exemption. + +`extends` is reserved in every input mapping, including provider-parameter maps, +across all source kinds. There is no opt-in switch or ordinary-data escape in the +released contract. Audit catalog and consumer payload needs before finalizing +their format; a literal provider field with that name cannot silently pass through +the configuration loader. Do not invent an encoding workaround in this assessment. + +The existing request-level override API remains a separate runtime boundary. +The feature roadmap must review its interaction with configured values without +silently subjecting runtime request data to configuration inheritance directives. ### Reloading, Errors, And Application Integration @@ -335,7 +341,7 @@ cost, but explanation generation and provenance analysis stay off the run path. | `internal/profile/filesystem_repository.go`: metadata scans, selected strict decode, stream-end checks, malformed-file selection arbitration | Replace with source acquisition, ConfigKit assembly/decoding, and indexed effective values; remove the metadata prepass and per-lookup filesystem path | | `internal/promptdef/filesystem_repository.go`: metadata selection and repeated decoding/content opening | Replace with eager prompt loads and an ID/version index; keep domain normalization and contained content reading | | Profile filesystem, `fs.FS`, eager, memory, and overlay repository variants | Collapse loading into source adapters plus immutable collections; preserve only interfaces that still serve a real boundary | -| `internal/profile/resolving_repository.go` | Replace per-operation traversal and zero-sentinel merging with once-per-snapshot presence-aware typed resolution; graph logic remains | +| `internal/profile/resolving_repository.go` | Remove custom graph traversal, cycle detection, and zero-sentinel merging in favor of ConfigKit sibling inheritance; retain only PromptKit domain validation | | `internal/engine` raw repository-stack assembly | Replace with ConfigKit source tiers and one effective typed collection; engine capacity/client assembly remains | | `internal/catalog` profile metadata extraction | Obtain explicit field information from ConfigKit; retain manifests, ownership, namespace, connection-metadata and secret checks | | `internal/filecatalog` | Retain/reduce discovery and containment helpers actually needed by the eager loader; do not claim the entire package disappears | @@ -377,10 +383,12 @@ input and produced-data size, not a blanket O(n) claim with an undefined n. captured content instead of reparsing it for metadata, selection, and validation. - Build identity indexes in one pass, with expected constant-time map lookup. Do not scan whole collections for every lookup or precedence decision. -- Process inheritance graph structure in O(V + E), where V is the number of - profiles and E the inheritance edges. Resolve shared ancestors once. Account - separately for the size of materialized settings and retained origins; copying - large inherited maps to many independent results necessarily costs more than V. +- Delegate inheritance expansion to ConfigKit once during loading rather than + rebuilding its graph in PromptKit. Assess scaling against input, materialized + inherited values, nested destination resolution, and retained history; do not + claim unconditional O(V + E) total work from the count of profiles and edges. + ConfigKit bounds expansion and retained evidence through its existing budgets; + representative catalog sizes must fit them without new per-run processing. - Avoid whole-collection copying per layer, repeated serialization, and repeated prefix/history traversal. Analyze layer processing against total input nodes and produced values/provenance, including intentionally retained history. @@ -397,17 +405,13 @@ The implementation plan should include focused benchmarks for consequential repeated-work risks, such as shared-ancestor profiles and many layers, rather than machine-dependent timing gates or benchmarks for every helper. -## 5. Additional Upstream Opportunities +## 5. Upstream Readiness And Remaining Opportunities -Native filesystem sources and public layered loading with origins are the -prerequisites described near the beginning, not optional follow-ups. The following -remain optional and do not block adoption unless the upstream design establishes -a concrete dependency: - -| Candidate | Recommendation and tradeoff | -| --- | --- | -| Generic named configuration inheritance | Requested separately in ConfigKit's `docs/roadmap/feature-request-promptkit-inheritance.md`, primarily for shared semantics across consumers. ConfigKit decides whether to adopt it; PromptKit can retain a small domain resolver if declined. Presence-aware source layering belongs to the prerequisite work, not this optional proposal. | -| Context-aware loading | Optional for reusable cancellable construction if large-source consumers need it. Promise cooperative cancellation only; no detached reads or instant interruption of arbitrary readers. | +Filesystem sources, underlays, programmatic mappings, source contexts, and sibling +inheritance are released in v0.5.0. No additional upstream feature is currently a +demonstrated prerequisite. Context-aware loading remains a possible future request +if concrete consumers need it, not a reason to delay adoption. Existing loading +is synchronous; do not simulate cancellation with detached background work. Do not propose a public raw AST, permissive identity-only decoder, or dynamic subtree conversion solely to preserve PromptKit's current scanning design. @@ -418,12 +422,12 @@ ConfigKit and PromptKit both declare LGPL-3.0-or-later. Adoption must still upda dependency notices under the OSS policy and inspect the exact selected release and its transitive notices. Their current Go minimums do not require a PromptKit toolchain increase. Use a released module dependency, not a sibling-directory -replacement. This assessment proposes upstream work, but makes no upstream edits -or licensing change. +replacement. Select v0.5.0 as the assessed adoption baseline; any newer selection +needs its own review. This assessment changes neither dependencies nor licenses. ## 6. Decisions And Migration -The agreed direction now incorporates ConfigKit's working roadmap. This supersedes +The agreed direction now incorporates ConfigKit's v0.5.0 contracts. This supersedes the earlier recommendations for independently typed documents, whole-definition shadowing, intermediate typed validation, and atomic maps/null-as-unspecified at the source boundary. @@ -438,9 +442,9 @@ the source boundary. typed loads followed by custom combination are not the selected direction. 3. **Presence and relationships.** Accept recursive source-tier overlay, retained children for empty mappings, explicit null replacement, and effective metadata. - Named inheritance remains separate pending the upstream decision; settle its - domain semantics before implementation rather than assuming either automatic - equivalence or an exception to ConfigKit's rules. + Adopt built-in sibling inheritance, including reserved `extends`, recursive + mapping combination, null cancellation, and inherited explicit presence. Do + not preserve a parallel PromptKit structural resolver. 4. **Diagnostic boundary.** Keep wrapping ConfigKit's public loading errors rather than mirroring them. Retain actionable domain relationship and resource errors without duplicating ConfigKit's generic diagnostic/provenance system. @@ -450,14 +454,17 @@ the source boundary. ordinary prompt execution. Finalize targeted query shapes and ownership in the feature roadmap, with presentation remaining application-owned. -Support the upstream recommendations for labeled mapping-based programmatic input -and direct filesystem resource-context queries. If real consumer examples show +Use released mapping-based programmatic input and direct filesystem resource-context +queries. If real consumer examples show mapping construction to be repetitive, consider small typed PromptKit conveniences with explicit presence rather than requiring an upstream typed-ingestion API. -Final upstream signatures and ownership details remain open dependencies of the -feature roadmap, not implemented guarantees. Resolve the named-inheritance -direction there as well; a PromptKit-owned resolver remains viable if ConfigKit -declines or defers that optional request. + +Remaining PromptKit decisions are the exact collection/version format, catalog +migration, programmatic ergonomics, operator-inspection surface, and validation +of partial base profiles. Also settle how operators discard unwanted provider +settings or old relationships using the released null semantics without a hidden +replacement pass. Check the reserved-key impact on arbitrary provider maps. These +are integration decisions, not unresolved ConfigKit API proposals. Expected consumer migration: @@ -467,11 +474,15 @@ Expected consumer migration: for historical YAML coercion. - Move standalone definitions into the agreed keyed contribution format. Review same-name contributions: disjoint fields within a tier can combine, and higher - tiers retain omitted lower fields, including `base_profile` references. + tiers retain omitted lower fields, including `extends` references. Convert + `base_profile` to sibling `extends`; use `extends: null` to cancel a lower-tier + relationship, not to erase all settings already supplied to the child. - Review zero/false/empty/null values explicitly. Provider maps merge across tiers, `{}` does not clear them, and null replaces rather than means omission. Use labeled programmatic mappings or approved presence-aware conveniences. - Document the separately selected inheritance semantics when settled. + Apply those same combination rules to inherited settings. Rename ordinary + configuration keys named `extends`; examine provider maps for cases where + renaming would change their substantive meaning and resolve those explicitly. - Handle effective-definition errors at construction, fix syntax/composition failures even in overridden inputs, and deliberately rebuild/swap engines after definition changes. @@ -486,8 +497,9 @@ Expected consumer migration: custom backend, fallback, and embedded workflows in the migration guidance. The subsequent feature roadmap should carry forward these choices, resolve the -upstream API details, and specify the target end state. The staged implementation -plan must sequence the prerequisite ConfigKit releases before dependent PromptKit -work, then cover deletions/replacements, contract tests, canonical documentation, +remaining PromptKit integration decisions, and specify the target end state. +The staged implementation plan can now adopt released ConfigKit support without +waiting for upstream feature work, coordinating any catalog format releases before +their consumption. Cover deletions/replacements, contract tests, canonical documentation, examples, attribution, and minor-release migration notes. Neither document should become a reason to preserve obsolete compatibility machinery.