From 34f59809733af784fbea5a143923740ac88ba213 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Tue, 29 Sep 2026 16:56:50 +0000 Subject: [PATCH] Clarify configuration inspection and execution boundaries --- docs/roadmap/configkit-assessment.md | 58 +++++++++++++++++++++++++--- 1 file changed, 52 insertions(+), 6 deletions(-) diff --git a/docs/roadmap/configkit-assessment.md b/docs/roadmap/configkit-assessment.md index 2f316d7..f8689d1 100644 --- a/docs/roadmap/configkit-assessment.md +++ b/docs/roadmap/configkit-assessment.md @@ -116,12 +116,18 @@ optional upstream credentials, and the distinction between failed generated content and operational errors. This project does not authorize silently dropping execution features to simplify the configuration layer. +Add explicit configuration inspection for consumers implementing operator-facing +CLI commands or debug output. This is a new capability motivated by this adoption, +not an assertion that the current prompt/profile inspection methods already +provide configuration provenance. + ## 2. Responsibility Mapping | Concern | Recommended owner and disposition | | --- | --- | | YAML grammar, scalar interpretation, one-document enforcement, unknown fields, typed ranges, null legality | Delegate to ConfigKit; remove corresponding PromptKit parser logic and redundant parser tests | -| Field presence and source coordinates | Delegate to ConfigKit; retain only the compact records needed for resolution and diagnostics | +| Field presence, source coordinates, and configuration provenance | Delegate to ConfigKit; retain its inspection state with the loaded snapshot for explicit operator queries, separate from execution-ready values | +| Operator-facing configuration inspection | Provide a PromptKit consumer API reusing ConfigKit inspection facilities; applications own CLI commands, presentation, and disclosure policy | | File/directory/embedded source selection and definition ID indexing | Retain a small PromptKit collection loader; ConfigKit loads documents, not a prompt catalog | | Configuration filesystem sources, relative imports, and generic layer origins | Delegate to the prerequisite ConfigKit APIs; do not maintain parallel reader-based loading machinery | | Definition discovery and relative message-content references | Retain PromptKit's domain resource adapters, using ConfigKit source context and standard `io/fs` facilities where sufficient | @@ -286,8 +292,41 @@ Applications using ConfigKit can present the same underlying loading diagnostics for their application settings and PromptKit files. They can also translate application-owned typed settings into PromptKit profiles/backends. Do not require applications to encode Go clients into YAML or adopt ConfigKit merely to call -PromptKit's runtime API. No full effective-configuration/explanation API is proposed -without an identified consumer requirement. +PromptKit's runtime API. The operator-inspection capability below supplies the +concrete consumer purpose for exposing effective configuration and its provenance. + +### Explicit Configuration Inspection + +Provide a consumer-facing API for inspecting effective configuration and its +provenance so applications can implement targeted CLI commands or debug output. +Keep three activities distinct: + +- **Construction:** load and resolve configuration, retain ConfigKit's provenance + and inspection state, and publish execution-ready definitions. Resolve any + presence and origin information needed for definition construction here. +- **Ordinary execution:** use the ready definitions directly. Do not analyze + configuration provenance, traverse origin/history records, reconstruct + inheritance explanations, or render effective-configuration reports as part of + a prompt run. Carry execution-relevant presence explicitly in resolved values + rather than querying configuration metadata during each run. +- **Explicit inspection:** query effective settings and their contributing + sources or explanations only when a consumer invokes the inspection API. + Inspection uses the same loaded configuration snapshot as the engine; it does + not reread configuration files, open referenced resources, or contact providers. + +Reuse ConfigKit's inspection and redaction facilities rather than maintaining a +second provenance system. The API should make targeted queries practical; exact +signatures, query scope, and safe result ownership belong in the feature roadmap. +Do not imply that configuration inspection predicts all runtime request overrides +or reveals the values of execution-time credentials. Existing prompt/profile +inspection, prepared-run details, and execution observations have distinct purposes; +configuration inspection does not replace them or add provenance to run events. + +Applications decide whether and how to expose inspection through operator tools. +PromptKit adds no CLI, implicit debug logging, or automatic explanation output. +Redacted configuration can still contain sensitive application content, so callers +retain responsibility for access control and disclosure. Retention has a memory +cost, but explanation generation and provenance analysis stay off the run path. ## 4. Simplification Accounting @@ -322,8 +361,9 @@ coverage because construction fixtures need migration. Runtime benefit: ordinary runs no longer rediscover definition directories, redecode definitions, reread template text, or traverse inheritance chains. Cost: construction loads even unused definitions and retains their text and -resolved values. ConfigKit also creates metadata and projections; keep only needed -records after loading. These are reasoned tradeoffs, not measured performance +resolved values. Retain ConfigKit inspection state needed by the explicit operator +API without duplicating its history in runtime definitions or per-run values. +These are reasoned tradeoffs, not measured performance claims. Adoption is not justified by the decoder replacement alone. ### Algorithmic Efficiency @@ -344,7 +384,8 @@ input and produced-data size, not a blanket O(n) claim with an undefined n. - Avoid whole-collection copying per layer, repeated serialization, and repeated prefix/history traversal. Analyze layer processing against total input nodes and produced values/provenance, including intentionally retained history. -- Keep configuration discovery and decoding out of ordinary execution. Copy only +- Keep configuration discovery, decoding, and provenance analysis out of ordinary + execution. Perform explanation work only on explicit inspection calls. Copy only where ownership requires it; do not trade caller isolation for speed. These are design targets, not promised complexity for every operation. Sorting @@ -403,6 +444,11 @@ the source boundary. 4. **Diagnostic boundary.** Keep wrapping ConfigKit's public loading errors rather than mirroring them. Retain actionable domain relationship and resource errors without duplicating ConfigKit's generic diagnostic/provenance system. +5. **Operator inspection.** Expose effective configuration and provenance through + an explicit consumer API backed by the engine's loaded snapshot. Retain the + required ConfigKit inspection state at construction; never analyze it during + ordinary prompt execution. Finalize targeted query shapes and ownership in the + feature roadmap, with presentation remaining application-owned. Support the upstream recommendations for labeled mapping-based programmatic input and direct filesystem resource-context queries. If real consumer examples show