eric/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Clarify configuration inspection and execution boundaries

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-29 16:56:50 +00:00
parent 7b012033a6
commit 34f5980973
1 changed files with 52 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

58
docs/roadmap/configkit-assessment.md
View File

@@ -116,12 +116,18 @@ optional upstream credentials, and the distinction between failed generated
content and operational errors. This project does not authorize silently
dropping execution features to simplify the configuration layer.
Add explicit configuration inspection for consumers implementing operator-facing
CLI commands or debug output. This is a new capability motivated by this adoption,
not an assertion that the current prompt/profile inspection methods already
provide configuration provenance.
## 2. Responsibility Mapping
| Concern | Recommended owner and disposition |
| --- | --- |
| YAML grammar, scalar interpretation, one-document enforcement, unknown fields, typed ranges, null legality | Delegate to ConfigKit; remove corresponding PromptKit parser logic and redundant parser tests |
| Field presence and source coordinates | Delegate to ConfigKit; retain only the compact records needed for resolution and diagnostics |
| Field presence, source coordinates, and configuration provenance | Delegate to ConfigKit; retain its inspection state with the loaded snapshot for explicit operator queries, separate from execution-ready values |
| Operator-facing configuration inspection | Provide a PromptKit consumer API reusing ConfigKit inspection facilities; applications own CLI commands, presentation, and disclosure policy |
| File/directory/embedded source selection and definition ID indexing | Retain a small PromptKit collection loader; ConfigKit loads documents, not a prompt catalog |
| Configuration filesystem sources, relative imports, and generic layer origins | Delegate to the prerequisite ConfigKit APIs; do not maintain parallel reader-based loading machinery |
| Definition discovery and relative message-content references | Retain PromptKit's domain resource adapters, using ConfigKit source context and standard `io/fs` facilities where sufficient |
@@ -286,8 +292,41 @@ Applications using ConfigKit can present the same underlying loading diagnostics
for their application settings and PromptKit files. They can also translate
application-owned typed settings into PromptKit profiles/backends. Do not require
applications to encode Go clients into YAML or adopt ConfigKit merely to call
PromptKit's runtime API. No full effective-configuration/explanation API is proposed
without an identified consumer requirement.
PromptKit's runtime API. The operator-inspection capability below supplies the
concrete consumer purpose for exposing effective configuration and its provenance.
### Explicit Configuration Inspection
Provide a consumer-facing API for inspecting effective configuration and its
provenance so applications can implement targeted CLI commands or debug output.
Keep three activities distinct:
- **Construction:** load and resolve configuration, retain ConfigKit's provenance
and inspection state, and publish execution-ready definitions. Resolve any
presence and origin information needed for definition construction here.
- **Ordinary execution:** use the ready definitions directly. Do not analyze
configuration provenance, traverse origin/history records, reconstruct
inheritance explanations, or render effective-configuration reports as part of
a prompt run. Carry execution-relevant presence explicitly in resolved values
rather than querying configuration metadata during each run.
- **Explicit inspection:** query effective settings and their contributing
sources or explanations only when a consumer invokes the inspection API.
Inspection uses the same loaded configuration snapshot as the engine; it does
not reread configuration files, open referenced resources, or contact providers.
Reuse ConfigKit's inspection and redaction facilities rather than maintaining a
second provenance system. The API should make targeted queries practical; exact
signatures, query scope, and safe result ownership belong in the feature roadmap.
Do not imply that configuration inspection predicts all runtime request overrides
or reveals the values of execution-time credentials. Existing prompt/profile
inspection, prepared-run details, and execution observations have distinct purposes;
configuration inspection does not replace them or add provenance to run events.
Applications decide whether and how to expose inspection through operator tools.
PromptKit adds no CLI, implicit debug logging, or automatic explanation output.
Redacted configuration can still contain sensitive application content, so callers
retain responsibility for access control and disclosure. Retention has a memory
cost, but explanation generation and provenance analysis stay off the run path.
## 4. Simplification Accounting
@@ -322,8 +361,9 @@ coverage because construction fixtures need migration.
Runtime benefit: ordinary runs no longer rediscover definition directories,
redecode definitions, reread template text, or traverse inheritance chains.
Cost: construction loads even unused definitions and retains their text and
resolved values. ConfigKit also creates metadata and projections; keep only needed
records after loading. These are reasoned tradeoffs, not measured performance
resolved values. Retain ConfigKit inspection state needed by the explicit operator
API without duplicating its history in runtime definitions or per-run values.
These are reasoned tradeoffs, not measured performance
claims. Adoption is not justified by the decoder replacement alone.
### Algorithmic Efficiency
@@ -344,7 +384,8 @@ input and produced-data size, not a blanket O(n) claim with an undefined n.
- Avoid whole-collection copying per layer, repeated serialization, and repeated
prefix/history traversal. Analyze layer processing against total input nodes
and produced values/provenance, including intentionally retained history.
- Keep configuration discovery and decoding out of ordinary execution. Copy only
- Keep configuration discovery, decoding, and provenance analysis out of ordinary
execution. Perform explanation work only on explicit inspection calls. Copy only
where ownership requires it; do not trade caller isolation for speed.
These are design targets, not promised complexity for every operation. Sorting
@@ -403,6 +444,11 @@ the source boundary.
4. **Diagnostic boundary.** Keep wrapping ConfigKit's public loading errors rather
than mirroring them. Retain actionable domain relationship and resource errors
without duplicating ConfigKit's generic diagnostic/provenance system.
5. **Operator inspection.** Expose effective configuration and provenance through
an explicit consumer API backed by the engine's loaded snapshot. Retain the
required ConfigKit inspection state at construction; never analyze it during
ordinary prompt execution. Finalize targeted query shapes and ownership in the
feature roadmap, with presentation remaining application-owned.
Support the upstream recommendations for labeled mapping-based programmatic input
and direct filesystem resource-context queries. If real consumer examples show