eric/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Define captured configuration comparison roadmap

Browse Source
This commit is contained in:
Eric Rakestraw
2026-10-09 15:41:53 +00:00
parent 18ca21fa45
commit 26ab566f38
1 changed files with 142 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

142
docs/roadmap/diff.md Normal file
View File

@@ -0,0 +1,142 @@
# Captured Configuration Comparison
Status: accepted; implementation pending.
## Purpose
Let consumers compare two captured configuration inspections for operator-facing
workflows: reviewing a candidate configuration, explaining the effect of an
overlay, or reporting changes between independently constructed engines.
PromptKit should expose a thin API over ConfigKit's comparison functionality,
leaving comparison and sensitivity handling upstream.
This feature extends inspection, not prompt execution. It does not require
PromptKit to retain previous engines, manage reloads, or decide whether a
candidate configuration should be activated.
## Current Baseline
PromptKit currently depends on ConfigKit v0.6.0. `Engine.InspectConfiguration`
returns a captured `ConfigurationInspection` with metadata, source descriptions,
diagnostics, path explanations, and effective-configuration output, but no
comparison operation. These methods already expose ConfigKit's public inspection
types while keeping the typed configuration result private.
ConfigKit v0.7.0 adds effective-configuration comparison without changing the
loading contracts used by PromptKit. The reviewed release corresponds to commit
`8f3396d6f0cbe03af4f92bb2c1a1c9798171dd52`. See its
[release notes](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.7.0/docs/releases/v0.7.0.md)
and [public comparison API](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.7.0/diff.go).
## Target Public API
Add a method to the existing inspection value:
```go
func (v ConfigurationInspection) Diff(other ConfigurationInspection) ([]configkit.Difference, error)
```
The receiver supplies the left (earlier) configuration and `other` supplies the
right (later) configuration. Consumers can compare views from separate engines
or compare a view with itself. Returning ConfigKit's public `Difference` type is
consistent with the existing inspection API and avoids a parallel PromptKit
representation of the same information.
Comparing either operand's zero inspection value returns `ErrInvalidConfig` and
no difference records, consistent with existing inspection operations that
require a captured view. Errors from the underlying comparison remain available
to the caller; a new comparison-specific error hierarchy is unnecessary.
## Comparison Meaning And Boundaries
Delegate to ConfigKit's frozen effective-value comparison. Equal effective
values produce no records even when their source or resolution histories differ.
ConfigKit owns comparison granularity, canonical path ordering, and detached
value snapshots. Its public contract distinguishes absent values from explicit
nulls, compares mappings recursively, and treats sequences as atomic values.
Sensitivity classifications from both views protect both sides of the output.
Pass through ConfigKit's `Redacted` and `HiddenChanged` information rather than
implementing masking or hidden-change detection in PromptKit. Existing PromptKit
sensitivity classifications remain applicable. Ordinary prompt and provider
content is not automatically safe to disclose; consumers still choose where and
to whom to present reports. See the upstream
[comparison guidance](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.7.0/docs/config.md#effective-configuration-diffs)
for the detailed semantics.
The comparison operates on the configuration captured at the ConfigKit handoff,
consistent with [ADR 0001](../adr/0001-configuration-boundary.md). It does not
reread sources, resolve credentials, render prompts, or call providers. It is an
explicit inspection operation, not an additional step during ordinary prompt
execution.
An empty diff means equality of the compared configuration values, not equality
of engine behavior. For example, a referenced template file can change while its
configured path stays the same, and secret contents can change while their
configured reference stays the same. Post-load execution preparation and
request-specific overrides are not part of this comparison.
## Work Needed
### Dependency Adoption
- Upgrade the ConfigKit requirement to v0.7.0 and update module checksums.
- Verify PromptKit's existing loading, inspection, and execution behavior against
the new dependency through the relevant tests and maintained examples.
- Refresh ConfigKit's version and exact upstream source references in the
[attribution document](../oss/attribution.md), following the
[OSS policy](../policy/oss.md). The reviewed release retains its existing
licensing arrangements.
### Thin Inspection Integration
- Provide a private comparison bridge for the retained ConfigKit results in
`internal/configuration`, preserving the private configuration type and the
existing inspection operations.
- Add the public method in `configuration_inspection.go`, delegating through
that bridge. Keep generic results and internal configuration structures out of
the public API.
- Reuse captured inspection state and upstream comparison logic. No changes to
configuration formats, source precedence, or execution routing are needed.
### Focused Verification
- Exercise the public method with independently constructed engines, an equal
pair, and a representative changed value. Include equal effective values
produced by different source histories.
- Verify operand direction, zero-view errors, and propagation of upstream
records through the private bridge.
- Verify paired sensitivity protection and hidden-change reporting using
PromptKit's existing credential-shaped provider fields. Check representative
presence and sequence behavior without duplicating ConfigKit's comparator
test suite.
- Check that comparison uses captured state after source changes and that
returned records do not let callers mutate retained inspection state.
- Exercise concurrent inspection comparisons under the race detector. Keep
tests offline and focused on PromptKit's integration boundary, following the
[testing policy](../policy/testing.md).
### Documentation
- Add GoDoc for the method's direction, errors, snapshot boundary, and upstream
result semantics.
- Add a short operator-oriented usage example to the
[consumer guide](../consumers/pkg-promptkit.md), linking to GoDoc and ConfigKit
for the detailed contract.
- Update the [internal source document](../internal/sources.md) to describe the
private comparison bridge and refresh dependency-specific discussion where
needed. Record the feature in the applicable release notes when preparing
its release.
## Scope And Completion
This unit of work is the dependency upgrade and a thin inspection comparison
method, with proportionate tests and documentation. Custom diff algorithms,
formatters, CLIs, patch application, automatic reloads, and execution-equivalence
analysis are outside this unit of work.
The feature is complete when consumers can compare captured inspection views
through PromptKit, receive ConfigKit's difference records without an additional
local interpretation layer, and find accurate documentation for the operation
and its boundaries. A separate staged implementation plan can define the
implementation sequence.