Retire completed configuration comparison roadmaps
This commit is contained in:
@@ -1,144 +0,0 @@
|
||||
# Captured Configuration Comparison
|
||||
|
||||
Status: Implemented (2026-10-09). PromptKit exposes captured configuration
|
||||
comparison through `ConfigurationInspection.Diff`; see the
|
||||
[implementation plan](implementation.md) for the completed work and verification.
|
||||
|
||||
## Purpose
|
||||
|
||||
Let consumers compare two captured configuration inspections for operator-facing
|
||||
workflows: reviewing a candidate configuration, explaining the effect of an
|
||||
overlay, or reporting changes between independently constructed engines.
|
||||
PromptKit should expose a thin API over ConfigKit's comparison functionality,
|
||||
leaving comparison and sensitivity handling upstream.
|
||||
|
||||
This feature extends inspection, not prompt execution. It does not require
|
||||
PromptKit to retain previous engines, manage reloads, or decide whether a
|
||||
candidate configuration should be activated.
|
||||
|
||||
## Current Baseline
|
||||
|
||||
PromptKit currently depends on ConfigKit v0.6.0. `Engine.InspectConfiguration`
|
||||
returns a captured `ConfigurationInspection` with metadata, source descriptions,
|
||||
diagnostics, path explanations, and effective-configuration output, but no
|
||||
comparison operation. These methods already expose ConfigKit's public inspection
|
||||
types while keeping the typed configuration result private.
|
||||
|
||||
ConfigKit v0.7.0 adds effective-configuration comparison without changing the
|
||||
loading contracts used by PromptKit. The reviewed release corresponds to commit
|
||||
`8f3396d6f0cbe03af4f92bb2c1a1c9798171dd52`. See its
|
||||
[release notes](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.7.0/docs/releases/v0.7.0.md)
|
||||
and [public comparison API](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.7.0/diff.go).
|
||||
|
||||
## Target Public API
|
||||
|
||||
Add a method to the existing inspection value:
|
||||
|
||||
```go
|
||||
func (v ConfigurationInspection) Diff(other ConfigurationInspection) ([]configkit.Difference, error)
|
||||
```
|
||||
|
||||
The receiver supplies the left (earlier) configuration and `other` supplies the
|
||||
right (later) configuration. Consumers can compare views from separate engines
|
||||
or compare a view with itself. Returning ConfigKit's public `Difference` type is
|
||||
consistent with the existing inspection API and avoids a parallel PromptKit
|
||||
representation of the same information.
|
||||
|
||||
Comparing either operand's zero inspection value returns `ErrInvalidConfig` and
|
||||
no difference records, consistent with existing inspection operations that
|
||||
require a captured view. Errors from the underlying comparison remain available
|
||||
to the caller; a new comparison-specific error hierarchy is unnecessary.
|
||||
|
||||
## Comparison Meaning And Boundaries
|
||||
|
||||
Delegate to ConfigKit's frozen effective-value comparison. Equal effective
|
||||
values produce no records even when their source or resolution histories differ.
|
||||
ConfigKit owns comparison granularity, canonical path ordering, and detached
|
||||
value snapshots. Its public contract distinguishes absent values from explicit
|
||||
nulls, compares mappings recursively, and treats sequences as atomic values.
|
||||
|
||||
Sensitivity classifications from both views protect both sides of the output.
|
||||
Pass through ConfigKit's `Redacted` and `HiddenChanged` information rather than
|
||||
implementing masking or hidden-change detection in PromptKit. Existing PromptKit
|
||||
sensitivity classifications remain applicable. Ordinary prompt and provider
|
||||
content is not automatically safe to disclose; consumers still choose where and
|
||||
to whom to present reports. See the upstream
|
||||
[comparison guidance](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.7.0/docs/config.md#effective-configuration-diffs)
|
||||
for the detailed semantics.
|
||||
|
||||
The comparison operates on the configuration captured at the ConfigKit handoff,
|
||||
consistent with [ADR 0001](../adr/0001-configuration-boundary.md). It does not
|
||||
reread sources, resolve credentials, render prompts, or call providers. It is an
|
||||
explicit inspection operation, not an additional step during ordinary prompt
|
||||
execution.
|
||||
|
||||
An empty diff means equality of the compared configuration values, not equality
|
||||
of engine behavior. For example, a referenced template file can change while its
|
||||
configured path stays the same, and secret contents can change while their
|
||||
configured reference stays the same. Post-load execution preparation and
|
||||
request-specific overrides are not part of this comparison.
|
||||
|
||||
## Work Needed
|
||||
|
||||
### Dependency Adoption
|
||||
|
||||
- Upgrade the ConfigKit requirement to v0.7.0 and update module checksums.
|
||||
- Verify PromptKit's existing loading, inspection, and execution behavior against
|
||||
the new dependency through the relevant tests and maintained examples.
|
||||
- Refresh ConfigKit's version and exact upstream source references in the
|
||||
[attribution document](../oss/attribution.md), following the
|
||||
[OSS policy](../policy/oss.md). The reviewed release retains its existing
|
||||
licensing arrangements.
|
||||
|
||||
### Thin Inspection Integration
|
||||
|
||||
- Provide a private comparison bridge for the retained ConfigKit results in
|
||||
`internal/configuration`, preserving the private configuration type and the
|
||||
existing inspection operations.
|
||||
- Add the public method in `configuration_inspection.go`, delegating through
|
||||
that bridge. Keep generic results and internal configuration structures out of
|
||||
the public API.
|
||||
- Reuse captured inspection state and upstream comparison logic. No changes to
|
||||
configuration formats, source precedence, or execution routing are needed.
|
||||
|
||||
### Focused Verification
|
||||
|
||||
- Exercise the public method with independently constructed engines, an equal
|
||||
pair, and a representative changed value. Include equal effective values
|
||||
produced by different source histories.
|
||||
- Verify operand direction, zero-view errors, and propagation of upstream
|
||||
records through the private bridge.
|
||||
- Verify paired sensitivity protection and hidden-change reporting using
|
||||
PromptKit's existing credential-shaped provider fields. Check representative
|
||||
presence and sequence behavior without duplicating ConfigKit's comparator
|
||||
test suite.
|
||||
- Check that comparison uses captured state after source changes and that
|
||||
returned records do not let callers mutate retained inspection state.
|
||||
- Exercise concurrent inspection comparisons under the race detector. Keep
|
||||
tests offline and focused on PromptKit's integration boundary, following the
|
||||
[testing policy](../policy/testing.md).
|
||||
|
||||
### Documentation
|
||||
|
||||
- Add GoDoc for the method's direction, errors, snapshot boundary, and upstream
|
||||
result semantics.
|
||||
- Add a short operator-oriented usage example to the
|
||||
[consumer guide](../consumers/pkg-promptkit.md), linking to GoDoc and ConfigKit
|
||||
for the detailed contract.
|
||||
- Update the [internal source document](../internal/sources.md) to describe the
|
||||
private comparison bridge and refresh dependency-specific discussion where
|
||||
needed. Record the feature in the applicable release notes when preparing
|
||||
its release.
|
||||
|
||||
## Scope And Completion
|
||||
|
||||
This unit of work is the dependency upgrade and a thin inspection comparison
|
||||
method, with proportionate tests and documentation. Custom diff algorithms,
|
||||
formatters, CLIs, patch application, automatic reloads, and execution-equivalence
|
||||
analysis are outside this unit of work.
|
||||
|
||||
The feature is complete when consumers can compare captured inspection views
|
||||
through PromptKit, receive ConfigKit's difference records without an additional
|
||||
local interpretation layer, and find accurate documentation for the operation
|
||||
and its boundaries. The companion [implementation plan](implementation.md)
|
||||
defines the staged implementation sequence.
|
||||
@@ -1,236 +0,0 @@
|
||||
# Captured Configuration Comparison Implementation Plan
|
||||
|
||||
Implement [the feature roadmap](diff.md) in the numbered order below. Each stage
|
||||
is one bounded implementation prompt for a Luna coding agent. The feature
|
||||
roadmap owns the purpose and target contract; this document owns the steps.
|
||||
|
||||
## Working Instructions
|
||||
|
||||
- Start with [the development guide](../development.md) and follow the
|
||||
[agent policy](../policy/agents.md). Read the architecture, testing,
|
||||
documentation, and OSS policies for the stages they govern.
|
||||
- Preserve [ADR 0001](../adr/0001-configuration-boundary.md): compare captured
|
||||
ConfigKit effective values for explicit inspection. Leave domain execution,
|
||||
configuration validation, and source processing unchanged.
|
||||
- Use the released ConfigKit dependency, not a committed local replacement or
|
||||
workspace file. The sibling checkout is a reference, not a build prerequisite.
|
||||
- Keep the upstream comparator and existing sensitivity classifier authoritative.
|
||||
Do not add local diff algorithms, masking rules, output limits, reloads,
|
||||
formatters, or execution-equivalence checks in this work.
|
||||
- After each stage, review the complete relevant diff, run its checks, and commit
|
||||
the completed unit with a plain-English message. Record actual validation
|
||||
results in the handoff; do not bypass hooks or publish a release.
|
||||
|
||||
## Current Baseline And Selected Decisions
|
||||
|
||||
At planning time, PromptKit uses ConfigKit v0.6.0. The public inspection methods
|
||||
are in `configuration_inspection.go`; `internal/configuration/inspection.go`
|
||||
defines their narrow `Inspection` interface and currently returns the retained
|
||||
`*configkit.Result[document]` directly. The internal engine stores that interface.
|
||||
Root inspection contract tests live in
|
||||
`configuration_inspection_contract_test.go`.
|
||||
|
||||
Use ConfigKit v0.7.0, whose reviewed commit is
|
||||
`8f3396d6f0cbe03af4f92bb2c1a1c9798171dd52`.
|
||||
The public addition is exactly:
|
||||
|
||||
```go
|
||||
func (v ConfigurationInspection) Diff(other ConfigurationInspection) ([]configkit.Difference, error)
|
||||
```
|
||||
|
||||
The receiver is left/earlier and the argument is right/later. Either zero view
|
||||
returns `ErrInvalidConfig` and no records. Successful calls forward ConfigKit's
|
||||
records; errors from ConfigKit remain inspectable through ordinary Go error
|
||||
handling. Do not promise a particular nil-versus-empty slice on success or exact
|
||||
error wording. No context parameter or PromptKit-specific difference type is
|
||||
needed.
|
||||
|
||||
For the private bridge, retain the existing `Inspection` interface and add an
|
||||
unexported accessor for the captured `*configkit.Result[document]`. Implement the
|
||||
interface with a private wrapper around that result. Existing inspection methods
|
||||
can be promoted from an embedded result; the accessor supplies the result to an
|
||||
internal-package comparison function. This keeps the implementation small without
|
||||
runtime type assertions or exposing the configuration type through the facade.
|
||||
This wrapper is an implementation choice, not a new consumer contract.
|
||||
|
||||
## Stage 1: Adopt ConfigKit v0.7.0
|
||||
|
||||
**Status: Complete.** The selected tag resolves to reviewed commit
|
||||
`8f3396d6f0cbe03af4f92bb2c1a1c9798171dd52`; upstream `COPYRIGHT` and `LICENSE`
|
||||
are unchanged from v0.6.0. `GOTOOLCHAIN=local go test ./...`,
|
||||
`GOTOOLCHAIN=local go vet ./...`,
|
||||
`GOTOOLCHAIN=local python3 scripts/check-examples.py`, and
|
||||
`git diff --check` passed.
|
||||
|
||||
Scope: dependency and attribution changes only.
|
||||
|
||||
1. Read the upstream v0.7.0 release notes and `diff.go`, and verify the selected
|
||||
tag's commit and licensing notices against the baseline above.
|
||||
2. Update the dependency with
|
||||
`go get gitea.maximumdirect.net/eric/ConfigKit@v0.7.0`, then `go mod tidy`.
|
||||
Review `go.mod` and `go.sum`; avoid unrelated upgrades, Go-version changes,
|
||||
or local replacements.
|
||||
3. Update the ConfigKit entry in `docs/oss/attribution.md`: selected version and
|
||||
all upstream source, COPYRIGHT, and LICENSE links must use the exact imported
|
||||
commit. Preserve the house format and actual upstream notices.
|
||||
4. Run `go test ./...`, `go vet ./...`, and
|
||||
`python3 scripts/check-examples.py`. Investigate regressions rather than
|
||||
weakening existing loading or execution contracts to accommodate them.
|
||||
5. Review the dependency/notice diff and run `git diff --check` before committing.
|
||||
|
||||
Completion: PromptKit uses the published v0.7.0 module, current behavior passes
|
||||
existing validation, and attribution matches the selected source revision.
|
||||
|
||||
## Stage 2: Add The Private Bridge And Public Method
|
||||
|
||||
**Status: Complete.** Added the private captured-result bridge and public
|
||||
`ConfigurationInspection.Diff` method. External-package tests cover changed and
|
||||
equal values, operand direction, self-comparison, and all zero-view pairs.
|
||||
`GOTOOLCHAIN=local go test . ./internal/configuration ./internal/engine` and
|
||||
`git diff --check` passed.
|
||||
|
||||
Scope: the small integration path and basic facade tests.
|
||||
|
||||
1. In `internal/configuration/inspection.go`, add the private wrapper and
|
||||
captured-result accessor described above. Update `loaded.Inspection` to
|
||||
return the wrapper for a captured result, keeping absent state represented by
|
||||
a nil inspection. Preserve all existing inspection operations.
|
||||
2. Add an internal-package `Diff(left, right Inspection)` function returning
|
||||
`([]configkit.Difference, error)`. Obtain the captured results through the
|
||||
accessor and call `configkit.Diff[document]` once. If an internal operand is
|
||||
absent, pass a nil result to ConfigKit so its existing error behavior applies.
|
||||
Do not reload, reconstruct snapshots, or independently compare values.
|
||||
3. Add `ConfigurationInspection.Diff` to the root facade. Check both source
|
||||
interfaces for nil first and return `nil, ErrInvalidConfig` for a zero view;
|
||||
otherwise delegate to the bridge. Return upstream errors directly unless
|
||||
useful context is needed, in which case wrap with `%w`.
|
||||
4. Add GoDoc for direction, zero-view errors, frozen effective-value scope, and
|
||||
ConfigKit-owned difference semantics. Note that reports can include ordinary
|
||||
application data and are not evidence of equivalent execution behavior.
|
||||
5. Add external-package root tests using independent engines with synthetic
|
||||
mapping inputs and an explicitly defined local backend/profile. Verify a
|
||||
model-value change at a known path, reversed operand direction, equal values,
|
||||
self-comparison, and the three zero-view combinations. Use `errors.Is` for
|
||||
`ErrInvalidConfig` and `len(records) == 0` for successful equality.
|
||||
6. Run `gofmt` on changed Go files and
|
||||
`go test . ./internal/configuration ./internal/engine`, then
|
||||
`git diff --check`. Existing inspection tests must continue to pass.
|
||||
|
||||
Completion: the public signature works from package `promptkit_test`, the private
|
||||
result remains inaccessible through the facade, and unchanged inspection methods
|
||||
still behave as before. No runner or domain-package integration is needed.
|
||||
|
||||
## Stage 3: Verify Result Semantics And Sensitive Changes
|
||||
|
||||
**Status: Complete.** Facade tests cover distinct source labels with equal
|
||||
effective values, absent versus null, atomic sequence changes, paired secret
|
||||
masking, sensitive shape replacement, and mutation of returned records.
|
||||
`GOTOOLCHAIN=local go test . ./internal/configuration` and
|
||||
`git diff --check` passed.
|
||||
|
||||
Scope: focused facade integration tests, not a second comparator test suite.
|
||||
|
||||
1. Extend the comparison tests with two valid engine configurations whose
|
||||
effective values are equal but whose underlays/source labels differ. Verify
|
||||
an empty diff; history differences must not become value differences.
|
||||
2. Use valid JSON-compatible values in a synthetic provider `extra_params` map
|
||||
to exercise absence versus explicit null, and a changed sequence. Verify the
|
||||
forwarded `Present` flags and one atomic sequence-region record. Compare
|
||||
semantic values rather than incidental YAML whitespace.
|
||||
3. Add a focused sensitivity case using an existing credential-shaped key such
|
||||
as `backends.local.extra_params.token`. Change its synthetic secret value and
|
||||
a public sibling. Verify that neither operand's secret appears in either
|
||||
snapshot, that redaction information and `HiddenChanged` survive the bridge,
|
||||
and that the public change remains visible.
|
||||
4. Include one valid shape-change case: an `extra_params.details` mapping with
|
||||
a nested `token` on one side and a scalar at `details` on the other. This
|
||||
places the sensitive descendant on only one side and exercises paired
|
||||
protection of the changed region. Use ConfigKit's documented masking and
|
||||
record granularity; do not require public descendants to remain visible in
|
||||
an incompatible shape change or alter `secretProviderPath`.
|
||||
5. Mutate a returned record's snapshot data and any mutable path representation
|
||||
offered by the upstream public API, then compare again. Verify retained
|
||||
state is unaffected without asserting upstream private allocation details.
|
||||
6. Run `go test . ./internal/configuration` and `git diff --check`; format any
|
||||
changed Go files. Reuse existing fixture helpers where they fit naturally.
|
||||
|
||||
Completion: tests protect PromptKit's forwarding, operand pairing, sensitive-data
|
||||
handling, and result ownership. Upstream retains ownership of exhaustive mapping,
|
||||
sequence, sensitivity, and ordering cases.
|
||||
|
||||
## Stage 4: Verify Frozen State And Concurrent Inspection
|
||||
|
||||
**Status: Complete.** The existing captured-source fixture now compares two
|
||||
views repeatedly after both sources become unreadable, and concurrently runs
|
||||
inspection, comparison, and offline execution against those captured engines.
|
||||
It confirms stable differences without source reads or comparison-triggered
|
||||
model calls. `GOTOOLCHAIN=local go test -race . ./internal/configuration ./internal/engine`
|
||||
and `git diff --check` passed.
|
||||
|
||||
Scope: lifecycle and concurrency integration tests.
|
||||
|
||||
1. Reuse or extend the existing captured-source inspection fixture and counting
|
||||
injected model client. Construct both engines while their sources are
|
||||
readable; afterward make those sources unreadable or change their contents.
|
||||
Obtain views and compare them repeatedly. Assert the captured differences
|
||||
remain unchanged and comparison causes no additional source reads or model
|
||||
calls. Set any environment values during setup, not concurrent operations.
|
||||
2. Exercise several concurrent comparisons on the same pair of views, together
|
||||
with an existing inspection operation. Validate returned semantic results
|
||||
and collect goroutine errors safely. Avoid sleeps, timing-based thresholds,
|
||||
or assertions about exact internal call sequences.
|
||||
3. Where useful, extend the existing concurrent inspection/execution fixture
|
||||
to include comparisons alongside offline runs. Reuse its coverage instead of
|
||||
building a duplicate lifecycle harness.
|
||||
4. Run `go test -race . ./internal/configuration ./internal/engine` and
|
||||
`git diff --check`; format changed Go files.
|
||||
|
||||
Completion: the comparison path demonstrably uses captured state and supports
|
||||
concurrent read-only use without involving prompt execution machinery.
|
||||
|
||||
## Stage 5: Document The Feature And Validate The Integrated Change
|
||||
|
||||
**Status: Complete.** Added the consumer example, comparison GoDoc, and internal
|
||||
bridge documentation; updated the feature roadmap to implemented. v0.13.0 is
|
||||
already published, so its note remains unchanged; include this feature in the
|
||||
next release note. `GOTOOLCHAIN=local go test -race ./...`,
|
||||
`GOTOOLCHAIN=local go vet ./...`,
|
||||
`GOTOOLCHAIN=local python3 scripts/check-examples.py`, affected local and
|
||||
ConfigKit v0.7.0 link-target checks, and `git diff --check` passed.
|
||||
|
||||
Scope: canonical documentation and final verification.
|
||||
|
||||
1. Add a small comparison example to the configuration-inspection section of
|
||||
`docs/consumers/pkg-promptkit.md`. Show two captured views, error handling,
|
||||
and iteration over returned records. Link to GoDoc and ConfigKit's v0.7.0
|
||||
comparison guidance for exact semantics rather than duplicating its API.
|
||||
2. Explain the private inspection wrapper and comparison bridge in
|
||||
`docs/internal/sources.md`. Refresh the existing ConfigKit v0.6.0-specific
|
||||
normalization discussion against v0.7.0 without changing message resource
|
||||
origins or execution normalization as part of this feature.
|
||||
3. Keep the example and GoDoc explicit about captured configuration equality:
|
||||
external file contents, resolved credential contents, post-load execution
|
||||
preparation, and per-request overrides are not execution-equivalence inputs.
|
||||
Retain the existing caller responsibility for report disclosure.
|
||||
4. If the pending v0.13.0 release document is still unpublished and is the chosen
|
||||
release for this work, add a short feature/dependency note with a canonical
|
||||
documentation link. Otherwise leave published notes unchanged and record the
|
||||
feature during the next release preparation. Do not tag, push, or run a
|
||||
publication command in this stage.
|
||||
5. Run `go test -race ./...`, `go vet ./...`,
|
||||
`python3 scripts/check-examples.py`, and `git diff --check`. Check affected
|
||||
local links and upstream links against the selected release's files. Review
|
||||
exported declarations to confirm only the intended method was added.
|
||||
6. Review the final diff against every work item in [the feature roadmap](diff.md),
|
||||
confirming comparison/masking are delegated upstream and runtime/domain
|
||||
components do not acquire ConfigKit inspection dependencies. Commit the
|
||||
completed documentation and any required integration corrections.
|
||||
|
||||
Completion: the dependency, API, tests, attribution, and canonical documentation
|
||||
are coherent and the full existing suite and maintained examples pass. Report
|
||||
stage commits and actual check results; release publication remains separate.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None. The accepted roadmap and the existing inspection error convention supply
|
||||
the decisions required for these stages.
|
||||
Reference in New Issue
Block a user