diff --git a/docs/roadmap/diff.md b/docs/roadmap/diff.md new file mode 100644 index 0000000..e8f48a2 --- /dev/null +++ b/docs/roadmap/diff.md @@ -0,0 +1,142 @@ +# Captured Configuration Comparison + +Status: accepted; implementation pending. + +## Purpose + +Let consumers compare two captured configuration inspections for operator-facing +workflows: reviewing a candidate configuration, explaining the effect of an +overlay, or reporting changes between independently constructed engines. +PromptKit should expose a thin API over ConfigKit's comparison functionality, +leaving comparison and sensitivity handling upstream. + +This feature extends inspection, not prompt execution. It does not require +PromptKit to retain previous engines, manage reloads, or decide whether a +candidate configuration should be activated. + +## Current Baseline + +PromptKit currently depends on ConfigKit v0.6.0. `Engine.InspectConfiguration` +returns a captured `ConfigurationInspection` with metadata, source descriptions, +diagnostics, path explanations, and effective-configuration output, but no +comparison operation. These methods already expose ConfigKit's public inspection +types while keeping the typed configuration result private. + +ConfigKit v0.7.0 adds effective-configuration comparison without changing the +loading contracts used by PromptKit. The reviewed release corresponds to commit +`8f3396d6f0cbe03af4f92bb2c1a1c9798171dd52`. See its +[release notes](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.7.0/docs/releases/v0.7.0.md) +and [public comparison API](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.7.0/diff.go). + +## Target Public API + +Add a method to the existing inspection value: + +```go +func (v ConfigurationInspection) Diff(other ConfigurationInspection) ([]configkit.Difference, error) +``` + +The receiver supplies the left (earlier) configuration and `other` supplies the +right (later) configuration. Consumers can compare views from separate engines +or compare a view with itself. Returning ConfigKit's public `Difference` type is +consistent with the existing inspection API and avoids a parallel PromptKit +representation of the same information. + +Comparing either operand's zero inspection value returns `ErrInvalidConfig` and +no difference records, consistent with existing inspection operations that +require a captured view. Errors from the underlying comparison remain available +to the caller; a new comparison-specific error hierarchy is unnecessary. + +## Comparison Meaning And Boundaries + +Delegate to ConfigKit's frozen effective-value comparison. Equal effective +values produce no records even when their source or resolution histories differ. +ConfigKit owns comparison granularity, canonical path ordering, and detached +value snapshots. Its public contract distinguishes absent values from explicit +nulls, compares mappings recursively, and treats sequences as atomic values. + +Sensitivity classifications from both views protect both sides of the output. +Pass through ConfigKit's `Redacted` and `HiddenChanged` information rather than +implementing masking or hidden-change detection in PromptKit. Existing PromptKit +sensitivity classifications remain applicable. Ordinary prompt and provider +content is not automatically safe to disclose; consumers still choose where and +to whom to present reports. See the upstream +[comparison guidance](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.7.0/docs/config.md#effective-configuration-diffs) +for the detailed semantics. + +The comparison operates on the configuration captured at the ConfigKit handoff, +consistent with [ADR 0001](../adr/0001-configuration-boundary.md). It does not +reread sources, resolve credentials, render prompts, or call providers. It is an +explicit inspection operation, not an additional step during ordinary prompt +execution. + +An empty diff means equality of the compared configuration values, not equality +of engine behavior. For example, a referenced template file can change while its +configured path stays the same, and secret contents can change while their +configured reference stays the same. Post-load execution preparation and +request-specific overrides are not part of this comparison. + +## Work Needed + +### Dependency Adoption + +- Upgrade the ConfigKit requirement to v0.7.0 and update module checksums. +- Verify PromptKit's existing loading, inspection, and execution behavior against + the new dependency through the relevant tests and maintained examples. +- Refresh ConfigKit's version and exact upstream source references in the + [attribution document](../oss/attribution.md), following the + [OSS policy](../policy/oss.md). The reviewed release retains its existing + licensing arrangements. + +### Thin Inspection Integration + +- Provide a private comparison bridge for the retained ConfigKit results in + `internal/configuration`, preserving the private configuration type and the + existing inspection operations. +- Add the public method in `configuration_inspection.go`, delegating through + that bridge. Keep generic results and internal configuration structures out of + the public API. +- Reuse captured inspection state and upstream comparison logic. No changes to + configuration formats, source precedence, or execution routing are needed. + +### Focused Verification + +- Exercise the public method with independently constructed engines, an equal + pair, and a representative changed value. Include equal effective values + produced by different source histories. +- Verify operand direction, zero-view errors, and propagation of upstream + records through the private bridge. +- Verify paired sensitivity protection and hidden-change reporting using + PromptKit's existing credential-shaped provider fields. Check representative + presence and sequence behavior without duplicating ConfigKit's comparator + test suite. +- Check that comparison uses captured state after source changes and that + returned records do not let callers mutate retained inspection state. +- Exercise concurrent inspection comparisons under the race detector. Keep + tests offline and focused on PromptKit's integration boundary, following the + [testing policy](../policy/testing.md). + +### Documentation + +- Add GoDoc for the method's direction, errors, snapshot boundary, and upstream + result semantics. +- Add a short operator-oriented usage example to the + [consumer guide](../consumers/pkg-promptkit.md), linking to GoDoc and ConfigKit + for the detailed contract. +- Update the [internal source document](../internal/sources.md) to describe the + private comparison bridge and refresh dependency-specific discussion where + needed. Record the feature in the applicable release notes when preparing + its release. + +## Scope And Completion + +This unit of work is the dependency upgrade and a thin inspection comparison +method, with proportionate tests and documentation. Custom diff algorithms, +formatters, CLIs, patch application, automatic reloads, and execution-equivalence +analysis are outside this unit of work. + +The feature is complete when consumers can compare captured inspection views +through PromptKit, receive ConfigKit's difference records without an additional +local interpretation layer, and find accurate documentation for the operation +and its boundaries. A separate staged implementation plan can define the +implementation sequence.