promptkit/docs/roadmap/diff.md

Captured Configuration Comparison

Status: Implemented (2026-10-09). PromptKit exposes captured configuration comparison through ConfigurationInspection.Diff; see the implementation plan for the completed work and verification.

Purpose

Let consumers compare two captured configuration inspections for operator-facing workflows: reviewing a candidate configuration, explaining the effect of an overlay, or reporting changes between independently constructed engines. PromptKit should expose a thin API over ConfigKit's comparison functionality, leaving comparison and sensitivity handling upstream.

This feature extends inspection, not prompt execution. It does not require PromptKit to retain previous engines, manage reloads, or decide whether a candidate configuration should be activated.

Current Baseline

PromptKit currently depends on ConfigKit v0.6.0. Engine.InspectConfiguration returns a captured ConfigurationInspection with metadata, source descriptions, diagnostics, path explanations, and effective-configuration output, but no comparison operation. These methods already expose ConfigKit's public inspection types while keeping the typed configuration result private.

ConfigKit v0.7.0 adds effective-configuration comparison without changing the loading contracts used by PromptKit. The reviewed release corresponds to commit 8f3396d6f0cbe03af4f92bb2c1a1c9798171dd52. See its release notes and public comparison API.

Target Public API

Add a method to the existing inspection value:

func (v ConfigurationInspection) Diff(other ConfigurationInspection) ([]configkit.Difference, error)

The receiver supplies the left (earlier) configuration and other supplies the right (later) configuration. Consumers can compare views from separate engines or compare a view with itself. Returning ConfigKit's public Difference type is consistent with the existing inspection API and avoids a parallel PromptKit representation of the same information.

Comparing either operand's zero inspection value returns ErrInvalidConfig and no difference records, consistent with existing inspection operations that require a captured view. Errors from the underlying comparison remain available to the caller; a new comparison-specific error hierarchy is unnecessary.

Comparison Meaning And Boundaries

Delegate to ConfigKit's frozen effective-value comparison. Equal effective values produce no records even when their source or resolution histories differ. ConfigKit owns comparison granularity, canonical path ordering, and detached value snapshots. Its public contract distinguishes absent values from explicit nulls, compares mappings recursively, and treats sequences as atomic values.

Sensitivity classifications from both views protect both sides of the output. Pass through ConfigKit's Redacted and HiddenChanged information rather than implementing masking or hidden-change detection in PromptKit. Existing PromptKit sensitivity classifications remain applicable. Ordinary prompt and provider content is not automatically safe to disclose; consumers still choose where and to whom to present reports. See the upstream comparison guidance for the detailed semantics.

The comparison operates on the configuration captured at the ConfigKit handoff, consistent with ADR 0001. It does not reread sources, resolve credentials, render prompts, or call providers. It is an explicit inspection operation, not an additional step during ordinary prompt execution.

An empty diff means equality of the compared configuration values, not equality of engine behavior. For example, a referenced template file can change while its configured path stays the same, and secret contents can change while their configured reference stays the same. Post-load execution preparation and request-specific overrides are not part of this comparison.

Work Needed

Dependency Adoption

• Upgrade the ConfigKit requirement to v0.7.0 and update module checksums.
• Verify PromptKit's existing loading, inspection, and execution behavior against the new dependency through the relevant tests and maintained examples.
• Refresh ConfigKit's version and exact upstream source references in the attribution document, following the OSS policy. The reviewed release retains its existing licensing arrangements.

Thin Inspection Integration

• Provide a private comparison bridge for the retained ConfigKit results in internal/configuration, preserving the private configuration type and the existing inspection operations.
• Add the public method in configuration_inspection.go, delegating through that bridge. Keep generic results and internal configuration structures out of the public API.
• Reuse captured inspection state and upstream comparison logic. No changes to configuration formats, source precedence, or execution routing are needed.

Focused Verification

• Exercise the public method with independently constructed engines, an equal pair, and a representative changed value. Include equal effective values produced by different source histories.
• Verify operand direction, zero-view errors, and propagation of upstream records through the private bridge.
• Verify paired sensitivity protection and hidden-change reporting using PromptKit's existing credential-shaped provider fields. Check representative presence and sequence behavior without duplicating ConfigKit's comparator test suite.
• Check that comparison uses captured state after source changes and that returned records do not let callers mutate retained inspection state.
• Exercise concurrent inspection comparisons under the race detector. Keep tests offline and focused on PromptKit's integration boundary, following the testing policy.

Documentation

• Add GoDoc for the method's direction, errors, snapshot boundary, and upstream result semantics.
• Add a short operator-oriented usage example to the consumer guide, linking to GoDoc and ConfigKit for the detailed contract.
• Update the internal source document to describe the private comparison bridge and refresh dependency-specific discussion where needed. Record the feature in the applicable release notes when preparing its release.

Scope And Completion

This unit of work is the dependency upgrade and a thin inspection comparison method, with proportionate tests and documentation. Custom diff algorithms, formatters, CLIs, patch application, automatic reloads, and execution-equivalence analysis are outside this unit of work.

The feature is complete when consumers can compare captured inspection views through PromptKit, receive ConfigKit's difference records without an additional local interpretation layer, and find accurate documentation for the operation and its boundaries. The companion implementation plan defines the staged implementation sequence.