Plan structured effective configuration diffs

docs/roadmap/diff.md

@@ -0,0 +1,259 @@
+# Effective Configuration Diff Roadmap
+
+Status: proposed; implementation has not started. Execute the stages below in
+order. This document owns the targeted end state, implementation sequence, and
+remaining decisions; it does not describe an available API.
+
+## Targeted end state
+
+ConfigKit compares a left, fully resolved effective configuration with a right,
+fully resolved effective configuration and returns structured differences.
+Consumers decide how to render those differences: a changes table, a tree, or
+a conventional diff are possible presentations. ConfigKit owns comparison and
+safe value disclosure, while applications own presentation and labels.
+
+Both configurations must have completed the normal loading lifecycle, including
+typed decoding, resolution, secret lookup, and validation. Invalid input on
+either side produces the existing load failure and diagnostics, with no usable
+diff. Comparing successful results must not rerun callbacks, reopen sources, or
+resolve credentials again. Later mutation of caller-owned `Result.Config` must
+not change the frozen values being compared.
+
+Each difference identifies a semantic path and the left and right values at that
+path. Absence must be distinguishable from explicit null. Additions, removals,
+and replacements must be identifiable without parsing presentation markers.
+Equal effective values produce no differences, even if their source locations,
+explicit-input status, or loading histories differ. Original comments,
+formatting, and mapping declaration order do not participate in equality.
+
+Secret handling must preserve detectable changes without exposing classified
+values. Compare the underlying retained configuration values before redaction;
+apply sensitivity from both sides before returning either side's value.
+A consumer must be able to indicate a changed sensitive value, for example as
+`[REDACTED]` on the left and `[CHANGED]` on the right. These strings are consumer
+presentation choices, not configuration values or the encoding of change state.
+The decisions below address nested redaction and the boundary around resolved
+credential bytes.
+
+The motivating workflows are:
+
+- Load `app.conf.bak` and `app.conf`, then inspect effective differences.
+- Load a base configuration and separately load it with a selected overlay,
+  then inspect the overlay's effective impact, including subsequent resolution.
+
+An overlay alone is not the right side of the second workflow: omitted overlay
+keys retain base values. Both workflows require successful, independent loads;
+applications control their definitions, options, and external inputs. Using the
+same definition and controlled inputs makes comparisons easier to interpret,
+but ConfigKit must not invent an identity check for callback functions.
+
+This feature does not include input-text comparison, invalid-document inspection,
+patch application, generating a reconstructing overlay, fingerprints, automatic
+reload, or a built-in renderer. A diff may describe removals that existing overlay
+semantics cannot express. Unchanged context for a unified renderer is a separate
+data-shape decision below.
+
+## Current baseline
+
+- [Deferred scope](deferred.md) currently excludes configuration diffs and
+  fingerprints. This roadmap proposes reconsidering only the diff boundary;
+  it does not silently change that document or approve fingerprints.
+- The [configuration contract](../config.md#typed-projection-and-ownership)
+  describes canonical typed projection and private structural comparison.
+  [The existing comparator](../../internal/engine/document_compare.go) reports
+  mapping additions, removals, and replacements, ignores mapping order, treats
+  sequences atomically, and sorts changes by pointer spelling. It ignores
+  provenance and sensitivity metadata.
+- Results retain private frozen inspection state. Existing
+  [inspection types](../../inspection.go) include `ValueSnapshot`, with
+  `Present`, `Redacted`, and deterministic `YAML`. Absence is distinct from null.
+  Existing explanation records describe one load's history, not a comparison
+  between two loads.
+- [Secret classification](../config.md#secret-classification-and-effective-rendering)
+  includes historical classification and dependency propagation. Existing
+  snapshots apply one load's sensitivity; a diff needs protection from both
+  loads, including removed paths and structural replacements.
+- [Typed secret references](../config.md#typed-secret-references) retain reference
+  names in inspection, not resolved credential bytes. The existing snapshot
+  comparator cannot detect rotation behind an unchanged reference.
+- The [public facade](../internal/overview.md) is root-owned and delegates to
+  the engine. Public diff types must follow that boundary and expose no parser
+  or internal document types.
+
+## Ordered implementation plan
+
+### 1. Settle the public contract and scope decision
+
+Resolve the Open Decisions below before implementing dependent behavior. Record
+the agreed API shape, equality and granularity rules, sensitivity behavior,
+credential boundary, and error behavior. Distinguish unavailable/nil results
+from valid configurations with no differences; recommend an error for the former.
+
+Review the extension against the architecture and release policies. Record the
+significant boundary change in an ADR and reconcile the diff exclusion in
+`deferred.md`, retaining unrelated exclusions. Keep planned behavior in the
+roadmap until it lands.
+
+Use a simple two-file consumer and a layered consumer with resolution and
+sensitive nested values to check that the proposed data is sufficient. Agree
+which rendering needs those examples demonstrate; do not add a renderer merely
+to exercise the API.
+
+Completion: a reviewable contract with no unresolved decisions that block the
+comparison or disclosure model.
+
+### 2. Implement comparison of frozen inspection state
+
+Add an engine operation over two successful frozen inspections. Reuse the
+existing comparator where its semantics match the selected contract. Do not
+change resolution or validation auditing semantics incidentally when sharing
+helpers. Compare canonical effective values, not serialized redacted output,
+caller-mutated configuration values, or historical replay.
+
+Define deterministic record ordering and parent/subtree reporting. Retain
+semantic key/index intent and proper path escaping. Construct only the value
+snapshots and additional metadata needed by the chosen result shape; avoid
+replaying every path's full explanation history.
+
+Completion: ordinary additions, removals, replacements, and equality work with
+the selected granularity and unavailable-result behavior.
+
+### 3. Apply cross-configuration sensitivity
+
+Combine the applicable sensitivity of the two inspections for disclosure on
+both sides. Account for classified ancestors and descendants, historical
+classification, dependency-derived sensitivity, missing paths, and container
+kind changes. Do not assume equal pointer text implies equal key/index shape.
+
+Detect changes before masking and preserve their existence even when safe YAML
+is identical. Keep redaction state separate from literal marker-looking input.
+For containers with masked descendants, implement the selected hidden-change
+representation without falsely claiming that every redacted child changed.
+Errors and returned records must not disclose classified values or credentials.
+
+Completion: changes remain identifiable under redaction, either side's
+classification protects both values, and returned data meets the agreed secret
+boundary.
+
+### 4. Expose the root API and demonstrate consumer use
+
+Add root-owned records and the selected comparison entry point, with explicit
+conversion across the engine boundary. Return detached paths and records;
+consumer mutation must not affect later comparisons or existing inspection.
+Document symbols beside declarations, including direction, equality, failures,
+snapshot lifetime, secret behavior, and the credential limitation if selected.
+
+Add executable examples for loading both files and handling each load failure,
+and for comparing a base result with a base-plus-overlay result. Demonstrate
+consumer-owned formatting and changed-secret indicators using disposable data.
+No usable diff should be presented when either load fails.
+
+Completion: consumers can implement the motivating workflows through the root
+package without internal imports or duplicating structural comparison.
+
+### 5. Validate behavior and update owning documentation
+
+Use engine tests for structural comparison and combined sensitivity, root tests
+for the public facade and ownership, and consumer examples for complete workflows.
+Cover meaningful risks without duplicating all existing loading tests:
+
+- equality despite mapping order or provenance changes; canonical value equality
+  after defaults and resolution;
+- addition/removal versus explicit null, empty containers, scalar and kind
+  replacements, sequence changes, and escaped or numeric mapping keys;
+- deterministic ordering and the agreed subtree granularity;
+- changed and unchanged sensitive values, classification on only one side,
+  removed secrets, classified containers, and mixed public/sensitive changes;
+- a changed hidden descendant whose rendered left/right values are identical,
+  and literal strings resembling redaction markers;
+- the selected reference/credential behavior with controlled environment state;
+- invalid left/right loads, unavailable results, frozen snapshot behavior, and
+  returned-record mutation isolation.
+
+Use disposable canaries to check non-disclosure, including failure paths.
+Run focused tests while implementing, then the everyday check and ownership
+race validation from the [development guide](../development.md). Validate local
+documentation links. No independent performance benchmarks are part of this
+roadmap; the existing request to avoid them remains in force.
+
+Update the configuration contract, consumer guidance, and component overview
+when their corresponding behavior lands. Keep symbol details in Go declaration
+documentation. Review compatibility and release notes under the release policy;
+publication is separate from feature completion.
+
+Completion: checks pass, consumer examples demonstrate the agreed data contract,
+and implemented documentation agrees with the public API. Mark the roadmap
+complete only after these outcomes are achieved.
+
+## Open Decisions
+
+### Comparison entry point and type compatibility
+
+Recommendation: start with a function over two successful results of the same
+configuration type, illustratively
+`Diff[T any](left, right *Result[T]) ([]Difference, error)`. This fits both
+motivating workflows, reuses normal load failures, and avoids a second loading
+lifecycle. Return an error for nil or uninitialized results. The signature and
+names remain proposals; sharing `T` does not prove equal defaults or callbacks.
+
+Alternative: compare results with different configuration types through a
+root-owned inspection handle or another deliberately designed API. This supports
+schema migrations and comparisons across applications, but introduces another
+public abstraction or more type parameters and requires explaining how differing
+schemas affect equality. Choose it if a concrete consumer needs that flexibility;
+same-type comparison is a recommended initial surface, not an intrinsic
+correctness restriction. A combined load-and-diff convenience operation is also
+viable, but adds failure aggregation and callback orchestration without being
+necessary for the requested workflows.
+
+### Value representation and renderer context
+
+Recommendation: use detached `Difference` records with `Path`, `Left`, and
+`Right`, reusing `ValueSnapshot` for each side. Presence identifies additions and
+removals, while record membership establishes a change. This reuses existing safe
+inspection semantics without exporting a second dynamic value model. Initially
+return changed regions only; do not promise that these records alone support
+unified hunks with unchanged context.
+
+Alternative: return a richer diff result with safe paired full-document context
+or a structured safe value tree. That better supports context-aware and non-YAML
+renderers, but increases the public representation and disclosure surface.
+Independently calling `WriteEffective` on each side is not automatically a safe
+substitute for paired context: it does not apply the other side's sensitivity.
+If conventional context-rich rendering is required immediately, include paired
+context in the contract rather than leaving consumers to reconstruct it unsafely.
+
+### Granularity and changes inside redacted containers
+
+Recommendation: retain recursive mapping comparison and atomic sequence/subtree
+replacement as the initial reporting model. It matches existing comparison and
+does not invent correspondence between sequence elements. For a wholly masked
+changed value, the difference record is sufficient for a consumer's `[CHANGED]`
+indicator. For mixed containers, add explicit metadata identifying whether the
+replacement includes hidden value changes; `ValueSnapshot.Redacted` alone only
+means that some content is masked. Select the exact metadata shape in stage 1.
+
+Alternative: expose finer-grained nested changes, including a defined sequence
+alignment, so renderers can identify individual changed hidden descendants.
+This gives more precise displays but needs rules for insertions, reordering,
+duplicates, and cross-kind replacements. Index-by-index comparison is simpler
+but can misleadingly present shifted elements as replacements. A conservative
+whole-container display is viable with atomic records, provided it labels the
+container as changed rather than asserting which masked children changed.
+
+### Resolved credentials versus retained configuration values
+
+Recommendation: compare classified values and secret reference names retained
+in the effective snapshots, and explicitly exclude detection of resolved
+credential rotation behind an unchanged reference. This preserves the existing
+inspection boundary and avoids retaining credential-derived comparison material.
+Confirm that this limitation satisfies the intended meaning of changed secrets
+before implementation.
+
+Alternative: support comparison of resolved credentials as well. It would meet
+the broader expectation that any effective credential change is detectable, but
+requires a deliberate trusted comparison mechanism beyond existing inspection.
+Reading mutable `Result.Config` would violate frozen-snapshot semantics; retaining
+credentials or comparison tokens introduces confidentiality and lifetime choices.
+Do not add raw hashes or credential retention as an incidental implementation
+detail. Resolve that design and its security consequences first if required.