Unify layered provenance history

docs/config.md

@@ -79,7 +79,7 @@ error without reporting one, ConfigKit emits a generic `version.hook_failed`
 diagnostic and suppresses the returned error text. Warnings alone are retained on a
 successful result. Callback panics remain programming failures.
 
-### Layered input/default metadata
+### Layered metadata
 
 `Result.Metadata` describes every final canonical path. Exact surviving input paths
 are explicit and have a base or profile origin; omitted paths have a default origin.
@@ -91,6 +91,23 @@ metadata while retaining their base definition and the parent replacement operat
 in history.
 All metadata queries and returned nested records are detached.
 
+History is chronological: a path can have a synthetic default record, its exact base
+definition, a profile replacement, and a recorded resolution operation. A descendant
+query uses the ancestor's path for a whole-container or whole-sequence replacement;
+the descendant can therefore retain history after `Lookup` stops reporting it as
+effective. Equal-valued input is still an explicit base definition, while a derived
+value for absent input remains non-explicit.
+
+A mapping assembled from the selected file and imports has one composite base origin
+whose locations retain source traversal order. A mapping also visited by a profile
+has a profile-layer composite origin with the lower contributing locations followed
+by the profile location; an empty mapping is still an explicit container even though
+it contributes no children. Resolution origins are synthetic: they have layer
+`resolution`, no fabricated file location, and a `derived_default` or `normalization`
+operation. Reasons and dependency paths retain first occurrence order with duplicates
+removed. Source records remain ordered as selected source, depth-first imports, then
+the selected profile.
+
 ### Recorded application resolution
 
 An optional `Definition.Resolve` callback runs after successful typed decoding. It

docs/internal/extraction.md

@@ -188,6 +188,18 @@ Assigned to layered metadata, redaction, rendering, and explanation:
   records, and redaction. Narratio commands, diff/digest APIs, generated artifact
   attribution, campaign logic, and party expansion remain application-owned.
 
+Layered-provenance implementation record (2026-09-06): ConfigKit integrated its
+previously extracted source claims, overlay displacement records, typed baseline,
+and audited resolution snapshots in `load_metadata.go`. The representative
+`TestLayeredProvenanceAgreesWithResolvedSnapshot` adapts Narratio's stable ownership,
+profile ownership, deterministic effective projection, and side-effect-free
+inspection cases across one default/import/profile/resolution workflow. ConfigKit
+adds chronological descendant history for parent and whole-sequence operations,
+composite empty-mapping origins, canonical resolution reasons/dependencies, private
+snapshot agreement, defensive nested copies, and concurrent load isolation. Value
+rendering, redaction, explanation, commands, generated attribution, diffs, and
+digests remain assigned to their later owning work.
+
 ### Defaults, presence, and domain resolution
 
 Assigned across schema, decoding, resolution, validation, and consumer examples:

docs/internal/overview.md

@@ -24,8 +24,8 @@ module and package documentation boundary. Its implemented public primitives are
   baseline/final trees, and initial default/input origins; and
 - public lifecycle coordination in `load.go`, bounded canonical import DFS in
   `imports.go`, pattern-directed glob expansion in `glob.go`, version-report contexts
-  in `validation.go`, and detached
+  in `validation.go`, and detached layered metadata/history assembly in
-  base/default metadata assembly in `load_metadata.go`; and
+  `load_metadata.go`; and
 - recorded application defaulting and normalization in `resolution.go`, including
   explicitness queries, helper verification, structural mutation audit, coalesced
   intent, private before/after snapshots, and resolution metadata integration; and
@@ -33,8 +33,8 @@ module and package documentation boundary. Its implemented public primitives are
   `validation.go`, completing the public callback lifecycle after composed input.
 
 The [configuration contract](../config.md) owns implemented input, schema, decoding,
-loading lifecycle, path imports, composition, profile overlay, version checking, recorded resolution,
+loading lifecycle, path imports, composition, profile overlay, version checking,
-semantic validation, metadata, and ownership behavior.
+recorded resolution, semantic validation, metadata, and ownership behavior.
 
 The source-extraction inventory records the application foundations assigned to
 future implementation work:

docs/roadmap/implementation.md

@@ -2,7 +2,7 @@
 
 ## Status and ownership
 
-Status: implementation in progress; Stages 1–13 are complete and later stages are
+Status: implementation in progress; Stages 1–14 are complete and later stages are
 pending.
 
 Audience: a `gpt-5.6-terra` coding agent implementing one stage per prompt, in
@@ -1038,7 +1038,31 @@ Acceptance:
 
 ## Stage 14 — Verify layered provenance and final snapshot agreement
 
-Status: pending.
+Status: complete.
+
+Completion evidence (2026-09-06):
+
+- Unified default, composed base, profile displacement, and resolution records in
+  the metadata history index. Container resolution now appends its operation to each
+  changed, added, or removed descendant while preserving the ancestor operation path
+  and path-specific previous origin; effective metadata still contains only the final
+  projected paths.
+- Added a representative public `Load` workflow spanning a fresh default, selected
+  source, depth-first import, profile, derived default, mapping deletion, and
+  whole-sequence normalization. It verifies source order, equal-to-default
+  explicitness, null state, composite empty-mapping origins, chronological parent
+  replacement history, deduplicated reasons/dependencies, typed/final snapshot
+  agreement, and unchanged siblings.
+- Verified nested metadata, source import locations, diagnostics, caller-owned
+  configuration, and private snapshots are detached. Repeated concurrent selected-
+  profile loads retain independent typed values, metadata, sources, and histories;
+  the race baseline passes.
+- Promoted layered metadata semantics and concrete composite/synthetic origin cases
+  to the [current metadata contract](../config.md#layered-metadata), updated Go
+  reference comments, and recorded the adapted Narratio ownership/inspection cases.
+  Rendering, redaction, value snapshots, and explanation remain within later stages.
+  Verified `go test ./...`, `go test -race ./...`, `go vet ./...`, `go build ./...`,
+  `go mod tidy -diff`, and `git diff --check` with Go 1.26.7 on linux/amd64.
 
 Read: feature [presence](initial.md#paths-and-presence),
 [history](initial.md#provenance-diagnostics-and-redaction), metadata API.

docs/roadmap/initial.md

@@ -430,23 +430,12 @@ conversion through the current loading lifecycle.
 
 ## Paths and presence
 
-Semantic path construction, JSON Pointer parsing, and base/default metadata
+Semantic path construction, JSON Pointer parsing, and layered metadata population
-population are implemented. Their current contracts are owned by the Go
+are implemented. Their current contracts are owned by the Go
 documentation in [`path.go`](../../path.go), [`metadata.go`](../../metadata.go),
-and the [current configuration contract](../config.md#base-default-metadata).
+and the [current metadata contract](../config.md#layered-metadata), including
-Layered replacement and resolution behavior below remains target lifecycle work.
+presence, composite origins, parent replacements, removed-path history, and
-
+synthetic resolution operations.
-Distinguish absent, explicitly null, explicitly zero/false/empty, and other
-explicit values. An explicit value equal to its default remains explicit.
-Presence queries describe the surviving input contribution after overlays;
-separate history queries describe earlier appearances. A parent mapping's
-presence does not make all of its children explicit. Replacing a subtree removes
-its descendants from effective presence and origin queries while retaining
-historical replacement records.
-
-Application normalization preserves the fact that a surviving input was
-explicit. A derived value for an absent input remains non-explicit. Its final
-origin is a resolution operation, with input origins retained separately.
 
 ## Composition and overlay
 
@@ -481,8 +470,7 @@ The [current application schema contract](../config.md#application-schema-bindin
 owns root/type support, field and tag binding, inlining, codec pairing, and
 schema-aware path rules. The [current defaults and typed-decoding contract](../config.md#defaults-and-typed-decoding)
 owns strict conversions, container application, presence, safe failures, and initial
-origins. Public lifecycle integration, layered provenance, secret handling, and
+origins. Secret handling and rendering remain target work below.
-rendering remain target work below.
 
 ## Recorded resolution and validation
 
@@ -490,7 +478,8 @@ Application-owned recorded resolution is implemented. The
 [current resolution contract](../config.md#recorded-application-resolution) owns
 explicitness queries, helpers and manual records, structural mutation auditing,
 coalescing, diagnostics, origins, history, and snapshot ownership. Layered input
-history, redacted explanation, and secret dependency propagation remain target work.
+history is owned by the [current metadata contract](../config.md#layered-metadata);
+redacted explanation and secret dependency propagation remain target work.
 
 Canonical typed projection, detached cloning, structural comparison, and publishing
 independently captured resolved results are implemented. The
@@ -507,16 +496,10 @@ callback lifetime, and application/runtime responsibility boundaries.
 Safe diagnostic/provenance value records, canonical diagnostic rendering, and
 `LoadError` access are implemented. Their current contracts are owned by
 [`records.go`](../../records.go) and [`diagnostic.go`](../../diagnostic.go).
-Base/default and resolution records are attached to the current loading lifecycle.
+Default, base, profile, and resolution records are attached to the loading lifecycle;
-Additional layers, safe historical values, redaction, and rendering remain target
+their source ordering, composite/synthetic origins, replacement history, and detached
-work below.
+queries are owned by the [current metadata contract](../config.md#layered-metadata).
-
+Safe historical values, redaction, and rendering remain target work below.
-Retain input sources, layer, line/column where available, import sites, effective
-origins, overridden definition records, and resolution operations. Container
-origins may be composite; never invent one file location for a mapping assembled
-from several sources. Defaults and resolution have synthetic origins rather
-than fabricated file coordinates. Preserve definition history without exposing
-mutable parser nodes or unrestricted raw value references through the API.
 
 Diagnostics have stable codes, severity, semantic path when available, a safe
 message, primary location, and related locations. Sort deterministically by
@@ -697,11 +680,11 @@ func ReadWorkerDefaults() (*configkit.Result[Config], error) {
 
 ### Paths and metadata
 
-The path API, detached metadata records, empty/unknown query behavior, and
+The path API, detached metadata records, empty/unknown query behavior, and layered
-base/default population are now implemented; [`path.go`](../../path.go),
+population are implemented; [`path.go`](../../path.go),
 [`records.go`](../../records.go), [`metadata.go`](../../metadata.go), and the
-[metadata configuration contract](../config.md#base-default-metadata) own current
+[metadata configuration contract](../config.md#layered-metadata) own current
-behavior. Layered callback history and safe historical values remain target behavior.
+behavior. Safe historical values remain target behavior for explanation.
 
 ### Resolution and validation callbacks
 

load_metadata.go

@@ -1,6 +1,10 @@
 package configkit
 
-import "gitea.maximumdirect.net/eric/ConfigKit/internal/document"
+import (
+	"slices"
+
+	"gitea.maximumdirect.net/eric/ConfigKit/internal/document"
+)
 
 func metadataFromDecoded(decoded decodedConfiguration) Metadata {
 	metadata := Metadata{
@@ -140,6 +144,7 @@ func applyResolutionMetadata(metadata *Metadata, resolved resolutionResult) {
 		}
 		metadata.history[pointer] = append(metadata.history[pointer], change)
 	}
+	appendResolutionDescendantHistory(metadata, resolved)
 	for _, delta := range resolved.deltas {
 		record := coveringResolutionSnapshot(resolved.snapshots, delta.path)
 		if record == nil {
@@ -157,6 +162,42 @@ func applyResolutionMetadata(metadata *Metadata, resolved resolutionResult) {
 	refreshResolutionFields(metadata.fields, resolved.final, resolved.snapshots)
 }
 
+func appendResolutionDescendantHistory(metadata *Metadata, resolved resolutionResult) {
+	finalPaths := make(map[string]*document.Node)
+	indexDocument(resolved.final, Path{}, finalPaths)
+	for _, delta := range resolved.deltas {
+		record := coveringResolutionSnapshot(resolved.snapshots, delta.path)
+		if record == nil {
+			continue
+		}
+		affected := make(map[string]bool)
+		for pointer := range metadata.history {
+			if path, err := ParsePath(pointer); err == nil && pathContains(delta.path, path) {
+				affected[pointer] = true
+			}
+		}
+		for pointer := range finalPaths {
+			if path, err := ParsePath(pointer); err == nil && pathContains(delta.path, path) {
+				affected[pointer] = true
+			}
+		}
+		pointers := make([]string, 0, len(affected))
+		for pointer := range affected {
+			if pointer != record.change.Path.String() {
+				pointers = append(pointers, pointer)
+			}
+		}
+		slices.Sort(pointers)
+		for _, pointer := range pointers {
+			change := cloneChange(record.change)
+			if info, ok := metadata.fields[pointer]; ok {
+				change.Previous = []Origin{cloneOrigin(info.Origin)}
+			}
+			metadata.history[pointer] = append(metadata.history[pointer], change)
+		}
+	}
+}
+
 func coveringResolutionSnapshot(snapshots []resolutionSnapshot, path Path) *resolutionSnapshot {
 	var covering *resolutionSnapshot
 	for index := range snapshots {

metadata.go

@@ -1,7 +1,7 @@
 package configkit
 
-// Metadata provides detached presence, origin, and history queries for a load.
+// Metadata provides detached effective presence/origin and chronological history
-// Its zero value contains no paths.
+// queries for a load. Its zero value contains no paths.
 type Metadata struct {
 	fields  map[string]FieldInfo
 	history map[string][]Change
@@ -27,8 +27,10 @@ func (m Metadata) IsExplicit(path Path) bool {
 	return ok && info.Explicit
 }
 
-// History returns detached definition and resolution records for path. Unknown and
+// History returns detached definition and resolution records for path. A record's
-// invalid paths return nil.
+// Path can be an ancestor when that container operation replaced or removed path.
+// Removed historical paths can have history even though Lookup returns false.
+// Unknown and invalid paths return nil.
 func (m Metadata) History(path Path) []Change {
 	if !path.valid() {
 		return nil

provenance_test.go

@@ -0,0 +1,218 @@
+package configkit
+
+import (
+	"errors"
+	"reflect"
+	"sync"
+	"testing"
+)
+
+type layeredItem struct {
+	Value string `yaml:"value"`
+}
+
+type layeredConfig struct {
+	Shared struct {
+		Equal    string `yaml:"equal"`
+		Imported string `yaml:"imported"`
+		Profile  string `yaml:"profile"`
+	} `yaml:"shared"`
+	Values   []int                  `yaml:"values"`
+	Items    map[string]layeredItem `yaml:"items"`
+	Value    any                    `yaml:"value"`
+	Nullable *string                `yaml:"nullable"`
+	Derived  int                    `yaml:"derived"`
+	Empty    map[string]string      `yaml:"empty"`
+}
+
+func TestLayeredProvenanceAgreesWithResolvedSnapshot(t *testing.T) {
+	directory := t.TempDir()
+	mainPath := writeImportFile(t, directory, "main.yml", `_configkit:
+  imports: [{path: imported.yml}]
+shared: {equal: same}
+values: [1, 2]
+items:
+  keep: {value: 'yes'}
+  remove: {value: 'no'}
+value: {child: base}
+nullable: base
+empty: {}
+`)
+	importPath := writeImportFile(t, directory, "imported.yml", "shared: {imported: base}\n")
+	profilePath := writeImportFile(t, directory, "profiles/testing.yml", `shared: {profile: selected}
+values: [3, 4]
+value: selected
+nullable: null
+empty: {}
+`)
+	defaultNullable := "default"
+	definition := Definition[layeredConfig]{
+		Defaults: func() layeredConfig {
+			defaults := layeredConfig{
+				Values: []int{0, 0}, Value: map[string]any{"child": "default"},
+				Nullable: &defaultNullable, Empty: map[string]string{"retained": "default"},
+			}
+			defaults.Shared.Equal = "same"
+			return defaults
+		},
+		Resolve: func(configuration *layeredConfig, resolution *Resolution) error {
+			Normalize(resolution, Keys("values"), &configuration.Values, []int{5},
+				"bounded values", Keys("shared", "imported"), Keys("shared", "imported"))
+			Normalize(resolution, Keys("values"), &configuration.Values, []int{5},
+				"bounded values", Keys("shared", "imported"))
+			delete(configuration.Items, "remove")
+			resolution.Record(Keys("items"), ChangeNormalization, "removed disabled items")
+			Default(resolution, Keys("derived"), &configuration.Derived, 7,
+				"derived from equal", Keys("shared", "equal"), Keys("shared", "equal"))
+			resolution.Warning(Keys("values"), "values were bounded", Keys("shared", "imported"))
+			return nil
+		},
+	}
+
+	result, err := Load[layeredConfig](File(mainPath), Options[layeredConfig]{
+		Imports: true, Profile: "testing", Definition: definition,
+	})
+	if err != nil {
+		var loadError *LoadError
+		_ = errors.As(err, &loadError)
+		t.Fatalf("%v: %#v", err, loadError.Diagnostics())
+	}
+	wantSources := []string{canonicalTestPath(t, mainPath), canonicalTestPath(t, importPath), canonicalTestPath(t, profilePath)}
+	sources := result.Sources()
+	gotSources := make([]string, len(sources))
+	for index, source := range sources {
+		gotSources[index] = source.Path
+	}
+	if !reflect.DeepEqual(gotSources, wantSources) || sources[2].Layer != LayerProfile {
+		t.Fatalf("sources = %#v", sources)
+	}
+	if result.Config.Shared.Equal != "same" || result.Config.Shared.Imported != "base" || result.Config.Shared.Profile != "selected" ||
+		!reflect.DeepEqual(result.Config.Values, []int{5}) || result.Config.Nullable != nil || result.Config.Derived != 7 ||
+		result.Config.Empty["retained"] != "default" {
+		t.Fatalf("configuration = %#v", result.Config)
+	}
+
+	metadata := result.Metadata()
+	equal, _ := metadata.Lookup(Keys("shared", "equal"))
+	derived, _ := metadata.Lookup(Keys("derived"))
+	nullable, _ := metadata.Lookup(Keys("nullable"))
+	empty, _ := metadata.Lookup(Keys("empty"))
+	firstValue, _ := metadata.Lookup(Keys("values").Index(0))
+	if !equal.Explicit || equal.Origin.Layer != LayerBase || derived.Explicit || derived.Origin.Operation != ChangeDerivedDefault ||
+		!nullable.Explicit || !nullable.Null || nullable.Origin.Layer != LayerProfile ||
+		!empty.Explicit || empty.Origin.Layer != LayerProfile || len(empty.Origin.Locations) != 2 ||
+		!firstValue.Explicit || firstValue.Origin.Operation != ChangeNormalization {
+		t.Fatalf("metadata = equal %#v derived %#v nullable %#v empty %#v value %#v", equal, derived, nullable, empty, firstValue)
+	}
+	if _, ok := metadata.Lookup(Keys("value", "child")); ok {
+		t.Fatal("profile-removed child remains effective")
+	}
+	if _, ok := metadata.Lookup(Keys("values").Index(1)); ok {
+		t.Fatal("resolution-removed sequence index remains effective")
+	}
+	if _, ok := metadata.Lookup(Keys("items", "remove", "value")); ok {
+		t.Fatal("resolution-removed mapping child remains effective")
+	}
+
+	assertHistoryLayersAndPaths(t, metadata.History(Keys("values").Index(1)),
+		[]Layer{LayerDefault, LayerBase, LayerProfile, LayerResolution},
+		[]string{"/values/1", "/values/1", "/values", "/values"})
+	assertHistoryLayersAndPaths(t, metadata.History(Keys("value", "child")),
+		[]Layer{LayerDefault, LayerBase, LayerProfile},
+		[]string{"/value/child", "/value/child", "/value"})
+	removedHistory := metadata.History(Keys("items", "remove", "value"))
+	assertHistoryLayersAndPaths(t, removedHistory,
+		[]Layer{LayerBase, LayerResolution},
+		[]string{"/items/remove/value", "/items"})
+	if !reflect.DeepEqual(removedHistory[1].Reasons, []string{"removed disabled items"}) {
+		t.Fatalf("removed history = %#v", removedHistory)
+	}
+	valuesHistory := metadata.History(Keys("values").Index(0))
+	lastValue := valuesHistory[len(valuesHistory)-1]
+	if !reflect.DeepEqual(lastValue.Reasons, []string{"bounded values"}) || len(lastValue.Dependencies) != 1 || lastValue.Dependencies[0].String() != "/shared/imported" {
+		t.Fatalf("canonical resolution history = %#v", lastValue)
+	}
+
+	schema, issue := buildSchema(reflect.TypeFor[layeredConfig]())
+	if issue != nil {
+		t.Fatal(issue)
+	}
+	projected, projectionIssue := projectConfiguration(schema, reflect.ValueOf(result.Config), defaultProjectionLimits())
+	if projectionIssue != nil || !documentNodesEqual(projected, result.final) {
+		t.Fatalf("typed projection and private snapshot disagree: %#v", projectionIssue)
+	}
+
+	metadata.history[Keys("values").Index(0).String()][0].Origin.Layer = "mutated"
+	sources[1].ImportedFrom.SourceID = "mutated"
+	diagnostics := result.Diagnostics()
+	diagnostics[0].Related[0].SourceID = "mutated"
+	result.Config.Values[0] = 99
+	freshHistory := result.Metadata().History(Keys("values").Index(0))
+	if freshHistory[0].Origin.Layer == "mutated" || result.Sources()[1].ImportedFrom.SourceID == "mutated" ||
+		result.Diagnostics()[0].Related[0].SourceID == "mutated" {
+		t.Fatal("returned inspection records alias private state")
+	}
+	snapshotValue, ok := documentAtPath(result.final, Keys("values").Index(0))
+	if !ok || snapshotValue.Text != "5" {
+		t.Fatal("caller configuration mutation changed the private final snapshot")
+	}
+}
+
+func TestSelectedProfileLoadsRemainIndependentConcurrently(t *testing.T) {
+	directory := t.TempDir()
+	mainPath := writeImportFile(t, directory, "main.yml", "values: [1, 2]\n")
+	writeImportFile(t, directory, "profiles/testing.yml", "values: [3, 4]\n")
+	options := Options[layeredConfig]{Profile: "testing"}
+
+	const loads = 16
+	results := make(chan *Result[layeredConfig], loads)
+	errors := make(chan error, loads)
+	var group sync.WaitGroup
+	for index := 0; index < loads; index++ {
+		group.Add(1)
+		go func() {
+			defer group.Done()
+			result, err := Load[layeredConfig](File(mainPath), options)
+			if err != nil {
+				errors <- err
+				return
+			}
+			results <- result
+		}()
+	}
+	group.Wait()
+	close(results)
+	close(errors)
+	for err := range errors {
+		t.Fatal(err)
+	}
+	var collected []*Result[layeredConfig]
+	for result := range results {
+		collected = append(collected, result)
+	}
+	if len(collected) != loads {
+		t.Fatalf("completed loads = %d", len(collected))
+	}
+	collected[0].Config.Values[0] = 99
+	collected[0].metadata.fields[Keys("values").String()] = FieldInfo{}
+	for _, result := range collected[1:] {
+		if !reflect.DeepEqual(result.Config.Values, []int{3, 4}) {
+			t.Fatalf("concurrent result contaminated = %#v", result.Config.Values)
+		}
+		if info, ok := result.Metadata().Lookup(Keys("values")); !ok || info.Origin.Layer != LayerProfile {
+			t.Fatalf("concurrent metadata contaminated = %#v, %v", info, ok)
+		}
+	}
+}
+
+func assertHistoryLayersAndPaths(t *testing.T, history []Change, layers []Layer, paths []string) {
+	t.Helper()
+	if len(history) != len(layers) || len(history) != len(paths) {
+		t.Fatalf("history = %#v", history)
+	}
+	for index, change := range history {
+		if change.Origin.Layer != layers[index] || change.Path.String() != paths[index] {
+			t.Fatalf("history[%d] = %#v", index, change)
+		}
+	}
+}

records.go

@@ -150,7 +150,8 @@ type Diagnostic struct {
 }
 
 // Origin describes where an effective definition came from. Multiple locations
-// represent a composite origin.
+// represent a composite origin; resolution origins are synthetic and identify their
+// operation instead of a file location.
 type Origin struct {
 	Layer     Layer
 	Locations []Location
@@ -164,8 +165,9 @@ type FieldInfo struct {
 	Origin   Origin
 }
 
-// Change describes a definition or application-resolution operation. Historical
+// Change describes a definition or application-resolution operation. Path can name
-// values are intentionally not exposed by this record.
+// an ancestor operation affecting the queried descendant. Historical values are
+// intentionally not exposed by this record.
 type Change struct {
 	Path         Path
 	Origin       Origin