ConfigKit/docs/roadmap/implementation.md

# Source origins during resolution: implementation plan

Status: not started. Implement stages in order, starting with Stage 1.
The [feature roadmap](source-origins-during-resolution.md) owns the target contract
and consumer intent. This document owns implementation steps, validation, and
handoffs. It is written for one gpt-5.6-terra coding agent per stage; no stage
requires parallel agents or changes to a downstream repository.

## Required reading and working rules

Before each stage, inspect the worktree and read the preceding stage's completion
record. Follow [development](../development.md) and all policies:
[architecture](../policy/architecture.md), [documentation](../policy/documentation.md),
[testing](../policy/testing.md), and [release](../policy/release.md).
Read the roadmap, the configuration contract's resolution and metadata sections,
and the relevant code before editing. Use the code graph for discovery and verify
current definitions rather than assuming line numbers or private signatures.

Preserve unrelated work. Keep each stage independently buildable with passing
tests; do not introduce stub public methods or defer known correctness failures.
Run the focused commands listed for the stage and `./scripts/check.sh` from the
repository root. Acquire required toolchains/dependencies explicitly before offline
checks. Document unsupported filesystem fixtures rather than silently omitting them.

Use real collaborators and temporary files for provenance. Keep default tests
offline, deterministic, and credential-free. Prefer codes, paths, layers, locations,
and observable snapshots over private call counts or incidental error messages.
Only use a narrow I/O or credential seam where it proves the no-extra-access rule.

At each handoff, update that stage's status here with changed files, tests/results,
actual toolchains, deviations, and remaining risks. Keep public reference and current
contract documentation synchronized with behavior as it lands. Commit coherent
stage work when authorized, but do not push, tag, select a version, or prepare a
speculative release note as part of this implementation sprint.

## Implementation decisions

The roadmap's two method signatures are fixed for this sprint:

```go
func (r *Resolution) InputInfo(path Path) (FieldInfo, bool)
func (r *Resolution) Source(id string) (SourceInfo, bool)
```

Use a private, immutable entry-state view, not live final metadata. Store only
effective field records, effective path shapes, and a source-ID index; do not build
or expose an additional historical view. Reuse `Metadata.Lookup` semantics by
representing the field/path part as a private `Metadata` value with only those
indexes populated. Never return that `Metadata` value publicly.

Build the view after `metadataFromDecoded` and `applyOverlayMetadata` in
`resolveLoadInput`, immediately before `runResolution`, only when a resolver exists.
Its path shapes come from `decoded.final`, not the raw document or mutable Go
configuration. Clone field records and their nested origins into a separate map;
do not shallow-copy the mutable final metadata map. Build a map keyed by ID from
`assembledInput.sources`, cloning `SourceInfo` records with existing helpers.
Missing or duplicate internally generated IDs are implementation-invariant failures
to fix, not new consumer-facing diagnostic categories.

Pass the private view as an explicit argument to `runResolution` and retain it in
`resolutionState`. Update all private callers/tests deliberately: construct their
entry view from their decoded fixtures and supplied sources where they test these
queries. Do not add optional/variadic arguments that silently omit provenance in
production. Private names are discretionary, but ownership and timing are not.

Queries first check callback activity through the existing context guard. Only
then may invalid paths or unknown IDs return misses. `InputInfo` delegates to the
entry metadata lookup; `Source` clones the matched source on return. Preserve
existing `IsExplicit`, diagnostics, resolution recording, and secret timing.

Do not clone input trees, re-project configuration, read files, resolve symlinks,
or access credentials during a query. Effective indexing is O(number of entry
paths), source indexing is O(number of sources), and each lookup must avoid a full
tree/source scan. Returned-copy cost is proportional to the returned record.

## Stage 1 — Factor reusable effective-path indexing

Status: not started. Dependencies: none.

Scope: `metadata.go`, `load_metadata.go`, and their focused test owners.

1. Extract the effective-path indexing portion of `finalizeMetadataPaths` into one
   private helper using existing document traversal and detached path construction.
   Keep historical indexing and result publication behavior unchanged.
2. Make result finalization use the extracted helper. Add the private entry-view
   constructor described above, accepting the pre-resolution metadata, decoded
   final tree, and acquired source records. Do not expose public methods yet.
3. Ensure the constructor copies field records and source records without copying
   history or configuration values. Reuse `cloneFieldInfo`, `cloneSourceInfo`, and
   existing shape matching instead of writing another path parser or numeric matcher.
4. Add focused coverage proving the private view recognizes defaulted fields,
   missing descendants, root, escaped keys, explicit key/index mismatches, and
   contextual canonical sequence indices. Prove later mutation of the original
   metadata and source records cannot alter the view. Reuse existing shape tests
   for exhaustive syntax cases rather than duplicating them.

Acceptance: all existing final metadata/history behavior is unchanged; a complete
entry view can be constructed before final publication without a historical index.
No public API or configuration contract changes are claimed by this stage.

Validation: `go test . -run 'Metadata|Path|Provenance|Historical'`, then the everyday
checker. Handoff should identify the helper and constructor signatures for Stage 2.

## Stage 2 — Wire the callback API and enforce lifetime

Status: not started. Dependencies: Stage 1.

Scope: `load_result.go`, `resolution.go`, necessary private callers, reference
comments, and resolution/callback ownership tests.

1. Build and pass the entry view at the existing pre-resolution boundary. Use
   `input.sources`, which includes sources acquired by import, profile, and CLI
   processing; do not reconstruct source identities from labels or field locations.
2. Implement both public methods with the existing activity guard and detached
   returns. Keep lookup state frozen when recording changes or mutating configuration.
3. Add public-load tests for a file-authored scalar, a defaulted field, explicit
   null, a missing path, empty/unknown source IDs, and `NoSource`. Assert exact
   `InputInfo.Explicit` agreement with `IsExplicit` for found paths.
4. Exercise nil/zero contexts and retained contexts after successful and failed
   callbacks. Both methods must use the existing fixed closed-context panic behavior.
   Saved return records remain usable after callback exit. Do not broaden this work
   into a new concurrency contract or a general callback framework refactor.
5. Add Go method documentation and update the configuration contract's resolution
   section to describe the implemented entry-state semantics, misses, lifetime,
   and source distinction. Fix the development guide's stale statement that this
   implementation plan has not yet been written.

Acceptance: both methods work during an ordinary public `Load`; calls cannot access
live mutable metadata or bypass callback scope. No-rule/no-resolver behavior and
existing mutation auditing remain unchanged.

Validation: `go test . -run 'Resolution|Callback|Metadata|Load'`, then the everyday
checker. Do not mark complete with public methods that work only in private tests.

## Stage 3 — Verify layered provenance and snapshot independence

Status: not started. Dependencies: Stage 2.

Scope: public lifecycle/provenance, import/profile/override, and ownership tests;
production fixes only for gaps exposed at those boundaries.

1. Build a real temporary import graph with path-bearing scalars and a sequence
   element. Include a composite mapping with contributions from distinct files.
   During resolution, assert scalar ownership, all container contributors, and
   successful source lookup for every origin location returned by the view.
2. Apply a profile replacing one scalar and an equal-text value; verify replacing
   origins and untouched siblings. Exercise equal-text CLI replacement separately.
   Keep source IDs derived from returned records, not inferred from value equality.
3. Exercise byte and reader roots with filesystem context and file imports: the
   memory root has no file path while imported files do. Also cover non-file input
   without imports, default-only fields, and unknown source IDs. Retain default
   descendants reintroduced by normal decoding instead of treating them as raw
   input declarations.
4. Extend shape coverage through inlined structs/maps, typed arrays, and dynamic
   structures. Assert misses for invalid indices, signed index tokens in sequences,
   and explicit key/index mismatches; signed mapping keys remain literal keys.
5. Normalize, delete, add, and move fields within a callback using valid records.
   Queries for original paths must stay fixed; absent destinations stay absent,
   and pre-existing destinations retain their entry information. Final metadata
   must still reflect resolution rather than that fixed entry view.
6. Mutate returned `Origin.Locations` and `SourceInfo.ImportedFrom`; verify repeated
   queries, saved records, final results, and another load are independent. Exercise
   independent loads under the race detector without sharing an active context.

Acceptance: roadmap criteria 2–8 and 11 are covered at their owning boundaries.
No duplicate provenance model, nearest-ancestor fallback, or source-order heuristic
is introduced. Any discovered defect is fixed before the stage is marked complete.

Validation: `go test . -run 'Resolution|Provenance|Import|Profile|Override|Metadata|Path'`,
`go test -race ./...`, then the everyday checker.

## Stage 4 — Demonstrate source-aware normalization and safe inspection

Status: not started. Dependencies: Stage 3.

Scope: maintained Notarius-shaped example code/fixtures, executable examples,
explanation/secret integration tests, and consumer guidance.

1. Add a small application-owned path-bearing configuration example in
   `examples/consumers`. Keep existing catalog/version examples working. Supply the
   invocation directory explicitly to the example loader rather than calling
   `os.Chdir` or adding process-global state to tests.
2. Implement the example's policy using the two public queries: absolute paths
   unchanged; CLI-relative paths based on the supplied invocation directory;
   default-relative paths based on the canonical main source; file-authored paths
   based on the scalar's declaring source. Use the existing primary-source identity
   convention for main-source lookup, not a guess about contributor ordering.
3. For this file-rooted example, require one unambiguous canonical declaring file
   for file-relative values. Report a safe application resolution error if that
   prerequisite is absent or ambiguous. Do not silently use a diagnostic label or
   working directory. Non-file-root fallback remains an explicitly separate
   application policy, not a new library option.
4. Record every changed path through `Normalize` or `Record`, including valid
   dependencies where derived values depend on other configuration paths. Prove
   runtime values, effective YAML, and explanation agree; retain the original
   relative declaration and normalization reason in history. No existence or
   confinement checks are part of the example.
5. Add a symlink fixture proving canonical target-directory semantics and an
   invocation-directory mismatch proving resolution does not depend on process CWD.
   Make unsupported symlink capability explicit under the testing policy.
6. Use synthetic sensitive fields to verify original/effective redaction remains
   intact. With existing reader/secret lookup seams, prove queries do not reread
   source data or resolve credentials during the resolver; normal later lookup
   remains unchanged. Do not create credentials or environment-dependent tests.
7. Document the public workflow and canonical symlink choice in the consumer guide;
   point to executable artifacts instead of duplicating large code examples.

Acceptance: roadmap criteria 1, 9, 10, and 12 are demonstrated in one-load public
workflows. Filesystem policy stays in the example application, not the root library.

Validation: `go test ./examples/consumers`,
`go test . -run 'Resolution|Explain|Secret|Example'`, then the everyday checker.

## Stage 5 — Audit the complete contract and hand off release readiness

Status: not started. Dependencies: Stage 4.

Scope: integration review, canonical documentation, roadmap completion records,
and proportionate fixes for remaining acceptance failures. No feature expansion.

1. Check all twelve roadmap acceptance criteria against named tests and current
   implementation. Record the mapping in this stage's completion entry. Passing
   broad suites alone is not evidence that a missing scenario was exercised.
2. Inspect every new query path for environment/filesystem access, raw values,
   mutable aliases, accidental callback-lifetime extension, and per-query whole-tree
   work. Confirm no history was duplicated solely for these queries and loads
   without a resolver avoid constructing an unused entry snapshot.
3. Review public signatures and current configuration behavior against the previous
   release. Confirm the capability is additive and no parser, path, composition,
   credential, or existing callback semantics changed unintentionally.
4. Update the internal overview and finish canonical Go/configuration/consumer
   documentation. Transfer implemented contract prose from the feature roadmap
   to its proper owners and retain a brief completion record with links. Preserve
   historical ADR rationale and published notes. No new ADR is required for this
   information-access extension unless implementation actually changes a governing
   decision; stop and obtain review rather than silently widening scope.
5. Run the full source checker with qualifying acquired toolchains, recording the
   actual versions and host. The following are the repository's recorded versions;
   follow the release runbook if selecting updated toolchains:

   ```sh
   CONFIGKIT_BASELINE_TOOLCHAIN=go1.25.5 \
   CONFIGKIT_NEWER_TOOLCHAIN=go1.26.7 \
   ./scripts/release-check.sh
   ```

6. Hand off remaining publication work separately: maintainer version choice,
   dependency reassessment, version-matched release notes, completed-roadmap
   retirement and link repair, exact-commit Linux/macOS validation, publication
   guards, and clean consumer retrieval. Follow [the runbook](../release.md).
   Historical one-release exceptions do not carry forward to this feature.

Acceptance: all target requirements have concrete evidence; available automated
checks pass; documentation distinguishes implemented behavior from release status.
Implementation completion is not publication approval. Do not create a release
tag or mark unavailable native macOS evidence as passed.

Validation: `./scripts/check.sh`, `go test -race ./...`, and the full checker above.

## Open Questions

None blocking implementation. The API, snapshot boundary, missing-value semantics,
source identity handling, ownership, and example fallback policy are specified by
the roadmap and decisions above. Release version and publication timing are later
maintainer decisions, not unresolved implementation choices. If new evidence
contradicts a target contract, document the conflict, a recommended correction and
its rationale, and at least one viable alternative before proceeding past it.