ConfigKit/docs/roadmap/implementation.md

# Source origins during resolution: implementation plan

Status: not started. Implement stages in order, starting with Stage 1.
The [feature roadmap](source-origins-during-resolution.md) owns the target contract
and consumer intent. This document owns implementation steps, validation, and
handoffs. It is written for one gpt-5.6-terra coding agent per stage; no stage
requires parallel agents or changes to a downstream repository.

## Required reading and working rules

Before each stage, inspect the worktree and read the preceding stage's completion
record. Follow [development](../development.md) and all policies:
[architecture](../policy/architecture.md), [documentation](../policy/documentation.md),
[testing](../policy/testing.md), and [release](../policy/release.md).
Read the roadmap, the configuration contract's resolution and metadata sections,
and the relevant code before editing. Use the code graph for discovery and verify
current definitions rather than assuming line numbers or private signatures.

Preserve unrelated work. Keep each stage independently buildable with passing
tests; do not introduce stub public methods or defer known correctness failures.
Run the focused commands listed for the stage and `./scripts/check.sh` from the
repository root. Acquire required toolchains/dependencies explicitly before offline
checks. Document unsupported filesystem fixtures rather than silently omitting them.

Use real collaborators and temporary files for provenance. Keep default tests
offline, deterministic, and credential-free. Prefer codes, paths, layers, locations,
and observable snapshots over private call counts or incidental error messages.
Only use a narrow I/O or credential seam where it proves the no-extra-access rule.

At each handoff, update that stage's status here with changed files, tests/results,
actual toolchains, deviations, and remaining risks. Keep public reference and current
contract documentation synchronized with behavior as it lands. Commit coherent
stage work when authorized, but do not push, tag, select a version, or prepare a
speculative release note as part of this implementation sprint.

## Implementation decisions

The roadmap's two method signatures are fixed for this sprint:

```go
func (r *Resolution) InputInfo(path Path) (FieldInfo, bool)
func (r *Resolution) Source(id string) (SourceInfo, bool)
```

The required behavior is a stable entry-state view and detached public return
records, not a particular internal representation. The recommended implementation
is a private `Metadata` value containing effective fields and path shapes, together
with a source-ID index. Reuse `Metadata.Lookup` semantics. A historical view is not
needed for this feature, and exposing one is outside this sprint's scope.

Build the view after `metadataFromDecoded` and `applyOverlayMetadata` in
`resolveLoadInput`, immediately before `runResolution`, only when a resolver exists.
Its path shapes come from `decoded.final`, not the raw document or mutable Go
configuration. Separate copies of field/source records are one straightforward way
to isolate the view; immutable sharing or ownership transfer is equally acceptable
when it preserves query stability and detached returns. Source records come from
`assembledInput.sources`. Avoid aliases that observable consumer operations or
subsequent metadata updates could mutate while the view remains usable.
Missing or duplicate internally generated IDs are implementation-invariant failures
to fix, not new consumer-facing diagnostic categories.

Prefer passing the private view explicitly to `runResolution` and retaining it in
`resolutionState`. Update all private callers/tests deliberately: construct their
entry view from their decoded fixtures and supplied sources where they test these
queries. Every production resolution callback must receive complete provenance.
Private signatures and representation may change when an idiomatic alternative
better serves that requirement; record the reasoning in the stage handoff.

Queries first check callback activity through the existing context guard. Only
then may invalid paths or unknown IDs return misses. `InputInfo` delegates to the
entry metadata lookup; `Source` clones the matched source on return. Preserve
existing `IsExplicit`, diagnostics, resolution recording, and secret timing.

Queries must not reload input, read files, resolve symlinks, access credentials,
or re-project mutable configuration instead of reading entry-state information.
Prefer reusing existing indexes and avoiding unnecessary whole-tree copies.
Per-load source indexing is a recommendation, not a complexity guarantee: a simple
scan may be appropriate for the existing bounded source collection. Assess costs
in context, including path lengths and nested record copies; this feature adds no
performance guarantees, resource caps, or benchmark gates.

## Stage 1 — Factor reusable effective-path indexing

Status: not started. Dependencies: none.

Scope: `metadata.go`, `load_metadata.go`, and their focused test owners.

1. Prepare reusable effective-path lookup for the entry view. The recommended
   approach extracts indexing from `finalizeMetadataPaths`; an equivalent design
   may reuse existing traversal directly. Keep historical indexing and result
   publication behavior unchanged, and preserve one set of semantic-path rules.
2. Add the private entry-view construction described above using pre-resolution
   metadata, the decoded final tree, and acquired source records. Share extracted
   machinery with result finalization where useful. Do not expose public methods yet.
3. Establish and document internal ownership sufficient for stable queries and
   detached returns. Reuse `cloneFieldInfo` and `cloneSourceInfo` where copying is
   needed; do not require eager copies or duplicate history solely to match this
   plan's recommended representation.
4. Add focused coverage proving the private view recognizes defaulted fields,
   missing descendants, root, escaped keys, explicit key/index mismatches, and
   contextual canonical sequence indices. Test stability across actual lifecycle
   operations and mutation of returned records, not arbitrary mutation of backing
   data that the implementation owns as immutable. Reuse existing shape tests for
   exhaustive syntax cases rather than duplicating them.

Acceptance: all existing final metadata/history behavior is unchanged; a complete
entry view can be constructed before final publication without a historical index.
No public API or configuration contract changes are claimed by this stage.

Validation: `go test . -run 'Metadata|Path|Provenance|Historical'`, then the everyday
checker. Handoff should identify the helper and constructor signatures for Stage 2.

## Stage 2 — Wire the callback API and enforce lifetime

Status: not started. Dependencies: Stage 1.

Scope: `load_result.go`, `resolution.go`, necessary private callers, reference
comments, and resolution/callback ownership tests.

1. Build and pass the entry view at the existing pre-resolution boundary. Use
   `input.sources`, which includes sources acquired by import, profile, and CLI
   processing; do not reconstruct source identities from labels or field locations.
2. Implement both public methods with the existing activity guard and detached
   returns. Keep lookup state frozen when recording changes or mutating configuration.
3. Add public-load tests for a file-authored scalar, a defaulted field, explicit
   null, a missing path, empty/unknown source IDs, and `NoSource`. Assert exact
   `InputInfo.Explicit` agreement with `IsExplicit` for found paths.
4. Exercise nil/zero contexts and retained contexts after successful and failed
   callbacks. Both methods must use the existing fixed closed-context panic behavior.
   Saved return records remain usable after callback exit. Do not broaden this work
   into a new concurrency contract or a general callback framework refactor.
5. Add Go method documentation and update the configuration contract's resolution
   section to describe the implemented entry-state semantics, misses, lifetime,
   and source distinction. Fix the development guide's stale statement that this
   implementation plan has not yet been written.

Acceptance: both methods work during an ordinary public `Load`; calls cannot access
live mutable metadata or bypass callback scope. No-rule/no-resolver behavior and
existing mutation auditing remain unchanged.

Validation: `go test . -run 'Resolution|Callback|Metadata|Load'`, then the everyday
checker. Do not mark complete with public methods that work only in private tests.

## Stage 3 — Verify layered provenance and snapshot independence

Status: not started. Dependencies: Stage 2.

Scope: public lifecycle/provenance, import/profile/override, and ownership tests;
production fixes only for gaps exposed at those boundaries.

1. Build a real temporary import graph with path-bearing scalars and a sequence
   element. Include a composite mapping with contributions from distinct files.
   During resolution, assert scalar ownership, all container contributors, and
   successful source lookup for every origin location returned by the view.
2. Apply a profile replacing one scalar and an equal-text value; verify replacing
   origins and untouched siblings. Exercise equal-text CLI replacement separately.
   Keep source IDs derived from returned records, not inferred from value equality.
3. Exercise byte and reader roots with filesystem context and file imports: the
   memory root has no file path while imported files do. Also cover non-file input
   without imports, default-only fields, and unknown source IDs. Retain default
   descendants reintroduced by normal decoding instead of treating them as raw
   input declarations.
4. Extend shape coverage through inlined structs/maps, typed arrays, and dynamic
   structures. Assert misses for invalid indices, signed index tokens in sequences,
   and explicit key/index mismatches; signed mapping keys remain literal keys.
5. Normalize, delete, add, and move fields within a callback using valid records.
   Queries for original paths must stay fixed; absent destinations stay absent,
   and pre-existing destinations retain their entry information. Final metadata
   must still reflect resolution rather than that fixed entry view.
6. Mutate returned `Origin.Locations` and `SourceInfo.ImportedFrom`; verify repeated
   queries, saved records, final results, and another load are independent. Exercise
   independent loads under the race detector without sharing an active context.

Acceptance: roadmap criteria 2–8 and 11 are covered at their owning boundaries.
No duplicate provenance model, nearest-ancestor fallback, or source-order heuristic
is introduced. Any discovered defect is fixed before the stage is marked complete.

Validation: `go test . -run 'Resolution|Provenance|Import|Profile|Override|Metadata|Path'`,
`go test -race ./...`, then the everyday checker.

## Stage 4 — Demonstrate source-aware normalization and safe inspection

Status: not started. Dependencies: Stage 3.

Scope: maintained Notarius-shaped example code/fixtures, executable examples,
explanation/secret integration tests, and consumer guidance.

1. Add a small application-owned path-bearing configuration example in
   `examples/consumers`. Keep existing catalog/version examples working. Supply the
   invocation directory explicitly to the example loader rather than calling
   `os.Chdir` or adding process-global state to tests.
2. Implement the example's policy using the two public queries: absolute paths
   unchanged; CLI-relative paths based on the supplied invocation directory;
   default-relative paths based on the canonical main source; file-authored paths
   based on the scalar's declaring source. Use the existing primary-source identity
   convention for main-source lookup, not a guess about contributor ordering.
3. Demonstrate file-relative normalization with fixtures that have an unambiguous
   declaring file. If the example handles absent or ambiguous file provenance,
   make that behavior an explicit example policy, not a correctness requirement
   or prescribed Notarius policy. Rejection, a caller-supplied fallback directory,
   retaining a relative value, or warning are application choices. The example's
   selected behavior must be documented and tested; ConfigKit must continue to
   expose the distinction rather than fabricate a file from a diagnostic label.
4. Record every changed path through `Normalize` or `Record`, including valid
   dependencies where derived values depend on other configuration paths. Prove
   runtime values, effective YAML, and explanation agree; retain the original
   relative declaration and normalization reason in history. No existence or
   confinement checks are part of the example.
5. Add a symlink fixture proving canonical target-directory semantics and an
   invocation-directory mismatch proving resolution does not depend on process CWD.
   Make unsupported symlink capability explicit under the testing policy.
6. Use synthetic sensitive fields to verify original/effective redaction remains
   intact. With existing reader/secret lookup seams, prove queries do not reread
   source data or resolve credentials during the resolver; normal later lookup
   remains unchanged. Do not create credentials or environment-dependent tests.
7. Document the public workflow and canonical symlink choice in the consumer guide;
   point to executable artifacts instead of duplicating large code examples.

Acceptance: roadmap criteria 1, 9, 10, and 12 are demonstrated in one-load public
workflows. Filesystem policy stays in the example application, not the root library.

Validation: `go test ./examples/consumers`,
`go test . -run 'Resolution|Explain|Secret|Example'`, then the everyday checker.

## Stage 5 — Audit the complete contract and hand off release readiness

Status: not started. Dependencies: Stage 4.

Scope: integration review, canonical documentation, roadmap completion records,
and proportionate fixes for remaining acceptance failures. No feature expansion.

1. Check all twelve roadmap acceptance criteria against named tests and current
   implementation. Record the mapping in this stage's completion entry. Passing
   broad suites alone is not evidence that a missing scenario was exercised.
2. Inspect every new query path for environment/filesystem access, raw values,
   caller-mutable aliases, and accidental callback-lifetime extension. Review
   allocation and traversal costs for proportionality without turning the chosen
   mechanism into a public guarantee or adding unsupported performance gates.
   Avoid unnecessary history duplication and unused work in no-resolver loads.
3. Review public signatures and current configuration behavior against the previous
   release. Confirm the capability is additive and no parser, path, composition,
   credential, or existing callback semantics changed unintentionally.
4. Update the internal overview and finish canonical Go/configuration/consumer
   documentation. Transfer implemented contract prose from the feature roadmap
   to its proper owners and retain a brief completion record with links. Preserve
   historical ADR rationale and published notes. No new ADR is required for this
   information-access extension unless implementation actually changes a governing
   decision; stop and obtain review rather than silently widening scope.
5. Run the full source checker with qualifying acquired toolchains, recording the
   actual versions and host. The following are the repository's recorded versions;
   follow the release runbook if selecting updated toolchains:

   ```sh
   CONFIGKIT_BASELINE_TOOLCHAIN=go1.25.5 \
   CONFIGKIT_NEWER_TOOLCHAIN=go1.26.7 \
   ./scripts/release-check.sh
   ```

6. Hand off remaining publication work separately: maintainer version choice,
   dependency reassessment, version-matched release notes, completed-roadmap
   retirement and link repair, exact-commit Linux/macOS validation, publication
   guards, and clean consumer retrieval. Follow [the runbook](../release.md).
   Historical one-release exceptions do not carry forward to this feature.

Acceptance: all target requirements have concrete evidence; available automated
checks pass; documentation distinguishes implemented behavior from release status.
Implementation completion is not publication approval. Do not create a release
tag or mark unavailable native macOS evidence as passed.

Validation: `./scripts/check.sh`, `go test -race ./...`, and the full checker above.

## Open Questions

None blocking implementation. The API, snapshot boundary, missing-value semantics,
source identity handling, and ownership are specified by the roadmap and decisions
above. Example fallback behavior is illustrative application policy, not a missing
library contract; record any selected example behavior without prescribing it to
consumers. Release version and publication timing are later
maintainer decisions, not unresolved implementation choices. If new evidence
contradicts a target contract, document the conflict, a recommended correction and
its rationale, and at least one viable alternative before proceeding past it.