eric/ConfigKit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Plan source origins during resolution

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-12 12:52:32 +00:00
parent 8a62b42503
commit 381430a24b
3 changed files with 499 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

267
docs/roadmap/implementation.md Normal file
View File

@@ -0,0 +1,267 @@
# Source origins during resolution: implementation plan
Status: not started. Implement stages in order, starting with Stage 1.
The [feature roadmap](source-origins-during-resolution.md) owns the target contract
and consumer intent. This document owns implementation steps, validation, and
handoffs. It is written for one gpt-5.6-terra coding agent per stage; no stage
requires parallel agents or changes to a downstream repository.
## Required reading and working rules
Before each stage, inspect the worktree and read the preceding stage's completion
record. Follow [development](../development.md) and all policies:
[architecture](../policy/architecture.md), [documentation](../policy/documentation.md),
[testing](../policy/testing.md), and [release](../policy/release.md).
Read the roadmap, the configuration contract's resolution and metadata sections,
and the relevant code before editing. Use the code graph for discovery and verify
current definitions rather than assuming line numbers or private signatures.
Preserve unrelated work. Keep each stage independently buildable with passing
tests; do not introduce stub public methods or defer known correctness failures.
Run the focused commands listed for the stage and `./scripts/check.sh` from the
repository root. Acquire required toolchains/dependencies explicitly before offline
checks. Document unsupported filesystem fixtures rather than silently omitting them.
Use real collaborators and temporary files for provenance. Keep default tests
offline, deterministic, and credential-free. Prefer codes, paths, layers, locations,
and observable snapshots over private call counts or incidental error messages.
Only use a narrow I/O or credential seam where it proves the no-extra-access rule.
At each handoff, update that stage's status here with changed files, tests/results,
actual toolchains, deviations, and remaining risks. Keep public reference and current
contract documentation synchronized with behavior as it lands. Commit coherent
stage work when authorized, but do not push, tag, select a version, or prepare a
speculative release note as part of this implementation sprint.
## Implementation decisions
The roadmap's two method signatures are fixed for this sprint:
```go
func (r *Resolution) InputInfo(path Path) (FieldInfo, bool)
func (r *Resolution) Source(id string) (SourceInfo, bool)
```
Use a private, immutable entry-state view, not live final metadata. Store only
effective field records, effective path shapes, and a source-ID index; do not build
or expose an additional historical view. Reuse `Metadata.Lookup` semantics by
representing the field/path part as a private `Metadata` value with only those
indexes populated. Never return that `Metadata` value publicly.
Build the view after `metadataFromDecoded` and `applyOverlayMetadata` in
`resolveLoadInput`, immediately before `runResolution`, only when a resolver exists.
Its path shapes come from `decoded.final`, not the raw document or mutable Go
configuration. Clone field records and their nested origins into a separate map;
do not shallow-copy the mutable final metadata map. Build a map keyed by ID from
`assembledInput.sources`, cloning `SourceInfo` records with existing helpers.
Missing or duplicate internally generated IDs are implementation-invariant failures
to fix, not new consumer-facing diagnostic categories.
Pass the private view as an explicit argument to `runResolution` and retain it in
`resolutionState`. Update all private callers/tests deliberately: construct their
entry view from their decoded fixtures and supplied sources where they test these
queries. Do not add optional/variadic arguments that silently omit provenance in
production. Private names are discretionary, but ownership and timing are not.
Queries first check callback activity through the existing context guard. Only
then may invalid paths or unknown IDs return misses. `InputInfo` delegates to the
entry metadata lookup; `Source` clones the matched source on return. Preserve
existing `IsExplicit`, diagnostics, resolution recording, and secret timing.
Do not clone input trees, re-project configuration, read files, resolve symlinks,
or access credentials during a query. Effective indexing is O(number of entry
paths), source indexing is O(number of sources), and each lookup must avoid a full
tree/source scan. Returned-copy cost is proportional to the returned record.
## Stage 1 — Factor reusable effective-path indexing
Status: not started. Dependencies: none.
Scope: `metadata.go`, `load_metadata.go`, and their focused test owners.
1. Extract the effective-path indexing portion of `finalizeMetadataPaths` into one
private helper using existing document traversal and detached path construction.
Keep historical indexing and result publication behavior unchanged.
2. Make result finalization use the extracted helper. Add the private entry-view
constructor described above, accepting the pre-resolution metadata, decoded
final tree, and acquired source records. Do not expose public methods yet.
3. Ensure the constructor copies field records and source records without copying
history or configuration values. Reuse `cloneFieldInfo`, `cloneSourceInfo`, and
existing shape matching instead of writing another path parser or numeric matcher.
4. Add focused coverage proving the private view recognizes defaulted fields,
missing descendants, root, escaped keys, explicit key/index mismatches, and
contextual canonical sequence indices. Prove later mutation of the original
metadata and source records cannot alter the view. Reuse existing shape tests
for exhaustive syntax cases rather than duplicating them.
Acceptance: all existing final metadata/history behavior is unchanged; a complete
entry view can be constructed before final publication without a historical index.
No public API or configuration contract changes are claimed by this stage.
Validation: `go test . -run 'Metadata|Path|Provenance|Historical'`, then the everyday
checker. Handoff should identify the helper and constructor signatures for Stage 2.
## Stage 2 — Wire the callback API and enforce lifetime
Status: not started. Dependencies: Stage 1.
Scope: `load_result.go`, `resolution.go`, necessary private callers, reference
comments, and resolution/callback ownership tests.
1. Build and pass the entry view at the existing pre-resolution boundary. Use
`input.sources`, which includes sources acquired by import, profile, and CLI
processing; do not reconstruct source identities from labels or field locations.
2. Implement both public methods with the existing activity guard and detached
returns. Keep lookup state frozen when recording changes or mutating configuration.
3. Add public-load tests for a file-authored scalar, a defaulted field, explicit
null, a missing path, empty/unknown source IDs, and `NoSource`. Assert exact
`InputInfo.Explicit` agreement with `IsExplicit` for found paths.
4. Exercise nil/zero contexts and retained contexts after successful and failed
callbacks. Both methods must use the existing fixed closed-context panic behavior.
Saved return records remain usable after callback exit. Do not broaden this work
into a new concurrency contract or a general callback framework refactor.
5. Add Go method documentation and update the configuration contract's resolution
section to describe the implemented entry-state semantics, misses, lifetime,
and source distinction. Fix the development guide's stale statement that this
implementation plan has not yet been written.
Acceptance: both methods work during an ordinary public `Load`; calls cannot access
live mutable metadata or bypass callback scope. No-rule/no-resolver behavior and
existing mutation auditing remain unchanged.
Validation: `go test . -run 'Resolution|Callback|Metadata|Load'`, then the everyday
checker. Do not mark complete with public methods that work only in private tests.
## Stage 3 — Verify layered provenance and snapshot independence
Status: not started. Dependencies: Stage 2.
Scope: public lifecycle/provenance, import/profile/override, and ownership tests;
production fixes only for gaps exposed at those boundaries.
1. Build a real temporary import graph with path-bearing scalars and a sequence
element. Include a composite mapping with contributions from distinct files.
During resolution, assert scalar ownership, all container contributors, and
successful source lookup for every origin location returned by the view.
2. Apply a profile replacing one scalar and an equal-text value; verify replacing
origins and untouched siblings. Exercise equal-text CLI replacement separately.
Keep source IDs derived from returned records, not inferred from value equality.
3. Exercise byte and reader roots with filesystem context and file imports: the
memory root has no file path while imported files do. Also cover non-file input
without imports, default-only fields, and unknown source IDs. Retain default
descendants reintroduced by normal decoding instead of treating them as raw
input declarations.
4. Extend shape coverage through inlined structs/maps, typed arrays, and dynamic
structures. Assert misses for invalid indices, signed index tokens in sequences,
and explicit key/index mismatches; signed mapping keys remain literal keys.
5. Normalize, delete, add, and move fields within a callback using valid records.
Queries for original paths must stay fixed; absent destinations stay absent,
and pre-existing destinations retain their entry information. Final metadata
must still reflect resolution rather than that fixed entry view.
6. Mutate returned `Origin.Locations` and `SourceInfo.ImportedFrom`; verify repeated
queries, saved records, final results, and another load are independent. Exercise
independent loads under the race detector without sharing an active context.
Acceptance: roadmap criteria 2–8 and 11 are covered at their owning boundaries.
No duplicate provenance model, nearest-ancestor fallback, or source-order heuristic
is introduced. Any discovered defect is fixed before the stage is marked complete.
Validation: `go test . -run 'Resolution|Provenance|Import|Profile|Override|Metadata|Path'`,
`go test -race ./...`, then the everyday checker.
## Stage 4 — Demonstrate source-aware normalization and safe inspection
Status: not started. Dependencies: Stage 3.
Scope: maintained Notarius-shaped example code/fixtures, executable examples,
explanation/secret integration tests, and consumer guidance.
1. Add a small application-owned path-bearing configuration example in
`examples/consumers`. Keep existing catalog/version examples working. Supply the
invocation directory explicitly to the example loader rather than calling
`os.Chdir` or adding process-global state to tests.
2. Implement the example's policy using the two public queries: absolute paths
unchanged; CLI-relative paths based on the supplied invocation directory;
default-relative paths based on the canonical main source; file-authored paths
based on the scalar's declaring source. Use the existing primary-source identity
convention for main-source lookup, not a guess about contributor ordering.
3. For this file-rooted example, require one unambiguous canonical declaring file
for file-relative values. Report a safe application resolution error if that
prerequisite is absent or ambiguous. Do not silently use a diagnostic label or
working directory. Non-file-root fallback remains an explicitly separate
application policy, not a new library option.
4. Record every changed path through `Normalize` or `Record`, including valid
dependencies where derived values depend on other configuration paths. Prove
runtime values, effective YAML, and explanation agree; retain the original
relative declaration and normalization reason in history. No existence or
confinement checks are part of the example.
5. Add a symlink fixture proving canonical target-directory semantics and an
invocation-directory mismatch proving resolution does not depend on process CWD.
Make unsupported symlink capability explicit under the testing policy.
6. Use synthetic sensitive fields to verify original/effective redaction remains
intact. With existing reader/secret lookup seams, prove queries do not reread
source data or resolve credentials during the resolver; normal later lookup
remains unchanged. Do not create credentials or environment-dependent tests.
7. Document the public workflow and canonical symlink choice in the consumer guide;
point to executable artifacts instead of duplicating large code examples.
Acceptance: roadmap criteria 1, 9, 10, and 12 are demonstrated in one-load public
workflows. Filesystem policy stays in the example application, not the root library.
Validation: `go test ./examples/consumers`,
`go test . -run 'Resolution|Explain|Secret|Example'`, then the everyday checker.
## Stage 5 — Audit the complete contract and hand off release readiness
Status: not started. Dependencies: Stage 4.
Scope: integration review, canonical documentation, roadmap completion records,
and proportionate fixes for remaining acceptance failures. No feature expansion.
1. Check all twelve roadmap acceptance criteria against named tests and current
implementation. Record the mapping in this stage's completion entry. Passing
broad suites alone is not evidence that a missing scenario was exercised.
2. Inspect every new query path for environment/filesystem access, raw values,
mutable aliases, accidental callback-lifetime extension, and per-query whole-tree
work. Confirm no history was duplicated solely for these queries and loads
without a resolver avoid constructing an unused entry snapshot.
3. Review public signatures and current configuration behavior against the previous
release. Confirm the capability is additive and no parser, path, composition,
credential, or existing callback semantics changed unintentionally.
4. Update the internal overview and finish canonical Go/configuration/consumer
documentation. Transfer implemented contract prose from the feature roadmap
to its proper owners and retain a brief completion record with links. Preserve
historical ADR rationale and published notes. No new ADR is required for this
information-access extension unless implementation actually changes a governing
decision; stop and obtain review rather than silently widening scope.
5. Run the full source checker with qualifying acquired toolchains, recording the
actual versions and host. The following are the repository's recorded versions;
follow the release runbook if selecting updated toolchains:
```sh
CONFIGKIT_BASELINE_TOOLCHAIN=go1.25.5 \
CONFIGKIT_NEWER_TOOLCHAIN=go1.26.7 \
./scripts/release-check.sh
```
6. Hand off remaining publication work separately: maintainer version choice,
dependency reassessment, version-matched release notes, completed-roadmap
retirement and link repair, exact-commit Linux/macOS validation, publication
guards, and clean consumer retrieval. Follow [the runbook](../release.md).
Historical one-release exceptions do not carry forward to this feature.
Acceptance: all target requirements have concrete evidence; available automated
checks pass; documentation distinguishes implemented behavior from release status.
Implementation completion is not publication approval. Do not create a release
tag or mark unavailable native macOS evidence as passed.
Validation: `./scripts/check.sh`, `go test -race ./...`, and the full checker above.
## Open Questions
None blocking implementation. The API, snapshot boundary, missing-value semantics,
source identity handling, ownership, and example fallback policy are specified by
the roadmap and decisions above. Release version and publication timing are later
maintainer decisions, not unresolved implementation choices. If new evidence
contradicts a target contract, document the conflict, a recommended correction and
its rationale, and at least one viable alternative before proceeding past it.