Complete resolution provenance readiness audit

This commit is contained in:
2026-09-12 13:25:38 +00:00
parent b9e4dfac92
commit 715364b079
6 changed files with 94 additions and 239 deletions

11
doc.go
View File

@@ -1,10 +1,9 @@
// Package configkit provides deterministic, application-owned configuration // Package configkit provides deterministic, application-owned configuration
// loading for Go programs. // loading for Go programs.
// //
// Load currently supports one file, byte, reader, or explicit no-input source with // Load supports file, byte, reader, and explicit no-input sources with strict YAML,
// strict YAML, typed defaults, schema-version checks, detached metadata, safe // typed defaults, imports, profiles, CLI overrides, schema-version checks, detached
// diagnostics, recorded application resolution, read-only semantic validation, and // metadata, recorded application resolution, typed secret references, safe effective
// structured load errors. Composition, profiles, overrides, secrets, and effective // rendering, read-only semantic validation, and structured load errors. The
// rendering are not available yet. The configuration contract is documented in // configuration contract is documented in docs/config.md.
// docs/config.md.
package configkit package configkit

View File

@@ -46,9 +46,10 @@ The completed string-and-integer schema-version implementation and remediation
records are preserved in Git history at `244104f`. The records are preserved in Git history at `244104f`. The
[configuration contract](config.md#application-schema-versions) owns current behavior. [configuration contract](config.md#application-schema-versions) owns current behavior.
Current implementation work: [source origins during resolution](roadmap/source-origins-during-resolution.md) The completed [source-origins during resolution record](roadmap/source-origins-during-resolution.md)
and its [staged plan](roadmap/implementation.md). The roadmap defines target links to the current contract and evidence. Its staged implementation handoffs remain
behavior; the implementation plan records progress and handoffs. in [implementation.md](roadmap/implementation.md) until release preparation retires
completed roadmap material.
## Baseline validation ## Baseline validation

View File

@@ -40,8 +40,9 @@ representations private. The implemented components and their owners are:
contexts in `validation.go`, and layered metadata/history assembly in contexts in `validation.go`, and layered metadata/history assembly in
`load_metadata.go`; `load_metadata.go`;
- recorded application defaulting and normalization in `resolution.go`, including - recorded application defaulting and normalization in `resolution.go`, including
explicitness queries, helper verification, structural mutation audit, coalesced callback-scoped fixed entry metadata/source queries, explicitness queries, helper
intent, private before/after snapshots, and resolution metadata integration; verification, structural mutation audit, coalesced intent, private before/after
snapshots, and resolution metadata integration;
- post-resolution schema-aware report collection and validator mutation detection in - post-resolution schema-aware report collection and validator mutation detection in
`validation.go`, completing the public callback lifecycle after composed input; `validation.go`, completing the public callback lifecycle after composed input;
- opaque typed environment references and per-load name resolution in - opaque typed environment references and per-load name resolution in

View File

@@ -1,6 +1,6 @@
# Source origins during resolution: implementation plan # Source origins during resolution: implementation plan
Status: not started. Implement stages in order, starting with Stage 1. Status: implementation complete (2026-09-12). All stages were completed in order.
The [feature roadmap](source-origins-during-resolution.md) owns the target contract The [feature roadmap](source-origins-during-resolution.md) owns the target contract
and consumer intent. This document owns implementation steps, validation, and and consumer intent. This document owns implementation steps, validation, and
handoffs. It is written for one gpt-5.6-terra coding agent per stage; no stage handoffs. It is written for one gpt-5.6-terra coding agent per stage; no stage
@@ -276,7 +276,7 @@ with Go 1.26.7 on Linux. No deviations or remaining Stage 4 risks were identifie
## Stage 5 — Audit the complete contract and hand off release readiness ## Stage 5 — Audit the complete contract and hand off release readiness
Status: not started. Dependencies: Stage 4. Status: complete (2026-09-12). Dependencies: Stage 4.
Scope: integration review, canonical documentation, roadmap completion records, Scope: integration review, canonical documentation, roadmap completion records,
and proportionate fixes for remaining acceptance failures. No feature expansion. and proportionate fixes for remaining acceptance failures. No feature expansion.
@@ -321,6 +321,49 @@ tag or mark unavailable native macOS evidence as passed.
Validation: `./scripts/check.sh`, `go test -race ./...`, and the full checker above. Validation: `./scripts/check.sh`, `go test -race ./...`, and the full checker above.
Completion record (2026-09-12): all acceptance criteria have named evidence:
1. `TestNotariusPathExampleUsesEntryProvenance` covers canonical main-file and
symlink target bases without process-CWD dependence.
2. `TestResolutionInputKeepsLayeredOriginsAndSources` covers imported scalars and
sequence elements.
3. The same layered test covers profile replacement and untouched base/import
siblings.
4. The layered test covers equal-text profile and CLI replacement.
5. `TestResolutionEntryQueriesExposeDetachedLoadInput` and
`TestResolutionInputDistinguishesNonFileRootsAndImportedFiles` cover defaults,
nulls, misses, CLI/file identities, `NoSource`, bytes, and readers.
6. The layered test verifies every composite location has a source;
`TestResolutionEntryQueriesExposeDetachedLoadInput` verifies unknown IDs miss
safely.
7. `TestResolutionInputRemainsFixedAcrossRecordedMutations` covers normalization,
deletion, addition, and move stability.
8. `TestResolutionInputUsesEntryShapesAcrossTypedAndDynamicValues` covers inline,
map, array, dynamic, escaped/signed keys, and invalid shape cases.
9. `TestNotariusPathExampleUsesEntryProvenance` verifies runtime, effective YAML,
and explanation history together.
10. `TestResolutionInputQueriesDoNotReadOrResolveSecrets` plus the Notarius test
verify no extra reader/credential access and preserved redaction.
11. `TestResolutionInputKeepsLayeredOriginsAndSources` and
`TestResolutionEntryQueriesExposeDetachedLoadInput` verify detached records.
12. `NotariusPathConfig`, its fixtures, executable example, and consumer-guide
link provide the maintained public-API/standard-library workflow.
The final review found that queries use only the callback guard and detached
metadata/source indexes; they do not extend callback lifetime, expose raw values, or
perform environment/filesystem work. The view is created only for resolver loads and
does not retain history. The comparison with `v0.2.0` found only additive exported
methods; loading, parser, path, composition, credential, and existing callback
semantics remain unchanged. Canonical Go, configuration, consumer, internal, and
roadmap documentation are synchronized; the feature roadmap is retained only until
release preparation. `./scripts/check.sh`, `go test -race ./...`, documentation and
import-boundary checks, and `CONFIGKIT_BASELINE_TOOLCHAIN=go1.25.5
CONFIGKIT_NEWER_TOOLCHAIN=go1.26.7 ./scripts/release-check.sh` passed on Linux with
Go 1.25.5 and Go 1.26.7. Publication remains unapproved: a maintainer must choose a
pre-v1 minor version, write its note, retire completed roadmaps, obtain exact
candidate native Linux/macOS evidence, and follow the release procedure. No version,
release note, candidate, tag, or push was created.
## Open Questions ## Open Questions
None blocking implementation. The API, snapshot boundary, missing-value semantics, None blocking implementation. The API, snapshot boundary, missing-value semantics,

View File

@@ -1,241 +1,50 @@
# Source origins during resolution # Source origins during resolution
Status: planned; not implemented. This roadmap replaces the Notarius feature Status: implemented (2026-09-12); release preparation remains separate.
request and owns the targeted end state. The [implementation plan](implementation.md)
owns execution order, progress, and handoffs.
This document does not authorize implementation or select a release version.
Current behavior remains defined by the [configuration contract](../config.md). ConfigKit now exposes fixed entry-state provenance during an active
The [architecture policy](../policy/architecture.md) governs responsibility `Definition.Resolve` callback:
boundaries, and the [documentation policy](../policy/documentation.md) governs
contract ownership and retirement of completed plans.
## Motivation
Notarius needs source-aware normalization of configured filesystem paths.
For example, `../references/party.txt` declared in `pipelines/dnd.yml` should
resolve relative to that declaring file, rather than the process working directory.
A profile replacement should use the profile file's directory even when it supplies
identical text; an untouched sibling should retain its original declaration.
Notarius separately chooses invocation-relative CLI paths, main-configuration-
relative defaults, and unchanged absolute paths. These are application policies,
not ConfigKit conventions. ConfigKit must provide the provenance needed to apply
them in one load without independently parsing YAML or reading files again.
Today, `Definition.Resolve` can inspect explicitness and record normalization, but
source-origin queries are available only after loading. Changing `Result.Config`
then would not change its frozen effective output or explanation. The target is to
make provenance available before normalization, while keeping runtime values and
inspection consistent through existing change recording.
## Target public surface
Add the following methods to the existing callback-scoped `Resolution` context:
```go ```go
func (r *Resolution) InputInfo(path Path) (FieldInfo, bool) func (r *Resolution) InputInfo(path Path) (FieldInfo, bool)
func (r *Resolution) Source(id string) (SourceInfo, bool) func (r *Resolution) Source(id string) (SourceInfo, bool)
``` ```
These signatures are the proposed target API, not callable APIs in the current These methods support application-owned normalization and other domain decisions
release. Reuse existing public `Path`, `FieldInfo`, `Origin`, `Location`, and before ConfigKit publishes its frozen result. They are additive: existing loading,
`SourceInfo` types. Do not introduce a second provenance record hierarchy, a composition, decoding, resolution recording, credential timing, and inspection
generic node API, or a new callback. semantics remain unchanged.
### InputInfo: a fixed pre-resolution view ## Canonical behavior
Despite its short name, `InputInfo` describes the **decoded configuration at entry Go reference comments on `Resolution` own the API/lifetime definition. The
to resolution**, including retained application defaults. It is not limited to [configuration contract](../config.md#recorded-application-resolution) owns entry
literal YAML nodes and is not a live view of callback mutations. state, exact path matching, source records, misses, detachment, and callback-scope
behavior. The [consumer adoption guide](../consumers/README.md#source-aware-application-path-policy)
and maintained [`NotariusPathConfig` example](../../examples/consumers/consumers.go)
own the application workflow, including its canonical symlink-base choice.
Its snapshot is taken after imports/composition, selected-profile application, `InputInfo` describes decoded configuration at callback entry, including retained
CLI overrides, defaults, and typed decoding, but before application resolution defaults and explicit nulls, and is unaffected by subsequent application changes.
and typed credential lookup. The query returns the same presence/origin meaning `Source` resolves records already acquired for that load without filesystem,
as effective metadata at that boundary: credential, or parsing work. Both return detached records and require the existing
active callback context.
| Path state at callback entry | Result | ConfigKit continues to provide provenance rather than filesystem policy. Consumers
| --- | --- | choose how to interpret relative paths, select default and CLI bases, and handle
| Surviving file, profile, or CLI contribution | Found; existing explicit/null and origin information | missing or composite file provenance. The maintained example leaves values unchanged
| Present default-only value | Found; non-explicit, default origin, no invented file location | with a warning when its chosen base is unavailable; it performs no existence or
| Explicit null accepted by the schema | Found; explicit and null | confinement check.
| Missing path or descendant removed before decoding | Not found unless ordinary decoding retained/reintroduced it through defaults |
| Invalid path or key/index shape mismatch | Not found |
A Go zero-valued field that exists in the decoded projection is not a missing ## Evidence and handoff
path. Zero metadata fields alone must not be used to infer absence; callers use
the returned boolean. Invalid/missing queries add no diagnostics.
Queries address exact semantic paths through nested structs, inlined fields, The completed staged record in [implementation.md](implementation.md) maps all
mapping entries, arrays, and sequence elements. Use existing path rules, including twelve acceptance criteria to focused lifecycle, provenance, ownership, consumer,
escaped keys, explicit key/index distinctions, and canonical contextual sequence explanation, and secret tests. The final implementation commit is expected to run
indices. Never substitute nearest-ancestor diagnostic lookup for exact field lookup. the source checker and the documented two-toolchain gate before any release
candidate is selected.
For a scalar, the origin identifies its surviving declaration, not the contributors Publication remains a maintainer decision. Release preparation must choose the
of its parent mapping. Equal-valued profile and CLI replacements retain the version, write a version-matched note, retire this completed roadmap with link
replacing origin. Containers retain their existing possibly composite locations; repair, obtain exact-candidate Linux/macOS evidence, and follow the
the API does not select a representative owner or single declaring directory. [release procedure](../release.md).
`InputInfo` and `IsExplicit` must agree for paths present in the entry snapshot.
The extension must not change existing `IsExplicit` behavior for paths outside
that view. Normalization records, additions, removals, and moves do not alter the
snapshot. A removed original path stays queryable; a new destination that did not
exist at entry remains absent from this view. If a destination already existed,
its query describes that original value, not the moved value.
### Source: resolve identities without filesystem work
`Source(id)` returns the existing source record for an ID belonging to this load.
Empty or unknown IDs return a zero record and false without adding diagnostics.
The source view contains the records already acquired before resolution, including
main input, imported files, the selected profile, and CLI records where applicable.
It does not discover files, reload input, or inspect unselected profiles.
A returned `SourceInfo.Path` is the canonical local path only when the existing
source model supplies one. File paths retain current canonicalization and symlink
semantics. Labels remain diagnostic labels, never a substitute for paths.
Bytes/readers and CLI records have no fabricated file path, even if a filesystem
context was supplied for imports or profiles. Defaults have no source record or
synthetic file identity.
Every source identity referenced by the entry view's origin locations must be
resolvable through this source view. Existing fields distinguish layers and import
relationships; do not require callers to guess scalar ownership from source order.
The existing main-source identity convention remains unchanged. Applications may
use the main source record for their own default-base policy; a non-file main
source still supplies no directory, and the application must choose its fallback.
Multiple locations do not imply one declaring file. Consumers needing a single
file must inspect the exact value's origin and source records and handle ambiguity
according to application policy. ConfigKit does not require rejection: an
application may choose an explicit fallback, retain a relative value, warn, or
report an error. No new `DeclaringFile` shortcut should silently discard contributors.
### Lifetime and detachment
Both methods require an active resolution callback. Nil, zero, or retained contexts
used after callback closure follow the existing fixed, value-free panic convention,
including when a callback returns an error. These methods do not make the context
safe for concurrent use.
Returned records are detached, including nested slices and `ImportedFrom`
locations. They may be retained after callback exit; modifying them must not affect
subsequent queries, other loads, final metadata, or explanations. The context's
lifetime restriction does not apply to those copied records.
## Consumer normalization workflow
The intended Notarius workflow is:
1. Query a path's entry-state field information before moving or normalizing it.
2. Resolve its origin source IDs and select a base using application policy.
3. Compute the application path without asking ConfigKit to interpret it.
4. Assign and record the result using existing `Normalize` or `Resolution.Record`
semantics, including dependencies where appropriate.
5. Use the completed result for both execution and configuration inspection.
Consumers may retain original origin records for later dependent work in the same
callback. ConfigKit does not infer ownership transfers, rewrite the input view, or
automatically record a move. Existing mutation auditing and record validation
continue to apply to all changes.
For canonical file sources, a symlink refers to the resolved target file's
directory, not necessarily the directory containing the link used to open it.
The maintained example must make this explicit. Applications wanting another
path-base policy own that policy and the additional information it requires.
## Intended internal architecture
Keep one provenance model and one loading lifecycle. The existing
`resolveLoadInput` in `load_result.go` already assembles decoded metadata and
overlay history before calling `runResolution`. Pass a read-only entry-state
provenance view and the load's acquired source records into the private resolution
state. No public parser types or new external dependencies are needed.
The view must support complete effective semantic-path lookup when the callback
starts. Current result metadata indexes are finalized later; exposing an unfinished
`Metadata` value would make legitimate lookups fail. Reuse or factor existing
effective-path traversal and shape matching so entry queries and final queries
cannot acquire different path semantics. A full historical index or public
`Metadata.History` interface during resolution is not required by this feature.
Keep queries stable across consumer normalization and return detached records.
Private copying, immutable sharing, or ownership transfer may satisfy that contract;
no particular backing representation or eager-copy strategy is required. Prefer
existing traversal/indexing machinery and proportionate allocation. A per-load
source index is a recommended mechanism, not a public complexity guarantee or a
prohibition on scanning the bounded existing source collection. Consumers should
not need to rebuild source bookkeeping themselves.
The source and input views remain bounded by existing load limits. They contain
provenance records, not configuration values or resolved credentials. Do not read
the environment, resolve secret references, perform filesystem I/O, or invoke
application codecs in response to a query.
Existing composition precedence, canonical decoding, normalization auditing,
resolution origins/history, redaction, and frozen result publication remain
authoritative. This is an information-access extension, not a new normalization
engine or permission to mutate post-load snapshots.
## Acceptance criteria
The completed feature must demonstrate the following through public API behavior,
using real temporary files where source identity matters:
1. A main-file scalar can be normalized relative to its canonical declaring file
during resolution, independent of invocation directory.
2. An imported scalar, including a sequence element, identifies its own declaring
file rather than the main file or a parent container's contributors.
3. A profile replacement identifies the profile source while untouched siblings
retain their base/import origins.
4. Equal-text profile and CLI replacements still identify the replacing source.
5. Defaults, missing paths, explicit null, CLI input, and file input are distinct;
`NoSource` and non-file input introduce no fictional declaring paths.
6. Composite containers retain all contributing locations without selecting a file.
Every referenced source ID resolves, and unknown IDs are harmless misses.
7. Queries remain fixed across normalization, deletion, addition, and moves;
retained copies remain useful and detached.
8. Exact lookup works through typed and dynamic structures, escaped mapping keys,
and sequence indices; invalid paths and shape mismatches do not borrow ancestor
or sibling provenance.
9. A successful single load yields identical runtime and effective values, with
explanation preserving original declarations and recorded normalization reasons.
10. Querying origins performs no extra load or credential lookup, and existing
secret redaction, safe diagnostics, and callback-lifetime rules remain intact.
11. Mutating returned records cannot affect later queries or final results.
12. Maintained Notarius-shaped examples demonstrate the policy using only the
public API and standard library, including documented canonical/symlink behavior.
Tests should extend existing provenance, resolution, ownership, and consumer
behavioral owners rather than duplicate every mechanism at every layer. Acceptance
does not require a live Notarius checkout, external credentials, or target-file
existence checks.
## Scope and existing boundaries
Application responsibilities remain application responsibilities: interpreting
path-valued strings, normalizing filesystem paths, choosing default/CLI bases,
and deciding whether to check existence, impose confinement, or use fallbacks.
This feature neither performs those operations nor prohibits consumers from doing so.
Actual Notarius integration remains downstream work.
Existing contracts, including import/profile composition and frozen post-load
inspection, remain unchanged. Their authority comes from the current contracts,
not from this roadmap's exclusions.
Source discovery, historical value access during resolution, automatic move
tracking, and a general provenance service are not included in this feature.
That scope choice is not a repository-wide prohibition on future proposals.
## Compatibility and canonical ownership
This is an additive public capability; existing callbacks, loading order, and
configuration semantics must remain unchanged. Release selection and validation
follow the [release policy](../policy/release.md) and
[release procedure](../release.md). No release gate is waived here.
In the completed feature, Go reference comments own API and lifetime definitions,
the configuration contract owns shared resolution/provenance semantics, and the
adoption guide and executable examples own consumer usage. Component documentation
describes the resulting internal boundaries. Accepted ADR rationale remains
historical; changes to governing decisions follow the documented supersession policy.

View File

@@ -322,7 +322,7 @@ func TestResolutionInputUsesEntryShapesAcrossTypedAndDynamicValues(t *testing.T)
Dynamic any `yaml:"dynamic"` Dynamic any `yaml:"dynamic"`
} }
var callbackErr error var callbackErr error
_, err := Load[config](Bytes("shapes.yml", []byte("name: inlined\nmappings: {'-0': key}\narray: [zero, one]\ndynamic: {nested: [dynamic]}\n")), Options[config]{Definition: Definition[config]{Resolve: func(_ *config, resolution *Resolution) error { _, err := Load[config](Bytes("shapes.yml", []byte("name: inlined\nmappings: {'-0': key, 'a/b~': escaped}\narray: [zero, one]\ndynamic: {nested: [dynamic]}\n")), Options[config]{Definition: Definition[config]{Resolve: func(_ *config, resolution *Resolution) error {
check := func(path Path, want bool) { check := func(path Path, want bool) {
_, found := resolution.InputInfo(path) _, found := resolution.InputInfo(path)
if found != want && callbackErr == nil { if found != want && callbackErr == nil {
@@ -330,8 +330,10 @@ func TestResolutionInputUsesEntryShapesAcrossTypedAndDynamicValues(t *testing.T)
} }
} }
parsedMapping, _ := ParsePath("/mappings/-0") parsedMapping, _ := ParsePath("/mappings/-0")
escapedMapping, _ := ParsePath("/mappings/a~1b~0")
check(Keys("name"), true) check(Keys("name"), true)
check(parsedMapping, true) check(parsedMapping, true)
check(escapedMapping, true)
check(Keys("mappings").Index(0), false) check(Keys("mappings").Index(0), false)
check(Keys("array").Index(1), true) check(Keys("array").Index(1), true)
check(Keys("array", "1"), false) check(Keys("array", "1"), false)