Complete resolution provenance readiness audit
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# Source origins during resolution: implementation plan
|
||||
|
||||
Status: not started. Implement stages in order, starting with Stage 1.
|
||||
Status: implementation complete (2026-09-12). All stages were completed in order.
|
||||
The [feature roadmap](source-origins-during-resolution.md) owns the target contract
|
||||
and consumer intent. This document owns implementation steps, validation, and
|
||||
handoffs. It is written for one gpt-5.6-terra coding agent per stage; no stage
|
||||
@@ -276,7 +276,7 @@ with Go 1.26.7 on Linux. No deviations or remaining Stage 4 risks were identifie
|
||||
|
||||
## Stage 5 — Audit the complete contract and hand off release readiness
|
||||
|
||||
Status: not started. Dependencies: Stage 4.
|
||||
Status: complete (2026-09-12). Dependencies: Stage 4.
|
||||
|
||||
Scope: integration review, canonical documentation, roadmap completion records,
|
||||
and proportionate fixes for remaining acceptance failures. No feature expansion.
|
||||
@@ -321,6 +321,49 @@ tag or mark unavailable native macOS evidence as passed.
|
||||
|
||||
Validation: `./scripts/check.sh`, `go test -race ./...`, and the full checker above.
|
||||
|
||||
Completion record (2026-09-12): all acceptance criteria have named evidence:
|
||||
|
||||
1. `TestNotariusPathExampleUsesEntryProvenance` covers canonical main-file and
|
||||
symlink target bases without process-CWD dependence.
|
||||
2. `TestResolutionInputKeepsLayeredOriginsAndSources` covers imported scalars and
|
||||
sequence elements.
|
||||
3. The same layered test covers profile replacement and untouched base/import
|
||||
siblings.
|
||||
4. The layered test covers equal-text profile and CLI replacement.
|
||||
5. `TestResolutionEntryQueriesExposeDetachedLoadInput` and
|
||||
`TestResolutionInputDistinguishesNonFileRootsAndImportedFiles` cover defaults,
|
||||
nulls, misses, CLI/file identities, `NoSource`, bytes, and readers.
|
||||
6. The layered test verifies every composite location has a source;
|
||||
`TestResolutionEntryQueriesExposeDetachedLoadInput` verifies unknown IDs miss
|
||||
safely.
|
||||
7. `TestResolutionInputRemainsFixedAcrossRecordedMutations` covers normalization,
|
||||
deletion, addition, and move stability.
|
||||
8. `TestResolutionInputUsesEntryShapesAcrossTypedAndDynamicValues` covers inline,
|
||||
map, array, dynamic, escaped/signed keys, and invalid shape cases.
|
||||
9. `TestNotariusPathExampleUsesEntryProvenance` verifies runtime, effective YAML,
|
||||
and explanation history together.
|
||||
10. `TestResolutionInputQueriesDoNotReadOrResolveSecrets` plus the Notarius test
|
||||
verify no extra reader/credential access and preserved redaction.
|
||||
11. `TestResolutionInputKeepsLayeredOriginsAndSources` and
|
||||
`TestResolutionEntryQueriesExposeDetachedLoadInput` verify detached records.
|
||||
12. `NotariusPathConfig`, its fixtures, executable example, and consumer-guide
|
||||
link provide the maintained public-API/standard-library workflow.
|
||||
|
||||
The final review found that queries use only the callback guard and detached
|
||||
metadata/source indexes; they do not extend callback lifetime, expose raw values, or
|
||||
perform environment/filesystem work. The view is created only for resolver loads and
|
||||
does not retain history. The comparison with `v0.2.0` found only additive exported
|
||||
methods; loading, parser, path, composition, credential, and existing callback
|
||||
semantics remain unchanged. Canonical Go, configuration, consumer, internal, and
|
||||
roadmap documentation are synchronized; the feature roadmap is retained only until
|
||||
release preparation. `./scripts/check.sh`, `go test -race ./...`, documentation and
|
||||
import-boundary checks, and `CONFIGKIT_BASELINE_TOOLCHAIN=go1.25.5
|
||||
CONFIGKIT_NEWER_TOOLCHAIN=go1.26.7 ./scripts/release-check.sh` passed on Linux with
|
||||
Go 1.25.5 and Go 1.26.7. Publication remains unapproved: a maintainer must choose a
|
||||
pre-v1 minor version, write its note, retire completed roadmaps, obtain exact
|
||||
candidate native Linux/macOS evidence, and follow the release procedure. No version,
|
||||
release note, candidate, tag, or push was created.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None blocking implementation. The API, snapshot boundary, missing-value semantics,
|
||||
|
||||
@@ -1,241 +1,50 @@
|
||||
# Source origins during resolution
|
||||
|
||||
Status: planned; not implemented. This roadmap replaces the Notarius feature
|
||||
request and owns the targeted end state. The [implementation plan](implementation.md)
|
||||
owns execution order, progress, and handoffs.
|
||||
This document does not authorize implementation or select a release version.
|
||||
Status: implemented (2026-09-12); release preparation remains separate.
|
||||
|
||||
Current behavior remains defined by the [configuration contract](../config.md).
|
||||
The [architecture policy](../policy/architecture.md) governs responsibility
|
||||
boundaries, and the [documentation policy](../policy/documentation.md) governs
|
||||
contract ownership and retirement of completed plans.
|
||||
|
||||
## Motivation
|
||||
|
||||
Notarius needs source-aware normalization of configured filesystem paths.
|
||||
For example, `../references/party.txt` declared in `pipelines/dnd.yml` should
|
||||
resolve relative to that declaring file, rather than the process working directory.
|
||||
A profile replacement should use the profile file's directory even when it supplies
|
||||
identical text; an untouched sibling should retain its original declaration.
|
||||
|
||||
Notarius separately chooses invocation-relative CLI paths, main-configuration-
|
||||
relative defaults, and unchanged absolute paths. These are application policies,
|
||||
not ConfigKit conventions. ConfigKit must provide the provenance needed to apply
|
||||
them in one load without independently parsing YAML or reading files again.
|
||||
|
||||
Today, `Definition.Resolve` can inspect explicitness and record normalization, but
|
||||
source-origin queries are available only after loading. Changing `Result.Config`
|
||||
then would not change its frozen effective output or explanation. The target is to
|
||||
make provenance available before normalization, while keeping runtime values and
|
||||
inspection consistent through existing change recording.
|
||||
|
||||
## Target public surface
|
||||
|
||||
Add the following methods to the existing callback-scoped `Resolution` context:
|
||||
ConfigKit now exposes fixed entry-state provenance during an active
|
||||
`Definition.Resolve` callback:
|
||||
|
||||
```go
|
||||
func (r *Resolution) InputInfo(path Path) (FieldInfo, bool)
|
||||
func (r *Resolution) Source(id string) (SourceInfo, bool)
|
||||
```
|
||||
|
||||
These signatures are the proposed target API, not callable APIs in the current
|
||||
release. Reuse existing public `Path`, `FieldInfo`, `Origin`, `Location`, and
|
||||
`SourceInfo` types. Do not introduce a second provenance record hierarchy, a
|
||||
generic node API, or a new callback.
|
||||
These methods support application-owned normalization and other domain decisions
|
||||
before ConfigKit publishes its frozen result. They are additive: existing loading,
|
||||
composition, decoding, resolution recording, credential timing, and inspection
|
||||
semantics remain unchanged.
|
||||
|
||||
### InputInfo: a fixed pre-resolution view
|
||||
## Canonical behavior
|
||||
|
||||
Despite its short name, `InputInfo` describes the **decoded configuration at entry
|
||||
to resolution**, including retained application defaults. It is not limited to
|
||||
literal YAML nodes and is not a live view of callback mutations.
|
||||
Go reference comments on `Resolution` own the API/lifetime definition. The
|
||||
[configuration contract](../config.md#recorded-application-resolution) owns entry
|
||||
state, exact path matching, source records, misses, detachment, and callback-scope
|
||||
behavior. The [consumer adoption guide](../consumers/README.md#source-aware-application-path-policy)
|
||||
and maintained [`NotariusPathConfig` example](../../examples/consumers/consumers.go)
|
||||
own the application workflow, including its canonical symlink-base choice.
|
||||
|
||||
Its snapshot is taken after imports/composition, selected-profile application,
|
||||
CLI overrides, defaults, and typed decoding, but before application resolution
|
||||
and typed credential lookup. The query returns the same presence/origin meaning
|
||||
as effective metadata at that boundary:
|
||||
`InputInfo` describes decoded configuration at callback entry, including retained
|
||||
defaults and explicit nulls, and is unaffected by subsequent application changes.
|
||||
`Source` resolves records already acquired for that load without filesystem,
|
||||
credential, or parsing work. Both return detached records and require the existing
|
||||
active callback context.
|
||||
|
||||
| Path state at callback entry | Result |
|
||||
| --- | --- |
|
||||
| Surviving file, profile, or CLI contribution | Found; existing explicit/null and origin information |
|
||||
| Present default-only value | Found; non-explicit, default origin, no invented file location |
|
||||
| Explicit null accepted by the schema | Found; explicit and null |
|
||||
| Missing path or descendant removed before decoding | Not found unless ordinary decoding retained/reintroduced it through defaults |
|
||||
| Invalid path or key/index shape mismatch | Not found |
|
||||
ConfigKit continues to provide provenance rather than filesystem policy. Consumers
|
||||
choose how to interpret relative paths, select default and CLI bases, and handle
|
||||
missing or composite file provenance. The maintained example leaves values unchanged
|
||||
with a warning when its chosen base is unavailable; it performs no existence or
|
||||
confinement check.
|
||||
|
||||
A Go zero-valued field that exists in the decoded projection is not a missing
|
||||
path. Zero metadata fields alone must not be used to infer absence; callers use
|
||||
the returned boolean. Invalid/missing queries add no diagnostics.
|
||||
## Evidence and handoff
|
||||
|
||||
Queries address exact semantic paths through nested structs, inlined fields,
|
||||
mapping entries, arrays, and sequence elements. Use existing path rules, including
|
||||
escaped keys, explicit key/index distinctions, and canonical contextual sequence
|
||||
indices. Never substitute nearest-ancestor diagnostic lookup for exact field lookup.
|
||||
The completed staged record in [implementation.md](implementation.md) maps all
|
||||
twelve acceptance criteria to focused lifecycle, provenance, ownership, consumer,
|
||||
explanation, and secret tests. The final implementation commit is expected to run
|
||||
the source checker and the documented two-toolchain gate before any release
|
||||
candidate is selected.
|
||||
|
||||
For a scalar, the origin identifies its surviving declaration, not the contributors
|
||||
of its parent mapping. Equal-valued profile and CLI replacements retain the
|
||||
replacing origin. Containers retain their existing possibly composite locations;
|
||||
the API does not select a representative owner or single declaring directory.
|
||||
|
||||
`InputInfo` and `IsExplicit` must agree for paths present in the entry snapshot.
|
||||
The extension must not change existing `IsExplicit` behavior for paths outside
|
||||
that view. Normalization records, additions, removals, and moves do not alter the
|
||||
snapshot. A removed original path stays queryable; a new destination that did not
|
||||
exist at entry remains absent from this view. If a destination already existed,
|
||||
its query describes that original value, not the moved value.
|
||||
|
||||
### Source: resolve identities without filesystem work
|
||||
|
||||
`Source(id)` returns the existing source record for an ID belonging to this load.
|
||||
Empty or unknown IDs return a zero record and false without adding diagnostics.
|
||||
The source view contains the records already acquired before resolution, including
|
||||
main input, imported files, the selected profile, and CLI records where applicable.
|
||||
It does not discover files, reload input, or inspect unselected profiles.
|
||||
|
||||
A returned `SourceInfo.Path` is the canonical local path only when the existing
|
||||
source model supplies one. File paths retain current canonicalization and symlink
|
||||
semantics. Labels remain diagnostic labels, never a substitute for paths.
|
||||
Bytes/readers and CLI records have no fabricated file path, even if a filesystem
|
||||
context was supplied for imports or profiles. Defaults have no source record or
|
||||
synthetic file identity.
|
||||
|
||||
Every source identity referenced by the entry view's origin locations must be
|
||||
resolvable through this source view. Existing fields distinguish layers and import
|
||||
relationships; do not require callers to guess scalar ownership from source order.
|
||||
The existing main-source identity convention remains unchanged. Applications may
|
||||
use the main source record for their own default-base policy; a non-file main
|
||||
source still supplies no directory, and the application must choose its fallback.
|
||||
|
||||
Multiple locations do not imply one declaring file. Consumers needing a single
|
||||
file must inspect the exact value's origin and source records and handle ambiguity
|
||||
according to application policy. ConfigKit does not require rejection: an
|
||||
application may choose an explicit fallback, retain a relative value, warn, or
|
||||
report an error. No new `DeclaringFile` shortcut should silently discard contributors.
|
||||
|
||||
### Lifetime and detachment
|
||||
|
||||
Both methods require an active resolution callback. Nil, zero, or retained contexts
|
||||
used after callback closure follow the existing fixed, value-free panic convention,
|
||||
including when a callback returns an error. These methods do not make the context
|
||||
safe for concurrent use.
|
||||
|
||||
Returned records are detached, including nested slices and `ImportedFrom`
|
||||
locations. They may be retained after callback exit; modifying them must not affect
|
||||
subsequent queries, other loads, final metadata, or explanations. The context's
|
||||
lifetime restriction does not apply to those copied records.
|
||||
|
||||
## Consumer normalization workflow
|
||||
|
||||
The intended Notarius workflow is:
|
||||
|
||||
1. Query a path's entry-state field information before moving or normalizing it.
|
||||
2. Resolve its origin source IDs and select a base using application policy.
|
||||
3. Compute the application path without asking ConfigKit to interpret it.
|
||||
4. Assign and record the result using existing `Normalize` or `Resolution.Record`
|
||||
semantics, including dependencies where appropriate.
|
||||
5. Use the completed result for both execution and configuration inspection.
|
||||
|
||||
Consumers may retain original origin records for later dependent work in the same
|
||||
callback. ConfigKit does not infer ownership transfers, rewrite the input view, or
|
||||
automatically record a move. Existing mutation auditing and record validation
|
||||
continue to apply to all changes.
|
||||
|
||||
For canonical file sources, a symlink refers to the resolved target file's
|
||||
directory, not necessarily the directory containing the link used to open it.
|
||||
The maintained example must make this explicit. Applications wanting another
|
||||
path-base policy own that policy and the additional information it requires.
|
||||
|
||||
## Intended internal architecture
|
||||
|
||||
Keep one provenance model and one loading lifecycle. The existing
|
||||
`resolveLoadInput` in `load_result.go` already assembles decoded metadata and
|
||||
overlay history before calling `runResolution`. Pass a read-only entry-state
|
||||
provenance view and the load's acquired source records into the private resolution
|
||||
state. No public parser types or new external dependencies are needed.
|
||||
|
||||
The view must support complete effective semantic-path lookup when the callback
|
||||
starts. Current result metadata indexes are finalized later; exposing an unfinished
|
||||
`Metadata` value would make legitimate lookups fail. Reuse or factor existing
|
||||
effective-path traversal and shape matching so entry queries and final queries
|
||||
cannot acquire different path semantics. A full historical index or public
|
||||
`Metadata.History` interface during resolution is not required by this feature.
|
||||
|
||||
Keep queries stable across consumer normalization and return detached records.
|
||||
Private copying, immutable sharing, or ownership transfer may satisfy that contract;
|
||||
no particular backing representation or eager-copy strategy is required. Prefer
|
||||
existing traversal/indexing machinery and proportionate allocation. A per-load
|
||||
source index is a recommended mechanism, not a public complexity guarantee or a
|
||||
prohibition on scanning the bounded existing source collection. Consumers should
|
||||
not need to rebuild source bookkeeping themselves.
|
||||
|
||||
The source and input views remain bounded by existing load limits. They contain
|
||||
provenance records, not configuration values or resolved credentials. Do not read
|
||||
the environment, resolve secret references, perform filesystem I/O, or invoke
|
||||
application codecs in response to a query.
|
||||
|
||||
Existing composition precedence, canonical decoding, normalization auditing,
|
||||
resolution origins/history, redaction, and frozen result publication remain
|
||||
authoritative. This is an information-access extension, not a new normalization
|
||||
engine or permission to mutate post-load snapshots.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
The completed feature must demonstrate the following through public API behavior,
|
||||
using real temporary files where source identity matters:
|
||||
|
||||
1. A main-file scalar can be normalized relative to its canonical declaring file
|
||||
during resolution, independent of invocation directory.
|
||||
2. An imported scalar, including a sequence element, identifies its own declaring
|
||||
file rather than the main file or a parent container's contributors.
|
||||
3. A profile replacement identifies the profile source while untouched siblings
|
||||
retain their base/import origins.
|
||||
4. Equal-text profile and CLI replacements still identify the replacing source.
|
||||
5. Defaults, missing paths, explicit null, CLI input, and file input are distinct;
|
||||
`NoSource` and non-file input introduce no fictional declaring paths.
|
||||
6. Composite containers retain all contributing locations without selecting a file.
|
||||
Every referenced source ID resolves, and unknown IDs are harmless misses.
|
||||
7. Queries remain fixed across normalization, deletion, addition, and moves;
|
||||
retained copies remain useful and detached.
|
||||
8. Exact lookup works through typed and dynamic structures, escaped mapping keys,
|
||||
and sequence indices; invalid paths and shape mismatches do not borrow ancestor
|
||||
or sibling provenance.
|
||||
9. A successful single load yields identical runtime and effective values, with
|
||||
explanation preserving original declarations and recorded normalization reasons.
|
||||
10. Querying origins performs no extra load or credential lookup, and existing
|
||||
secret redaction, safe diagnostics, and callback-lifetime rules remain intact.
|
||||
11. Mutating returned records cannot affect later queries or final results.
|
||||
12. Maintained Notarius-shaped examples demonstrate the policy using only the
|
||||
public API and standard library, including documented canonical/symlink behavior.
|
||||
|
||||
Tests should extend existing provenance, resolution, ownership, and consumer
|
||||
behavioral owners rather than duplicate every mechanism at every layer. Acceptance
|
||||
does not require a live Notarius checkout, external credentials, or target-file
|
||||
existence checks.
|
||||
|
||||
## Scope and existing boundaries
|
||||
|
||||
Application responsibilities remain application responsibilities: interpreting
|
||||
path-valued strings, normalizing filesystem paths, choosing default/CLI bases,
|
||||
and deciding whether to check existence, impose confinement, or use fallbacks.
|
||||
This feature neither performs those operations nor prohibits consumers from doing so.
|
||||
Actual Notarius integration remains downstream work.
|
||||
|
||||
Existing contracts, including import/profile composition and frozen post-load
|
||||
inspection, remain unchanged. Their authority comes from the current contracts,
|
||||
not from this roadmap's exclusions.
|
||||
|
||||
Source discovery, historical value access during resolution, automatic move
|
||||
tracking, and a general provenance service are not included in this feature.
|
||||
That scope choice is not a repository-wide prohibition on future proposals.
|
||||
|
||||
## Compatibility and canonical ownership
|
||||
|
||||
This is an additive public capability; existing callbacks, loading order, and
|
||||
configuration semantics must remain unchanged. Release selection and validation
|
||||
follow the [release policy](../policy/release.md) and
|
||||
[release procedure](../release.md). No release gate is waived here.
|
||||
|
||||
In the completed feature, Go reference comments own API and lifetime definitions,
|
||||
the configuration contract owns shared resolution/provenance semantics, and the
|
||||
adoption guide and executable examples own consumer usage. Component documentation
|
||||
describes the resulting internal boundaries. Accepted ADR rationale remains
|
||||
historical; changes to governing decisions follow the documented supersession policy.
|
||||
Publication remains a maintainer decision. Release preparation must choose the
|
||||
version, write a version-matched note, retire this completed roadmap with link
|
||||
repair, obtain exact-candidate Linux/macOS evidence, and follow the
|
||||
[release procedure](../release.md).
|
||||
|
||||
Reference in New Issue
Block a user