ConfigKit/docs/roadmap/source-origins-during-resolution.md

# Source origins during resolution

Status: planned; not implemented. This roadmap replaces the Notarius feature
request and owns the targeted end state. The [implementation plan](implementation.md)
owns execution order, progress, and handoffs.
This document does not authorize implementation or select a release version.

Current behavior remains defined by the [configuration contract](../config.md).
The [architecture policy](../policy/architecture.md) governs responsibility
boundaries, and the [documentation policy](../policy/documentation.md) governs
contract ownership and retirement of completed plans.

## Motivation

Notarius needs source-aware normalization of configured filesystem paths.
For example, `../references/party.txt` declared in `pipelines/dnd.yml` should
resolve relative to that declaring file, rather than the process working directory.
A profile replacement should use the profile file's directory even when it supplies
identical text; an untouched sibling should retain its original declaration.

Notarius separately chooses invocation-relative CLI paths, main-configuration-
relative defaults, and unchanged absolute paths. These are application policies,
not ConfigKit conventions. ConfigKit must provide the provenance needed to apply
them in one load without independently parsing YAML or reading files again.

Today, `Definition.Resolve` can inspect explicitness and record normalization, but
source-origin queries are available only after loading. Changing `Result.Config`
then would not change its frozen effective output or explanation. The target is to
make provenance available before normalization, while keeping runtime values and
inspection consistent through existing change recording.

## Target public surface

Add the following methods to the existing callback-scoped `Resolution` context:

```go
func (r *Resolution) InputInfo(path Path) (FieldInfo, bool)
func (r *Resolution) Source(id string) (SourceInfo, bool)
```

These signatures are the proposed target API, not callable APIs in the current
release. Reuse existing public `Path`, `FieldInfo`, `Origin`, `Location`, and
`SourceInfo` types. Do not introduce a second provenance record hierarchy, a
generic node API, or a new callback.

### InputInfo: a fixed pre-resolution view

Despite its short name, `InputInfo` describes the **decoded configuration at entry
to resolution**, including retained application defaults. It is not limited to
literal YAML nodes and is not a live view of callback mutations.

Its snapshot is taken after imports/composition, selected-profile application,
CLI overrides, defaults, and typed decoding, but before application resolution
and typed credential lookup. The query returns the same presence/origin meaning
as effective metadata at that boundary:

| Path state at callback entry | Result |
| --- | --- |
| Surviving file, profile, or CLI contribution | Found; existing explicit/null and origin information |
| Present default-only value | Found; non-explicit, default origin, no invented file location |
| Explicit null accepted by the schema | Found; explicit and null |
| Missing path or descendant removed before decoding | Not found unless ordinary decoding retained/reintroduced it through defaults |
| Invalid path or key/index shape mismatch | Not found |

A Go zero-valued field that exists in the decoded projection is not a missing
path. Zero metadata fields alone must not be used to infer absence; callers use
the returned boolean. Invalid/missing queries add no diagnostics.

Queries address exact semantic paths through nested structs, inlined fields,
mapping entries, arrays, and sequence elements. Use existing path rules, including
escaped keys, explicit key/index distinctions, and canonical contextual sequence
indices. Never substitute nearest-ancestor diagnostic lookup for exact field lookup.

For a scalar, the origin identifies its surviving declaration, not the contributors
of its parent mapping. Equal-valued profile and CLI replacements retain the
replacing origin. Containers retain their existing possibly composite locations;
the API does not select a representative owner or single declaring directory.

`InputInfo` and `IsExplicit` must agree for paths present in the entry snapshot.
The extension must not change existing `IsExplicit` behavior for paths outside
that view. Normalization records, additions, removals, and moves do not alter the
snapshot. A removed original path stays queryable; a new destination that did not
exist at entry remains absent from this view. If a destination already existed,
its query describes that original value, not the moved value.

### Source: resolve identities without filesystem work

`Source(id)` returns the existing source record for an ID belonging to this load.
Empty or unknown IDs return a zero record and false without adding diagnostics.
The source view contains the records already acquired before resolution, including
main input, imported files, the selected profile, and CLI records where applicable.
It does not discover files, reload input, or inspect unselected profiles.

A returned `SourceInfo.Path` is the canonical local path only when the existing
source model supplies one. File paths retain current canonicalization and symlink
semantics. Labels remain diagnostic labels, never a substitute for paths.
Bytes/readers and CLI records have no fabricated file path, even if a filesystem
context was supplied for imports or profiles. Defaults have no source record or
synthetic file identity.

Every source identity referenced by the entry view's origin locations must be
resolvable through this source view. Existing fields distinguish layers and import
relationships; do not require callers to guess scalar ownership from source order.
The existing main-source identity convention remains unchanged. Applications may
use the main source record for their own default-base policy; a non-file main
source still supplies no directory, and the application must choose its fallback.

Multiple locations do not imply one declaring file. Consumers needing a single
file must inspect the exact value's origin and source records and handle ambiguity
according to application policy. ConfigKit does not require rejection: an
application may choose an explicit fallback, retain a relative value, warn, or
report an error. No new `DeclaringFile` shortcut should silently discard contributors.

### Lifetime and detachment

Both methods require an active resolution callback. Nil, zero, or retained contexts
used after callback closure follow the existing fixed, value-free panic convention,
including when a callback returns an error. These methods do not make the context
safe for concurrent use.

Returned records are detached, including nested slices and `ImportedFrom`
locations. They may be retained after callback exit; modifying them must not affect
subsequent queries, other loads, final metadata, or explanations. The context's
lifetime restriction does not apply to those copied records.

## Consumer normalization workflow

The intended Notarius workflow is:

1. Query a path's entry-state field information before moving or normalizing it.
2. Resolve its origin source IDs and select a base using application policy.
3. Compute the application path without asking ConfigKit to interpret it.
4. Assign and record the result using existing `Normalize` or `Resolution.Record`
   semantics, including dependencies where appropriate.
5. Use the completed result for both execution and configuration inspection.

Consumers may retain original origin records for later dependent work in the same
callback. ConfigKit does not infer ownership transfers, rewrite the input view, or
automatically record a move. Existing mutation auditing and record validation
continue to apply to all changes.

For canonical file sources, a symlink refers to the resolved target file's
directory, not necessarily the directory containing the link used to open it.
The maintained example must make this explicit. Applications wanting another
path-base policy own that policy and the additional information it requires.

## Intended internal architecture

Keep one provenance model and one loading lifecycle. The existing
`resolveLoadInput` in `load_result.go` already assembles decoded metadata and
overlay history before calling `runResolution`. Pass a read-only entry-state
provenance view and the load's acquired source records into the private resolution
state. No public parser types or new external dependencies are needed.

The view must support complete effective semantic-path lookup when the callback
starts. Current result metadata indexes are finalized later; exposing an unfinished
`Metadata` value would make legitimate lookups fail. Reuse or factor existing
effective-path traversal and shape matching so entry queries and final queries
cannot acquire different path semantics. A full historical index or public
`Metadata.History` interface during resolution is not required by this feature.

Keep queries stable across consumer normalization and return detached records.
Private copying, immutable sharing, or ownership transfer may satisfy that contract;
no particular backing representation or eager-copy strategy is required. Prefer
existing traversal/indexing machinery and proportionate allocation. A per-load
source index is a recommended mechanism, not a public complexity guarantee or a
prohibition on scanning the bounded existing source collection. Consumers should
not need to rebuild source bookkeeping themselves.

The source and input views remain bounded by existing load limits. They contain
provenance records, not configuration values or resolved credentials. Do not read
the environment, resolve secret references, perform filesystem I/O, or invoke
application codecs in response to a query.

Existing composition precedence, canonical decoding, normalization auditing,
resolution origins/history, redaction, and frozen result publication remain
authoritative. This is an information-access extension, not a new normalization
engine or permission to mutate post-load snapshots.

## Acceptance criteria

The completed feature must demonstrate the following through public API behavior,
using real temporary files where source identity matters:

1. A main-file scalar can be normalized relative to its canonical declaring file
   during resolution, independent of invocation directory.
2. An imported scalar, including a sequence element, identifies its own declaring
   file rather than the main file or a parent container's contributors.
3. A profile replacement identifies the profile source while untouched siblings
   retain their base/import origins.
4. Equal-text profile and CLI replacements still identify the replacing source.
5. Defaults, missing paths, explicit null, CLI input, and file input are distinct;
   `NoSource` and non-file input introduce no fictional declaring paths.
6. Composite containers retain all contributing locations without selecting a file.
   Every referenced source ID resolves, and unknown IDs are harmless misses.
7. Queries remain fixed across normalization, deletion, addition, and moves;
   retained copies remain useful and detached.
8. Exact lookup works through typed and dynamic structures, escaped mapping keys,
   and sequence indices; invalid paths and shape mismatches do not borrow ancestor
   or sibling provenance.
9. A successful single load yields identical runtime and effective values, with
   explanation preserving original declarations and recorded normalization reasons.
10. Querying origins performs no extra load or credential lookup, and existing
    secret redaction, safe diagnostics, and callback-lifetime rules remain intact.
11. Mutating returned records cannot affect later queries or final results.
12. Maintained Notarius-shaped examples demonstrate the policy using only the
    public API and standard library, including documented canonical/symlink behavior.

Tests should extend existing provenance, resolution, ownership, and consumer
behavioral owners rather than duplicate every mechanism at every layer. Acceptance
does not require a live Notarius checkout, external credentials, or target-file
existence checks.

## Scope and existing boundaries

Application responsibilities remain application responsibilities: interpreting
path-valued strings, normalizing filesystem paths, choosing default/CLI bases,
and deciding whether to check existence, impose confinement, or use fallbacks.
This feature neither performs those operations nor prohibits consumers from doing so.
Actual Notarius integration remains downstream work.

Existing contracts, including import/profile composition and frozen post-load
inspection, remain unchanged. Their authority comes from the current contracts,
not from this roadmap's exclusions.

Source discovery, historical value access during resolution, automatic move
tracking, and a general provenance service are not included in this feature.
That scope choice is not a repository-wide prohibition on future proposals.

## Compatibility and canonical ownership

This is an additive public capability; existing callbacks, loading order, and
configuration semantics must remain unchanged. Release selection and validation
follow the [release policy](../policy/release.md) and
[release procedure](../release.md). No release gate is waived here.

In the completed feature, Go reference comments own API and lifetime definitions,
the configuration contract owns shared resolution/provenance semantics, and the
adoption guide and executable examples own consumer usage. Component documentation
describes the resulting internal boundaries. Accepted ADR rationale remains
historical; changes to governing decisions follow the documented supersession policy.