From 818d36f836a882b37999b1c4acb0d448f24f6afc Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sat, 12 Sep 2026 13:05:00 +0000 Subject: [PATCH] Clarify implementation discipline and provenance feature scope --- AGENTS.md | 33 +++++++ docs/roadmap/implementation.md | 88 +++++++++++-------- .../source-origins-during-resolution.md | 36 +++++--- 3 files changed, 108 insertions(+), 49 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 34531ca..68a03c4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,2 +1,35 @@ Please review `docs/development.md` for initial orientation in this repository and follow its task-specific reading guide. + +## Implementation Discipline + +Simplify the mechanism, not the user's choices. + +**Keep implementation machinery proportionate.** Use straightforward, +idiomatic code and introduce only the machinery needed to support the requested +behavior and flexibility. Optimize for conceptual simplicity and +maintainability, not for the fewest lines, fewest options, or narrowest product +surface. + +**Do not invent restrictions.** Do not reject, prohibit, constrain, cap, +harden, or otherwise narrow behavior unless required by the user, an explicit +repository contract, or an unavoidable correctness constraint. Preserve +operator and caller choice where policy has not been established. If an +additional restriction appears advisable, present it as a decision rather than +silently implementing it. + +A correctness constraint must follow from the requested behavior or an +established invariant, not from the convenience of the chosen implementation. +Explain the specific failure it prevents and why a less restrictive +implementation cannot satisfy the contract. + +**An implementation detail is not repository policy.** Tests and documentation +must not elevate incidental behavior into a durable guarantee or restriction +unless that behavior was expressly selected as part of the contract. Existing +explicit contracts remain authoritative; changing or weakening one requires a +deliberate decision rather than being inferred from this principle. + +Agent-authored plans, tests, documentation, and prior implementations are not +evidence that an additional restriction was deliberately approved. Identify the +authorizing requirement or decision; when that authority is unclear, surface +the ambiguity rather than treating repetition as approval. diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 4ffb1de..4632b81 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -42,36 +42,42 @@ func (r *Resolution) InputInfo(path Path) (FieldInfo, bool) func (r *Resolution) Source(id string) (SourceInfo, bool) ``` -Use a private, immutable entry-state view, not live final metadata. Store only -effective field records, effective path shapes, and a source-ID index; do not build -or expose an additional historical view. Reuse `Metadata.Lookup` semantics by -representing the field/path part as a private `Metadata` value with only those -indexes populated. Never return that `Metadata` value publicly. +The required behavior is a stable entry-state view and detached public return +records, not a particular internal representation. The recommended implementation +is a private `Metadata` value containing effective fields and path shapes, together +with a source-ID index. Reuse `Metadata.Lookup` semantics. A historical view is not +needed for this feature, and exposing one is outside this sprint's scope. Build the view after `metadataFromDecoded` and `applyOverlayMetadata` in `resolveLoadInput`, immediately before `runResolution`, only when a resolver exists. Its path shapes come from `decoded.final`, not the raw document or mutable Go -configuration. Clone field records and their nested origins into a separate map; -do not shallow-copy the mutable final metadata map. Build a map keyed by ID from -`assembledInput.sources`, cloning `SourceInfo` records with existing helpers. +configuration. Separate copies of field/source records are one straightforward way +to isolate the view; immutable sharing or ownership transfer is equally acceptable +when it preserves query stability and detached returns. Source records come from +`assembledInput.sources`. Avoid aliases that observable consumer operations or +subsequent metadata updates could mutate while the view remains usable. Missing or duplicate internally generated IDs are implementation-invariant failures to fix, not new consumer-facing diagnostic categories. -Pass the private view as an explicit argument to `runResolution` and retain it in +Prefer passing the private view explicitly to `runResolution` and retaining it in `resolutionState`. Update all private callers/tests deliberately: construct their entry view from their decoded fixtures and supplied sources where they test these -queries. Do not add optional/variadic arguments that silently omit provenance in -production. Private names are discretionary, but ownership and timing are not. +queries. Every production resolution callback must receive complete provenance. +Private signatures and representation may change when an idiomatic alternative +better serves that requirement; record the reasoning in the stage handoff. Queries first check callback activity through the existing context guard. Only then may invalid paths or unknown IDs return misses. `InputInfo` delegates to the entry metadata lookup; `Source` clones the matched source on return. Preserve existing `IsExplicit`, diagnostics, resolution recording, and secret timing. -Do not clone input trees, re-project configuration, read files, resolve symlinks, -or access credentials during a query. Effective indexing is O(number of entry -paths), source indexing is O(number of sources), and each lookup must avoid a full -tree/source scan. Returned-copy cost is proportional to the returned record. +Queries must not reload input, read files, resolve symlinks, access credentials, +or re-project mutable configuration instead of reading entry-state information. +Prefer reusing existing indexes and avoiding unnecessary whole-tree copies. +Per-load source indexing is a recommendation, not a complexity guarantee: a simple +scan may be appropriate for the existing bounded source collection. Assess costs +in context, including path lengths and nested record copies; this feature adds no +performance guarantees, resource caps, or benchmark gates. ## Stage 1 — Factor reusable effective-path indexing @@ -79,20 +85,23 @@ Status: not started. Dependencies: none. Scope: `metadata.go`, `load_metadata.go`, and their focused test owners. -1. Extract the effective-path indexing portion of `finalizeMetadataPaths` into one - private helper using existing document traversal and detached path construction. - Keep historical indexing and result publication behavior unchanged. -2. Make result finalization use the extracted helper. Add the private entry-view - constructor described above, accepting the pre-resolution metadata, decoded - final tree, and acquired source records. Do not expose public methods yet. -3. Ensure the constructor copies field records and source records without copying - history or configuration values. Reuse `cloneFieldInfo`, `cloneSourceInfo`, and - existing shape matching instead of writing another path parser or numeric matcher. +1. Prepare reusable effective-path lookup for the entry view. The recommended + approach extracts indexing from `finalizeMetadataPaths`; an equivalent design + may reuse existing traversal directly. Keep historical indexing and result + publication behavior unchanged, and preserve one set of semantic-path rules. +2. Add the private entry-view construction described above using pre-resolution + metadata, the decoded final tree, and acquired source records. Share extracted + machinery with result finalization where useful. Do not expose public methods yet. +3. Establish and document internal ownership sufficient for stable queries and + detached returns. Reuse `cloneFieldInfo` and `cloneSourceInfo` where copying is + needed; do not require eager copies or duplicate history solely to match this + plan's recommended representation. 4. Add focused coverage proving the private view recognizes defaulted fields, missing descendants, root, escaped keys, explicit key/index mismatches, and - contextual canonical sequence indices. Prove later mutation of the original - metadata and source records cannot alter the view. Reuse existing shape tests - for exhaustive syntax cases rather than duplicating them. + contextual canonical sequence indices. Test stability across actual lifecycle + operations and mutation of returned records, not arbitrary mutation of backing + data that the implementation owns as immutable. Reuse existing shape tests for + exhaustive syntax cases rather than duplicating them. Acceptance: all existing final metadata/history behavior is unchanged; a complete entry view can be constructed before final publication without a historical index. @@ -185,11 +194,13 @@ explanation/secret integration tests, and consumer guidance. default-relative paths based on the canonical main source; file-authored paths based on the scalar's declaring source. Use the existing primary-source identity convention for main-source lookup, not a guess about contributor ordering. -3. For this file-rooted example, require one unambiguous canonical declaring file - for file-relative values. Report a safe application resolution error if that - prerequisite is absent or ambiguous. Do not silently use a diagnostic label or - working directory. Non-file-root fallback remains an explicitly separate - application policy, not a new library option. +3. Demonstrate file-relative normalization with fixtures that have an unambiguous + declaring file. If the example handles absent or ambiguous file provenance, + make that behavior an explicit example policy, not a correctness requirement + or prescribed Notarius policy. Rejection, a caller-supplied fallback directory, + retaining a relative value, or warning are application choices. The example's + selected behavior must be documented and tested; ConfigKit must continue to + expose the distinction rather than fabricate a file from a diagnostic label. 4. Record every changed path through `Normalize` or `Record`, including valid dependencies where derived values depend on other configuration paths. Prove runtime values, effective YAML, and explanation agree; retain the original @@ -222,9 +233,10 @@ and proportionate fixes for remaining acceptance failures. No feature expansion. implementation. Record the mapping in this stage's completion entry. Passing broad suites alone is not evidence that a missing scenario was exercised. 2. Inspect every new query path for environment/filesystem access, raw values, - mutable aliases, accidental callback-lifetime extension, and per-query whole-tree - work. Confirm no history was duplicated solely for these queries and loads - without a resolver avoid constructing an unused entry snapshot. + caller-mutable aliases, and accidental callback-lifetime extension. Review + allocation and traversal costs for proportionality without turning the chosen + mechanism into a public guarantee or adding unsupported performance gates. + Avoid unnecessary history duplication and unused work in no-resolver loads. 3. Review public signatures and current configuration behavior against the previous release. Confirm the capability is additive and no parser, path, composition, credential, or existing callback semantics changed unintentionally. @@ -260,8 +272,10 @@ Validation: `./scripts/check.sh`, `go test -race ./...`, and the full checker ab ## Open Questions None blocking implementation. The API, snapshot boundary, missing-value semantics, -source identity handling, ownership, and example fallback policy are specified by -the roadmap and decisions above. Release version and publication timing are later +source identity handling, and ownership are specified by the roadmap and decisions +above. Example fallback behavior is illustrative application policy, not a missing +library contract; record any selected example behavior without prescribing it to +consumers. Release version and publication timing are later maintainer decisions, not unresolved implementation choices. If new evidence contradicts a target contract, document the conflict, a recommended correction and its rationale, and at least one viable alternative before proceeding past it. diff --git a/docs/roadmap/source-origins-during-resolution.md b/docs/roadmap/source-origins-during-resolution.md index a771c7f..fe11afb 100644 --- a/docs/roadmap/source-origins-during-resolution.md +++ b/docs/roadmap/source-origins-during-resolution.md @@ -107,7 +107,9 @@ source still supplies no directory, and the application must choose its fallback Multiple locations do not imply one declaring file. Consumers needing a single file must inspect the exact value's origin and source records and handle ambiguity -explicitly. No new `DeclaringFile` shortcut should silently discard contributors. +according to application policy. ConfigKit does not require rejection: an +application may choose an explicit fallback, retain a relative value, warn, or +report an error. No new `DeclaringFile` shortcut should silently discard contributors. ### Lifetime and detachment @@ -150,18 +152,20 @@ overlay history before calling `runResolution`. Pass a read-only entry-state provenance view and the load's acquired source records into the private resolution state. No public parser types or new external dependencies are needed. -The view must have complete effective semantic-path indexing when the callback +The view must support complete effective semantic-path lookup when the callback starts. Current result metadata indexes are finalized later; exposing an unfinished `Metadata` value would make legitimate lookups fail. Reuse or factor existing effective-path traversal and shape matching so entry queries and final queries cannot acquire different path semantics. A full historical index or public `Metadata.History` interface during resolution is not required by this feature. -Keep the view independent of metadata updates performed after resolution. -Implementations may share immutable backing data privately while returning detached -records; do not copy an entire configuration tree per query. Source lookup should -use a per-load index or equivalent bounded existing mechanism, rather than requiring -each consumer to rebuild source bookkeeping. +Keep queries stable across consumer normalization and return detached records. +Private copying, immutable sharing, or ownership transfer may satisfy that contract; +no particular backing representation or eager-copy strategy is required. Prefer +existing traversal/indexing machinery and proportionate allocation. A per-load +source index is a recommended mechanism, not a public complexity guarantee or a +prohibition on scanning the bounded existing source collection. Consumers should +not need to rebuild source bookkeeping themselves. The source and input views remain bounded by existing load limits. They contain provenance records, not configuration values or resolved credentials. Do not read @@ -207,14 +211,22 @@ behavioral owners rather than duplicate every mechanism at every layer. Acceptan does not require a live Notarius checkout, external credentials, or target-file existence checks. -## Explicit exclusions +## Scope and existing boundaries -Do not add automatic path-string recognition, filesystem path normalization, -existence checks, path confinement, default/CLI base selection, source discovery, -new import/profile composition rules, historical value access during resolution, -automatic move tracking, post-load mutability, or a general provenance service. +Application responsibilities remain application responsibilities: interpreting +path-valued strings, normalizing filesystem paths, choosing default/CLI bases, +and deciding whether to check existence, impose confinement, or use fallbacks. +This feature neither performs those operations nor prohibits consumers from doing so. Actual Notarius integration remains downstream work. +Existing contracts, including import/profile composition and frozen post-load +inspection, remain unchanged. Their authority comes from the current contracts, +not from this roadmap's exclusions. + +Source discovery, historical value access during resolution, automatic move +tracking, and a general provenance service are not included in this feature. +That scope choice is not a repository-wide prohibition on future proposals. + ## Compatibility and canonical ownership This is an additive public capability; existing callbacks, loading order, and