Finalize ConfigKit design and implementation plan

docs/roadmap/implementation.md

@@ -0,0 +1,784 @@
+# ConfigKit Initial Implementation Plan
+
+## Status and ownership
+
+Status: ready for implementation; all stages are pending.
+
+Audience: a `gpt-5.6-terra` coding agent implementing one stage per prompt, in
+ascending order. Each stage depends on every earlier stage being complete.
+Do not combine stages merely because they share files. Finish the requested
+stage's acceptance checks and handoff before moving to the next requested stage.
+
+The [feature roadmap](initial.md) owns the desired end-state and consumer intent.
+This document owns execution decisions, tasks, acceptance gates, and progress.
+Read [development](../development.md), all four policies
+([architecture](../policy/architecture.md), [documentation](../policy/documentation.md),
+[testing](../policy/testing.md), [release](../policy/release.md)), and the relevant
+[ADRs](../development.md#task-specific-reading-guide) before implementation.
+Both [success criteria](../policy/architecture.md#success-criteria) apply to every
+stage. This plan does not authorize feature expansion or downstream application
+releases.
+
+## Execution rules and stage handoff
+
+1. Inspect the actual worktree, prior stage evidence, relevant source, and tests.
+   Preserve unrelated work. This repository initially has documentation only.
+2. Read the linked feature sections for the requested stage. Follow links to
+   current contracts if earlier work has moved a definition out of the roadmap.
+3. Extract applicable Narratio code and cases before writing equivalent new
+   mechanisms. Record source revision, symbols, adaptations, and retained notices.
+   Do not import sibling applications or their internal packages into this module.
+4. Implement only the stage boundary. Partial internals may be tested directly;
+   do not export placeholder APIs, silently ignore unsupported options, or claim
+   that later features work. Introduce public operations only when their stated
+   supported boundary is functional. No release occurs between these stages.
+5. Run focused checks while iterating, then the stage baseline. Add durable tests
+   at the owning boundary; do not replicate every case through `Load`.
+6. Update Go reference comments, implemented configuration contracts, component
+   inventory, and examples in the same change as behavior. Move implemented
+   normative text from the feature roadmap to its current owner, leaving links.
+   Keep this plan's tasks and completion evidence here, not in the feature roadmap.
+7. Set the stage's status to complete only after its acceptance gates pass. Append
+   a short evidence entry under that stage: files/contract links, extraction
+   references, checks and outcomes, deviations, and remaining later-stage work.
+   Do not treat compilation against stubs as runtime acceptance.
+
+Baseline after Stage 1: `go test ./...`, `go vet ./...`, `go build ./...`, and
+`git diff --check`, plus inspection of new files, documentation links/fragments,
+and formatting. These are future implementation commands; Stage 1 establishes
+them in the developer guide. Tests must run without network access or credentials
+after explicit dependency acquisition. Do not put network calls in default tests.
+
+For a failed gate, fix the implementation or explicitly resolve a contract issue;
+do not change expectations solely to get a pass. Record external tool/platform
+limitations accurately. Do not mark a required check passed if it was skipped.
+
+## Implementation decisions
+
+These decisions close choices left illustrative in the feature roadmap. They
+are instructions for implementation, not additional configuration conventions.
+Promote their externally observable parts into current contracts when implemented.
+
+| Area | Decision and reason |
+| --- | --- |
+| Module and package | Module `gitea.maximumdirect.net/eric/ConfigKit`, matching the existing origin's case; root package `configkit`. No `cmd` application. |
+| Go baseline | `go 1.25.0`; use a currently supported patched toolchain at or above that minimum for development, and verify minimum-version compatibility separately. The existing Narratio/Seriatim baselines and local 1.25.5 toolchain support this language baseline. No toolchain directive forcing a newer consumer toolchain. |
+| Parser | Pin `go.yaml.in/yaml/v3 v3.0.5` initially, inside `internal/yaml`. It preserves the v3 node-oriented extraction seam while using the YAML organization's maintained module. Do not inherit its scalar coercions. The [upstream v3 reference](https://pkg.go.dev/go.yaml.in/yaml/v3@v3.0.5) documents this version and provenance. |
+| Internal representation | A ConfigKit-owned tree in `internal/document`: ordered mapping entries, sequences, typed scalar kind plus original numeric spelling, null, key/value locations, and source ID. It imports neither the public package nor the YAML parser. The YAML adapter translates into it. |
+| Dependency direction | Public package coordinates document operations and reflection binding. Private parser imports `internal/document`; document code never imports parser or public package. Keep composition, binding, provenance, hooks, and rendering coordination in root-package private files until a demonstrated boundary needs another package. |
+| Public spelling | Implement the feature roadmap's `Load`, `Definition`, `Options`, sources, path builders, metadata queries, hook helpers, version rule, result methods, and diagnostic methods as written. No functional-options framework or separate builder. |
+| Input limits | Fixed initial ceilings: 8 MiB per source, 64 MiB total loaded bytes, depth 256 for input/projected trees, 64 active import edges, and 1,024 accepted sources. Allow at most 1,000,000 input nodes in aggregate and 1,000,000 nodes in each composed/projected tree; copying a snapshot does not consume the input budget again. Bound glob directory-entry visits separately at 1,000,000 per load. Include the selected profile in source/byte counts. Root depth is zero. No public limit-tuning options initially. Test limit mechanisms with smaller private test parameters. |
+| Filesystem support | Initial supported targets: Linux and macOS on amd64/arm64. Test real symlink semantics on those systems. No Windows path-compatibility promise in this release. Inputs use slash-separated relative import syntax and native paths for selected file/root/base options. |
+| Schema implementation | Build one reflection descriptor graph per load, reuse it throughout that load, and avoid global mutable caches initially. Recursive type definitions are permitted when actual values are acyclic. |
+| Working configuration | Clone the defaults factory's result before applying input. Decode explicitly present nodes into that detached baseline according to schema descriptors; never round-trip defaults through YAML. Snapshot final values independently. |
+| Error boundary | Safe library diagnostics with phase-specific codes; no raw parser/custom-hook error string or raw-value wrapper chain. `LoadError` supports `errors.As`; there is no initial `Unwrap` promise. I/O classifications are exposed as diagnostic codes. |
+| Consumer integration scope | Build copied, sanitized consumer-shaped models/fixtures and executable examples in ConfigKit. Actual changes to Narratio, Notarius, WeatherReporter, and Seriatim are separate consumer migration tasks. Do not claim those apps are migrated when these local examples pass. |
+| Publication scope | Prepare library source and a release runbook/checker. Do not tag, push, publish, or invent a completed release during these stages. Release-candidate validation/publication follows the adopted release procedure as a separate maintainer action. |
+
+The parser dependency choice is based on its own published documentation, not a
+promise that v3.0.5 will always be the newest patch. A later verified dependency
+fix can change the pin with focused YAML regression checks and normal review.
+Do not silently switch parser families or use `@latest` during implementation.
+
+### Metadata and diagnostics layout
+
+Use opaque `Path`, `Source`, `Metadata`, `Resolution`, and `Validation` state.
+Expose detached value records for inspection, with these initial fields:
+
+| Type | Fields |
+| --- | --- |
+| `Location` | `SourceID string`, `Line int`, `Column int`; zero coordinates mean unavailable/synthetic |
+| `SourceInfo` | `ID string`, `Label string`, `Path string` (canonical local path or empty), `Layer Layer`, `ImportedFrom *Location` |
+| `Diagnostic` | `Code Code`, `Severity Severity`, `Path Path`, `Message string`, `Primary *Location`, `Related []Location`; absent primary means no known source |
+| `Origin` | `Layer Layer`, `Locations []Location`, `Operation ChangeKind`; multiple locations represent a composite origin |
+| `FieldInfo` | `Explicit bool`, `Null bool`, `Origin Origin` |
+| `Change` | `Path Path`, `Origin Origin`, `Previous []Origin`, `Reasons []string`, `Dependencies []Path`; no public historical values |
+
+`Layer` is a string enum: `default`, `base`, `profile`, `resolution`. A Go zero
+baseline uses layer `default` without claiming an application deliberately chose
+every zero value. `ChangeKind` has the empty value for input/default origins,
+`derived_default`, and `normalization`. `Severity` is `error` or `warning`.
+Define exported constants with the type prefix (for example `LayerBase`,
+`ChangeDerivedDefault`, `SeverityError`).
+
+Use stable string codes grouped by `options.*`, `source.*`, `yaml.*`, `schema.*`,
+`decode.*`, `compose.*`, `import.*`, `profile.*`, `version.*`, `resolve.*`, and
+`validate.*`. Include distinct missing/permission/read, duplicate, invalid-path,
+limit-exceeded, hook-failed, and unrecorded-change classifications where relevant.
+Application reports use their phase's `error`/`warning` code, not arbitrary code
+registration. Define exact exported code constants with their owning feature;
+private helpers do not need new public codes for every branch.
+
+Sources get deterministic per-load IDs `source-1`, `source-2`, and so on in
+traversal order. The selected profile is last. `NoSource` creates no source ID.
+Public query slices and nested slices/pointers are detached. Default fmt/error
+rendering must not reveal hidden value histories.
+
+### Clarifications required for a coherent implementation
+
+- `ParsePath` retains textual numeric tokens until traversal knows the container;
+  `Keys("0")` is a mapping key and `Index(0)` is a sequence index. Reject invalid
+  escapes and negative indices. The zero `Path` is the root. Empty mapping keys
+  remain representable. Resolve parsed numeric indices without leading zeros;
+  the same text may address a literal mapping key when traversing a mapping.
+- Path queries on missing fields return the feature-defined empty result. Hook
+  diagnostics permit schema-valid missing fields/map entries; malformed paths or
+  unknown struct fields produce a hook diagnostic. Version hooks use the untyped
+  composed tree and nearest known container because decoding has not run yet.
+- Defaults apply to the surviving composed/profile input; intermediate overridden
+  nodes do not create reset markers. An effective null suppresses defaults at
+  its path. Missing children of an effective mapping can retain defaults.
+- Applying nonempty input to a nil pointer-to-struct creates the object and uses
+  zero values for omitted children. No per-entry defaults are invented. Applying
+  a sequence allocates a replacement; it never overlays default indices.
+- Support paired text marshal/unmarshal scalar methods on value or pointer method
+  sets, with exact `time.Duration` handled first. Treat such scalars atomically.
+  Their canonical text supplies projection and detached reconstruction. Never
+  call YAML-specific marshal/unmarshal hooks; an otherwise ordinary struct with
+  such a method is bound structurally under ConfigKit's rules.
+- Ignore ordinary unexported and `yaml:"-"` fields; zero them in detached ordinary
+  struct results rather than preserve hidden mutable aliases. Paired text scalars
+  are the exception: their declared scalar codec owns their internal state.
+  Reject unsupported schema kinds and unsupported `yaml` tag options rather than
+  silently reinterpret them. Support `inline` and ignore `omitempty`; defer `flow`.
+- Repeated helper writes to one path compare only the last helper-assigned value
+  with the final projected value. Coalesce reasons/dependencies. For mixed
+  operations, a path record is normalization if any record normalized it; otherwise
+  it is a derived default. Record coverage of a container includes descendants.
+- Query explicitness against surviving input contributions. A normalization of an
+  existing path preserves its input explicitness. A newly created path is not
+  explicit just because the resolver copied another explicit field; dependencies
+  carry its explanation. Deletions remain historical, not effective entries.
+- Validator mutation is a programming error detected by before/after projection;
+  return `validate.mutation` and no result. Do not recover user callback panics.
+  Retained contexts are invalid after the hook. Calls on a closed context panic
+  with a fixed, value-free programming-error message and cannot mutate a published
+  result; this is callback misuse, not a late load diagnostic.
+- The initial implementation assumes stable, application-controlled local input
+  trees during a load. Resolve symlinks and enforce containment, but do not claim
+  atomic multi-file snapshots or protection against concurrent hostile filesystem
+  replacement. Reject directories and non-regular file targets; arbitrary streams
+  are accepted only through the explicit reader API.
+
+## Stage 1 — Establish the module and extraction baseline
+
+Status: pending.
+
+Read: feature [foundation](initial.md#existing-implementation-foundation),
+[consumer fit](initial.md#consumer-fit-and-intentional-migrations), all policies.
+
+Tasks:
+
+1. Create the module and root package documentation with the decisions above.
+   Keep the existing repository license. Add no application command, runtime
+   dependencies, framework, or placeholder exported loader.
+2. Reinspect Narratio's referenced revision and current worktree. Create
+   `docs/internal/extraction.md` recording source revisions/symbols and the exact
+   loading/presence/default/diagnostic cases to extract. Include applicable notices
+   with copied material. Copy only minimal sanitized fixtures needed by subsequent
+   stages; no audio, transcripts, secrets, or entire application package trees.
+3. Document the planned parser pin here; add its module requirement only when the
+   real adapter imports it in Stage 3 so tidy does not leave unused dependencies.
+4. Update the developer guide to runnable Go baseline commands and prerequisites.
+   Correct its stale claim that this plan is deferred. Update documentation-policy
+   navigation to this now-existing plan without changing ownership rules. Record
+   the implemented root package in the overview and retain pre-feature status.
+
+Acceptance:
+
+- Module path/case agrees with origin, root package builds, and baseline passes.
+- Extraction notes distinguish copied/generalized code from new work and domain
+  logic retained in consumers. No sibling checkout is required to build/test.
+- No misleading usable-loader example or unrelated files enter the module.
+
+## Stage 2 — Implement semantic paths and safe diagnostic records
+
+Status: pending.
+
+Read: feature [paths](initial.md#paths-and-presence),
+[diagnostics](initial.md#provenance-diagnostics-and-redaction), public metadata API.
+
+Tasks:
+
+1. Implement `Path`, builders, parsing/stringification, private traversal helpers,
+   and the metadata/diagnostic value types specified above. Keep records detached.
+2. Implement diagnostic accumulation, canonical ordering, safe `LoadError`,
+   `WriteDiagnostics`, and error-code constants needed so far. Source-less errors
+   sort before source errors; then source order, line/column, path, code, severity,
+   and safe message break ties. Canonicalize related locations deterministically.
+3. Render control characters in labels/messages/paths as escaped text so source
+   labels cannot corrupt terminal output. Never include values or parser errors.
+4. Document the implemented path and diagnostic API next to declarations.
+
+Acceptance:
+
+- Round-trip escaped slash/tilde/empty/dotted/numeric keys and indexed paths;
+  distinguish a malformed path from a valid missing path.
+- `errors.As` finds `LoadError`; mutation of returned nested records cannot alter
+  later results. Safe writer failures propagate, including partial output.
+- Reordered independent diagnostic submissions render identically; no blanket
+  golden snapshot of unstable prose is required. Baseline passes.
+
+## Stage 3 — Extract source reading and enforce YAML policy
+
+Status: pending.
+
+Read: feature [YAML policy](initial.md#yaml-and-scalar-policy),
+[source constructors](initial.md#entry-point-and-reusable-application-definition).
+
+Tasks:
+
+1. Adapt Narratio's file/reader loading helpers into the opaque source constructors
+   and bounded private reader. Add `NoSource`, byte snapshotting, source labels,
+   regular-file checks, deterministic I/O classification, and reader single-use
+   state shared by source copies. File loads reopen each time; readers are not closed.
+2. Add the pinned YAML adapter and ConfigKit-owned document tree. Decode into parser
+   nodes only; translate key/value locations and scalar spelling before discarding
+   parser-specific nodes. Do not use parser typed decode for application structs.
+3. Enforce exactly one mapping document, duplicate keys, tag/alias/anchor/merge-key
+   rejection, and ConfigKit's own scalar grammar. A second successful decode is
+   an error, including an empty document; do not preserve Narratio's trailing-doc bug.
+4. Enforce byte ceilings before parsing and tree depth/node ceilings during adapter
+   translation. The parser can reject stricter malformed/nesting inputs first, but
+   none may panic or expose input text. Add private small-limit injection for tests.
+5. Create `docs/config.md` for implemented syntax/input behavior and link from
+   package documentation. Move the implemented definitions out of the feature
+   roadmap, preserving navigation for this plan.
+
+Acceptance:
+
+- Real file and reader tests cover missing/permission/read failures, non-regular
+  targets, byte snapshots, second reader use, and no caller-reader close.
+- Policy tables cover duplicate keys, all forbidden YAML constructs, null spellings,
+  quoted literals, JSON numeric syntax, legacy booleans/numbers, timestamp-like
+  strings, multiple/empty documents, and root type. Numeric nodes never round via float.
+- Small-limit boundary tests and malformed input tests return safe diagnostics with
+  available coordinates. This stage does not promise a typed public `Load` yet.
+
+## Stage 4 — Build the reusable schema binding graph
+
+Status: pending.
+
+Read: feature [typed decoding](initial.md#typed-decoding) and
+[type/projection contract](initial.md#supported-types-and-projection).
+
+Tasks:
+
+1. Implement reflection descriptors for supported root structs/maps, scalar kinds,
+   pointers, nested structs, string-keyed maps, slices, fixed arrays, and `any`.
+2. Bind exported field names, tags, inline structs/pointers and the optional inline
+   map. Detect ambiguous names, multiple inline maps, unsupported tag options,
+   conflicting defaults-map names (checked against runtime defaults in Stage 6),
+   unsupported kinds, and missing paired text codecs. Identify secret-tag paths.
+3. Support recursive descriptors without infinite construction; actual value cycles
+   are checked by projection. Use descriptors for schema-aware path validation.
+4. Keep reflection traversal private. Do not expose generic subtree conversion,
+   module registries, YAML hooks, or interfaces for arbitrary decoders.
+
+Acceptance:
+
+- Focused tests cover nested/inline conflicts, ignored/private fields, maps of
+  structs, named scalar types, pointer method sets, root rejection, recursive
+  acyclic models, and unsupported kinds. Descriptor building has no side effects.
+- No application parser-specific hook executes. Baseline passes.
+
+## Stage 5 — Implement canonical projection and detached copies
+
+Status: pending.
+
+Read: feature [ownership](initial.md#recorded-resolution-and-validation) and
+[type/projection contract](initial.md#supported-types-and-projection).
+
+Tasks:
+
+1. Use Stage 4 descriptors to project ordinary typed values into a ConfigKit tree
+   without YAML serialization. Include zero values and preserve nil/empty distinctions.
+2. Clone ordinary configuration into independently owned values. Reconstruct paired
+   text scalars through canonical text into fresh instances; preserve type identity
+   and supported dynamic concrete values without retaining mutable input aliases.
+3. Detect cyclic values, non-finite floats, invalid dynamic types, projection limit
+   violations, and custom codec errors safely. Do not reveal the codec error text.
+4. Add structural comparison used later by resolution audit and validator-mutation
+   detection. Track paths of changed values, container replacements, and removals;
+   do not create an index-wise correspondence between replaced sequences.
+
+Acceptance:
+
+- Mutating input/default maps, pointers, slices, or paired text-scalar backing data
+  cannot change a clone or snapshot. Ignored fields do not preserve hidden aliases.
+- Nil versus empty, exact wide integers, durations, canonical scalar text, recursive
+  acyclic objects, cycles, unsupported dynamic values, and codec failures are covered.
+- Comparison reports meaningful paths and container changes; baseline passes.
+
+## Stage 6 — Decode scalar and container values over defaults
+
+Status: pending.
+
+Read: feature [defaults/null](initial.md#static-defaults-and-null) and typed decoding.
+
+Tasks:
+
+1. Implement schema-driven typed decoding into a Stage 5 clone of the fresh default
+   baseline. Allocate replacements for sequences and recurse mappings by the feature
+   contract; do not let parser behavior determine merging or optionality.
+2. Enforce numeric syntax/range and nullability. Support dynamic `int64`/`float64`,
+   finite floats, exact-duration string parsing, named scalar types, paired text
+   codecs, arrays with exact length, and strict nested map-of-struct decoding.
+3. Reject unknown fields, tag binding conflicts, and invalid defaults consistently.
+   Collect independent decode errors while leaving the failed working object private.
+4. Return a private decoded configuration plus its surviving input presence index,
+   projected baseline/final trees, and per-path default/input origins for Stage 7.
+   Defaults never become explicit; equal-valued input remains explicit.
+
+Acceptance:
+
+- A compact table proves absent/false/zero/empty/null behavior, default map entries,
+  `{}`, `[]`, pointer objects, new entries, dynamic null, and non-nil default isolation.
+- Overflow, non-finite values, quoted-number coercions, integer durations, inline
+  conflicts, and unknown nested fields fail at original input paths/coordinates.
+- Adapt Narratio's presence/default test cases with intentional differences noted.
+  Repeated loads do not alias each other. Baseline passes.
+
+## Stage 7 — Assemble single-source loading and schema-version checks
+
+Status: pending.
+
+Read: feature [lifecycle](initial.md#resolution-lifecycle),
+[version hook](initial.md#schema-checking-and-secret-classification),
+[results](initial.md#results-errors-and-inspection).
+
+Tasks:
+
+1. Introduce `Definition`, `Options`, `Load`, and `Result` at the single-source
+   boundary. Wire source/options checks, parsing, version checking, fresh defaults,
+   decode, and detached result. Accept zero or one options argument.
+2. Implement `VersionRule`, `Version`, and the pre-decode `Validation` report context,
+   including safe error/warning messages, related input paths, absent version from
+   `NoSource`, and generic fallback only when no explicit error was reported.
+3. Implement base/default `Metadata.Lookup`, `IsExplicit`, `History`, source listing,
+   and warning/error lists. Keep snapshot internals private. No effective writer
+   is exposed before its redaction implementation in Stage 15.
+4. Reject `_configkit` when imports are disabled. Until their later stages land,
+   reject non-nil resolver/validator hooks and import/active-profile options with
+   an explicit unreleased-feature options error rather than ignore them. Document
+   only the supported single-source boundary. Remove each gate in its owning stage.
+5. Ensure `NoSource` performs version/default/decode/projection processing with no
+   source records and rejects filesystem/import/active-profile settings.
+
+Acceptance:
+
+- Actual `Load` integration tests cover File/Bytes/Reader/NoSource and nil result on
+  source, schema, version, or decode failure. Defaults factory runs only after a
+  successful version check and once per valid load.
+- String version, missing/null policy, integer rejection, warnings, actionable
+  safe reports, raw error suppression, and invalid options are exercised.
+- Metadata reflects input versus defaults and detached ownership. Baseline passes.
+
+## Stage 8 — Implement recorded application resolution
+
+Status: pending.
+
+Read: feature [resolution callbacks](initial.md#resolution-and-validation-callbacks).
+
+Tasks:
+
+1. Add per-load `Resolution`, `Default`, `Normalize`, `Record`, `Error`, and `Warning`.
+   Hook context queries refer to surviving input, not guesses from typed zero values.
+2. Run the resolver after decoding; capture before/final projection, detect net
+   changes, and require record coverage. Support map-entry copy/writeback helpers,
+   container records, deletions, dependencies, repeated writes, and equal-valued
+   derived-default intent using the clarification rules above.
+3. Verify final helper values against declared paths. Reject invalid paths, invalid
+   kinds, invalid projected values, missing records, or use after context closure.
+   Store reasons/dependencies without raw public history values.
+4. Attach phase diagnostics and stop on reported errors even with nil return. A
+   non-nil return adds generic safe failure only if no error was already reported.
+   Leave callback panics as programming failures. Remove the resolver options gate.
+
+Acceptance:
+
+- Execute the feature's timeout and dynamic artifact examples against real code.
+- Test explicit-null/zero preservation, baseline replacement for absent input,
+  normalization of explicit input, map writeback, unrelated pointer/path mismatch,
+  unrecorded mutation, equal-value intent, repeated helper writes, and subtree removal.
+- Mutating result configuration cannot change resolution snapshots/history. No
+  hooks or errors expose secrets. Independent map traversal yields stable records.
+
+## Stage 9 — Add semantic validation and finish the single-source lifecycle
+
+Status: pending.
+
+Read: feature [diagnostic callbacks](initial.md#resolution-and-validation-callbacks)
+and [schema reports](initial.md#schema-checking-and-secret-classification).
+
+Tasks:
+
+1. Reuse `Validation` for post-resolution schema-aware application reports. Validate
+   primary and related paths, supply nearest source locations for omitted values,
+   and collect multiple independent errors/warnings.
+2. Detect validator mutation through projection and fail without publishing a
+   result. Close contexts after callbacks and remove the validator options gate.
+3. Finalize all single-source metadata after resolution/validation success. Ensure
+   failure carries only safe diagnostics and sources, not a partially usable config.
+4. Add executable examples for minimal/defaults-only loading, related-path duplicate
+   errors, version guidance, and resolution followed by validation. Document actual
+   hook lifecycle and private-field/runtime-state boundaries.
+
+Acceptance:
+
+- Hook-order tests prove later phases never run after prior errors; warnings continue.
+- Test missing fields, dynamic keys, related locations, validator mutation, and safe
+  hook failure fallback. Compile and run representative examples, not API stubs.
+- Compare value, explicitness, origin, and history after a complete single-source load.
+  Baseline passes. Imported/profile loading is still deliberately unavailable.
+
+## Stage 10 — Implement strict document composition
+
+Status: pending.
+
+Read: feature [composition table](initial.md#composition-and-overlay).
+
+Tasks:
+
+1. Implement a private strict composer over ConfigKit document trees. Recurse only
+   mapping pairs; treat any other present overlap as a collision, including equal
+   values, null, and sequences. Empty mappings contribute no children.
+2. Retain both locations for collisions and aggregate composite container origins.
+   Preserve input order only for traversal/diagnostics, never implicit precedence.
+3. Build provenance updates alongside composition; no subsequent pass may invent
+   an arbitrary single source for a multi-source mapping. Avoid mutating input trees.
+4. Keep the composer private and test it through the smallest stable private
+   boundary; public file graph wiring belongs to subsequent stages.
+
+Acceptance:
+
+- Exercise each operation-table category plus ancestor/scalar conflicts, empty maps,
+  sequence collisions, equal-value duplicates, and disjoint named entries.
+- Duplicate diagnostics identify both exact source locations; retained inputs and
+  origins survive without aliasing. No default layer participates in strict collision.
+
+## Stage 11 — Implement explicit file-import traversal
+
+Status: pending.
+
+Read: feature [import graph](initial.md#explicit-imports-and-source-identity).
+
+Tasks:
+
+1. Parse the reserved root directive and `path` declarations strictly; preserve
+   import-site locations and strip directives before application decode. Recognize
+   `glob` declarations but retain an explicit temporary unsupported gate until Stage 12.
+2. Resolve paths from their declaring source, canonicalize symlinks, enforce root
+   containment, and perform deterministic depth-first traversal. Keep active-stack
+   and visited indexes distinct. Track both routes for cycles and repeated imports.
+3. Read accepted sources once; enforce count/depth/aggregate budgets. Optional paths
+   suppress only target nonexistence. Invalid directives, directories, unreadable
+   files, canonicalization failures, and out-of-root targets remain errors.
+4. Reject schema-version contributions from fragments before composition, even when
+   no main-file version exists. Ensure ancestor replacements of a nested version
+   path cannot smuggle a fragment version past the placement rule.
+5. Wire composition and version checking into `Load`, removing the import gate for
+   supported path declarations. Document the temporarily limited import boundary.
+
+Acceptance:
+
+- Temporary-directory fixtures cover nested relative paths, list-order traversal,
+  symlink aliases, full-cycle routes, diamond repeats, optional versus required
+  missing files, out-of-root traversal, regular-file checks, and budget boundaries.
+- Version placement and duplicate value diagnostics retain their owning sources.
+- Bytes/reader imports require explicit base/root; URI labels never imply a path.
+
+## Stage 12 — Complete glob imports and filesystem option handling
+
+Status: pending.
+
+Read: feature import syntax and source-context API.
+
+Tasks:
+
+1. Implement segment glob matching with sorted slash-relative results and no `**`.
+   Use an error-reporting directory walker: do not use a convenience glob API that
+   silently swallows permission/I/O errors. Reuse Stage 11 identity and traversal.
+2. Reject invalid patterns before walking; canonicalize and contain visited
+   directories/targets, including symlinked directories. Optional zero matches
+   must not conceal a walk or permission failure. Bound accepted candidates before
+   accumulating unbounded source work; charge directory-entry visits to their
+   separate traversal budget, including entries that do not match the pattern.
+3. Validate all filesystem context combinations, profile-name-independent root setup,
+   absolute import/URL rejection, and selected-file versus imported-file identity.
+   `default` means no active profile for `NoSource` checks.
+4. Remove the glob gate. Exercise explicitly declared `conf.d/*.yml` without adding
+   any implicit directory scan or extension fallback.
+
+Acceptance:
+
+- Fixtures cover lexical order, multiple declarations, overlapping matches, matches
+  that are directories, hard links as separate paths, symlink escapes, unmatched
+  optional/required patterns, invalid patterns, and visible permission failures.
+- Repeated runs produce identical source IDs, composition, and diagnostic ordering.
+  Full import contract and baseline pass.
+
+## Stage 13 — Implement profile overlays and wire final precedence
+
+Status: pending.
+
+Read: feature [profiles](initial.md#profiles), composition/overlay table.
+
+Tasks:
+
+1. Implement private recursive overlay with atomic scalar/sequence/null and
+   cross-kind replacement. Record overridden definitions as the operation occurs.
+2. Resolve exactly one constrained profile name to its fixed `.yml` path under the
+   root, with canonical containment. Empty/lowercase `default` means no profile;
+   selected missing profiles fail. No path inputs, inheritance, or extension fallback.
+3. Reject profile directives and schema-version contributions before overlay.
+   Also reject an ancestor scalar/null replacement that would remove a declared
+   nested schema-version value; an unchanged nested version supplied by a profile
+   is still forbidden.
+4. Apply profile after base graph composition/version checking and before defaults
+   and decoding. Remove the last optional-feature gate from `Load`.
+
+Acceptance:
+
+- Profile scalar, mapping sibling, full sequence, empty sequence, null, and cross-kind
+  replacement cases obey the operation table. Defaults remain lower precedence.
+- Removed descendants disappear from effective presence/origin, remain explainable
+  through history, and cannot inherit stale sequence indices.
+- Base YAML policy errors are not masked by profiles; effective-tree schema checks
+  and all profile graph/version restrictions pass. Baseline passes.
+
+## Stage 14 — Verify layered provenance and final snapshot agreement
+
+Status: pending.
+
+Read: feature [presence](initial.md#paths-and-presence),
+[history](initial.md#provenance-diagnostics-and-redaction), metadata API.
+
+Tasks:
+
+1. Integrate default, base, profile, and resolution records into a coherent query
+   index; reuse prior-stage provenance rather than reconstructing sources from values.
+2. Finalize histories for replaced containers, empty mappings, deleted paths, and
+   whole-sequence changes. Retain history lookups for removed paths without claiming
+   those paths exist in the effective snapshot. Canonicalize reasons/dependencies.
+3. Test composition/profile/resolution interaction through one representative public
+   workflow and fill only uncovered cross-layer cases. Keep operation-table coverage
+   in its owning tests instead of duplicating it all through `Load`.
+4. Update metadata reference docs with concrete composite and synthetic origin cases.
+
+Acceptance:
+
+- A configured value equal to its default remains explicit; derived values do not.
+- Final typed projection, lookup/null state, effective origins, source order, and
+  histories agree after parent and sequence replacement plus resolution.
+- All returned metadata/source/diagnostic nested values are detached, and successive
+  or concurrent independent loads cannot contaminate one another. Baseline passes.
+
+## Stage 15 — Implement secret classification and effective rendering
+
+Status: pending.
+
+Read: feature [redaction](initial.md#provenance-diagnostics-and-redaction),
+[secret predicate](initial.md#schema-checking-and-secret-classification), result API.
+
+Tasks:
+
+1. Combine descriptor secret tags with the application path predicate over current
+   and historical paths. A secret container marks descendants. Apply dependency
+   propagation to a fixed point so callback/map submission order is irrelevant;
+   cycles in dependency annotations terminate and propagate sensitivity conservatively.
+2. Ensure replaced/deleted secret definitions remain classified. The predicate can
+   add sensitivity, never remove tagged/historical sensitivity. Do not expose raw
+   retained values through debug formatting, diagnostic wrappers, or public records.
+3. Add `Result.WriteEffective` over the frozen canonical snapshot. Sort mapping keys
+   by decoded string, preserve sequence order, use two-space indentation and a final
+   newline, and substitute the quoted string `[REDACTED]` at secret roots. Preserve
+   supported nil/empty distinctions. No unredacted option or implicit logging.
+4. Marshal only an adapter tree made from the safe projection; never call application
+   YAML marshal hooks. Finish all text-codec projection work before returning a
+   successful load. Writer errors may leave partial redacted output.
+
+Acceptance:
+
+- Canary secrets stay absent from ConfigKit-rendered errors, warnings, metadata,
+  effective YAML, and histories across parse/decode/hook failures and overwritten
+  definitions. Check ordinary and Go-syntax formatting of metadata/error wrappers
+  so private histories cannot leak. Typed `Result.Config`, including application
+  formatting of it, is the explicit trusted boundary.
+- Test secret containers, dynamic paths, dependencies, cycles, custom scalar values,
+  snapshot mutation isolation, and writer failures. Document safe application message
+  responsibility and that redacted output is not promised reloadable.
+- Render actual resolved values including defaults/normalization; baseline passes.
+
+## Stage 16 — Prove consumer ergonomics and migration contracts
+
+Status: pending.
+
+Read: feature consumer-fit table, module options, external values, and both criteria.
+
+Tasks:
+
+1. Create maintained executable examples and minimal synthetic fixtures for four
+   consumer shapes: Narratio-like defaults/partial imports/artifacts; Notarius-like
+   object bindings/string versions/dynamic options; WeatherReporter-like defaults-only
+   and report/module options; Seriatim-like ordered speaker/autocorrect rule files.
+2. Generalize the relevant copied Narratio tests and helpers, recording source
+   revision and notices. Use actual consumer field patterns but no private data or
+   dependencies on sibling application modules. Keep runtime algorithms in example
+   application functions rather than adding library APIs for them.
+3. Compare preserved outcomes against consumer evidence and list intentional changes:
+   explicit empty/null behavior, canonical objects, versions, integer representation,
+   strict YAML, and retired environment/CLI configuration overrides. Verify imports
+   and profiles do not absorb application LLM/pipeline-profile semantics.
+4. Convert every feature-roadmap API example into a real compiled/run example or a
+   focused behavioral test; document the happy path in README and `docs/consumers/`.
+   Verify dynamic options without YAML reserialization or generic subtree conversion.
+5. Add a short consumer-fit assessment to current consumer guidance: what plumbing
+   disappears, what domain code remains, and what downstream migrations are still
+   separate tasks. Do not use line-count targets as acceptance criteria.
+
+Acceptance:
+
+- All maintained YAML examples load as intended; negative migration fixtures fail
+  with stable classifications and useful paths, not copied raw parser text.
+- Simple consumers need no imports/profiles and no custom parser hooks. Complex
+  examples preserve domain meaning under the same conventions.
+- No external configuration reads, environment mutation, LLM calls, remote services,
+  or sibling checkout requirements occur in default tests. Baseline passes.
+
+## Stage 17 — Harden boundaries and verify independent loads
+
+Status: pending.
+
+Read: testing policy, ownership/projection, import limits, redaction guarantees.
+
+Tasks:
+
+1. Add bounded fuzz targets for path parsing, YAML translation, and document
+   composition/overlay invariants. Retain deterministic regression seeds for any
+   defects found; avoid a large permanent corpus of redundant cases.
+2. Add parallel independent-load coverage for shared read-only definitions and
+   fresh defaults; exercise defensive-copy and custom-scalar paths under the race
+   detector. Source/reader single-use behavior must be race-safe, not silently reused.
+3. Review private limit enforcement through reads, traversal, projection, and
+   rendering. Add only missing consequential boundary tests. Prevent unbounded
+   expansion through permissive aliases/providers; these remain unsupported.
+4. Run default suite, vet/build, and `go test -race ./...`. Run each selected fuzz
+   target with `-run='^$' -fuzz=<exact-name> -fuzztime=30s -parallel=2` in its owning
+   package. Record exact names/commands in the developer guide. No fuzz loop in CI
+   is required beyond deterministic seeds unless separately configured.
+
+Acceptance:
+
+- Race checks and bounded fuzz runs pass on a supported native platform, with any
+  discovered defects fixed and retained at their narrowest durable test boundary.
+- No unexplained test skips, mutable shared defaults, raw-secret render paths,
+  unsupported-feature gates, or stale placeholder APIs remain.
+- A fresh offline baseline after dependency acquisition passes without reading any
+  developer-specific configuration path.
+
+## Stage 18 — Prepare release validation and reconcile documentation
+
+Status: pending.
+
+Read: release policy in full and [release-readiness gaps](release-readiness.md).
+
+Tasks:
+
+1. Create `docs/release.md` with executable commands for the existing origin/main,
+   module path, source-only tags, candidate recording, branch/tag/cleanliness guards,
+   precise publication refs, and clean consumer module retrieval/build verification.
+   Include private-host access prerequisites without credentials or environment dumps.
+2. Implement `scripts/release-check.sh` as a fail-fast source-candidate checker.
+   It must perform module hygiene (tidy in a disposable copy and diff manifests),
+   cached dependency verification, offline default tests, native race tests, vet,
+   native builds, `CGO_ENABLED=0` cross-builds for the four supported targets,
+   formatting/whitespace checks including untracked files, maintained examples,
+   and local documentation-link checks. Acquire toolchains/dependencies explicitly
+   before running the offline checker. Use the baseline toolchain and a newer
+   available supported toolchain when assessing the declared Go support range.
+3. Make links to sibling consumer source files optional only when those checkouts
+   are absent in isolated release environments: verify local links to current
+   ConfigKit owners strictly, preserve recorded external revision references, and
+   document the external-reference applicability instead of requiring private repos.
+4. List manual release gates: compatibility/migration review, secrets/unintended-file
+   inspection, platform execution evidence where native hosts differ, roadmap
+   retirement, release-note review, exact remote commit/tag guards, and post-publication
+   retrieval. Static-analysis baseline is vet; do not silently claim a linter was run.
+5. Exercise the checker on current source and reconcile all documentation links,
+   API comments, `docs/config.md`, consumer examples, and component inventory.
+   Replace implemented feature-roadmap definitions with current contract links;
+   repair ADR navigation without changing accepted rationale. Keep this execution
+   plan until release preparation retires completed roadmap material.
+6. Update release-readiness evidence for the actual runbook/checker. Keep any
+   unvalidated release-candidate/publication gap explicit and retain its bootstrap
+   guard until the maintainer closes it. Do not fabricate release notes for an
+   unpublished candidate or mark remote verification completed. No publication is
+   authorized by this stage.
+
+Acceptance:
+
+- The checker actually passes applicable automated gates, and the runbook explicitly
+  owns every manual/inapplicable gate required by policy. No nonexistent commands
+  are presented as operational. Any missing native-platform or publication evidence
+  remains an explicit release gate, with its owner and closure check recorded.
+- Current docs describe implemented behavior; no duplicated roadmap contracts or
+  stale deferred-plan statements remain. All feature areas below have completion
+  evidence, and both architecture success criteria are demonstrated locally.
+- No tag, push, downstream migration, or release publication occurred as part of
+  implementation. Maintainer release preparation remains a separate procedure.
+
+## Coverage and completion audit
+
+Use this map to ensure completion means the whole feature roadmap, not merely a
+working loader. The linked feature sections own semantics; stages own work.
+
+| Feature area | Owning stages |
+| --- | --- |
+| Extraction, ordinary consumer types, package boundaries | 1, 4–6, 16 |
+| YAML subset, precise numeric syntax, safe source failures | 2–3, 6 |
+| Paths, presence, origins, definition history | 2, 6–8, 10, 13–14 |
+| Static defaults, null, container rules, snapshot ownership | 5–7, 14, 17 |
+| File/bytes/reader/defaults-only inputs and filesystem context | 3, 7, 11–12 |
+| Version hook and graph-placement restrictions | 7, 11, 13 |
+| Strict imports, glob ordering, identity, limits, collisions | 10–12, 17 |
+| Named profiles, precedence, atomic replacement | 13–14 |
+| Recorded normalization/derived defaults and hook errors | 8–9, 14 |
+| Semantic validation, related diagnostics, safe messages | 2, 7–9 |
+| Secret tags/predicate/dependencies and effective inspection | 5, 15, 17 |
+| Module-specific options, consistency and intentional migrations | 16 |
+| Runnable docs, examples, source-release checks and runbook | 1, 16, 18 |
+| Deferred-feature boundaries | Every stage; final audit in 18 |
+
+At final handoff, report completed stages, exact checks, current documentation
+owners, source-extraction evidence, and remaining release/consumer migration work.
+A green unit suite alone is not proof that examples, metadata safety, filesystem
+behavior, documentation, or both success criteria are satisfied.
+
+## Open Questions
+
+No unresolved product or API question blocks the ordered stages above. The
+implementation decisions explicitly select the module identity, Go baseline,
+parser, limits, supported platform targets, metadata shapes, and consumer scope.
+Do not reopen settled feature policies solely to preserve a legacy convention.
+
+Two external execution facts must still be verified when their stage runs; they
+are not authorization to silently weaken acceptance:
+
+1. **Is a native Linux runner available for the declared support matrix?**
+   Recommended: execute filesystem/race tests on native Linux and macOS before
+   release; cross-builds alone cannot verify symlink/permission behavior. Stage 18
+   can prepare the checker on the available host while keeping missing native
+   evidence as an explicit release gate. Alternative: formally narrow the initial
+   supported platforms to those with execution evidence, with a documented scope
+   decision; this is more honest than asserting untested support, but less useful
+   for the likely server consumers. Neither option blocks the platform-independent
+   stages or permits claiming a skipped native check passed.
+2. **Are the authoritative host and module retrieval credentials available at
+   publication time?** Recommended: verify the exact version from a clean consumer
+   module after authorized publication using the host's normal access mechanism;
+   keep credential setup outside ConfigKit and out of tests. Alternative: use an
+   explicitly configured internal Go proxy mirroring the exact immutable tag and
+   verify its source identity; it adds infrastructure but can suit restricted
+   environments. Local replacement directives are useful for development but are
+   not a viable substitute for release retrieval verification.