Files
ConfigKit/docs/roadmap/implementation.md

1421 lines
94 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# ConfigKit Initial Implementation Plan
## Status and ownership
Status: implementation in progress; Stages 1–9 are complete and later stages are
pending.
Audience: a `gpt-5.6-terra` coding agent implementing one stage per prompt, in
ascending order. Each stage depends on every earlier stage being complete.
Do not combine stages merely because they share files. Finish the requested
stage's acceptance checks and handoff before moving to the next requested stage.
The [feature roadmap](initial.md) owns the desired end-state and consumer intent.
This document owns execution decisions, tasks, acceptance gates, and progress.
Read [development](../development.md), all four policies
([architecture](../policy/architecture.md), [documentation](../policy/documentation.md),
[testing](../policy/testing.md), [release](../policy/release.md)), and the relevant
[ADRs](../development.md#task-specific-reading-guide) before implementation.
Both [success criteria](../policy/architecture.md#success-criteria) apply to every
stage. This plan does not authorize feature expansion or downstream application
releases.
## Execution rules and stage handoff
1. Inspect the actual worktree, prior stage evidence, relevant source, and tests.
Preserve unrelated work. This repository initially has documentation only.
2. Read the linked feature sections for the requested stage. Follow links to
current contracts if earlier work has moved a definition out of the roadmap.
3. Extract applicable Narratio and WeatherReporter code and cases before writing equivalent new
mechanisms. Record source revision, symbols, and adaptations.
Do not import sibling applications or their internal packages into this module.
4. Implement only the stage boundary. Partial internals may be tested directly;
do not export placeholder APIs, silently ignore unsupported options, or claim
that later features work. Introduce public operations only when their stated
supported boundary is functional. No release occurs between these stages.
5. Run focused checks while iterating, then the stage baseline. Add durable tests
at the owning boundary; do not replicate every case through `Load`.
6. Update Go reference comments, implemented configuration contracts, component
inventory, and examples in the same change as behavior. Move implemented
normative text from the feature roadmap to its current owner, leaving links.
Keep this plan's tasks and completion evidence here, not in the feature roadmap.
7. Set the stage's status to complete only after its acceptance gates pass. Append
a short evidence entry under that stage: files/contract links, extraction
references, checks and outcomes, deviations, and remaining later-stage work.
Do not treat compilation against stubs as runtime acceptance.
Baseline after Stage 1: `go test ./...`, `go vet ./...`, `go build ./...`, and
`git diff --check`, plus inspection of new files, documentation links/fragments,
and formatting. These are future implementation commands; Stage 1 establishes
them in the developer guide. Tests must run without network access or credentials
after explicit dependency acquisition. Do not put network calls in default tests.
For a failed gate, fix the implementation or explicitly resolve a contract issue;
do not change expectations solely to get a pass. Record external tool/platform
limitations accurately. Do not mark a required check passed if it was skipped.
## Implementation decisions
These decisions close choices left illustrative in the feature roadmap. They
are instructions for implementation, not additional configuration conventions.
Promote their externally observable parts into current contracts when implemented.
| Area | Decision and reason |
| --- | --- |
| Module and package | Module `gitea.maximumdirect.net/eric/ConfigKit`, matching the existing origin's case; root package `configkit`. No `cmd` application. |
| Go baseline | `go 1.25.0`; use a currently supported patched toolchain at or above that minimum for development, and verify minimum-version compatibility separately. The existing Narratio/Seriatim baselines and local 1.25.5 toolchain support this language baseline. No toolchain directive forcing a newer consumer toolchain. |
| Parser | Pin `go.yaml.in/yaml/v4 v4.0.0-rc.6` initially, inside `internal/yaml`. Use the v4 streaming loader and dumper with explicit internal options. This selects upstream's active development line while preserving the node-oriented extraction approach. Do not inherit its scalar coercions. The [exact RC reference](https://pkg.go.dev/go.yaml.in/yaml/v4@v4.0.0-rc.6) documents the selected dependency. |
| Internal representation | A ConfigKit-owned tree in `internal/document`: ordered mapping entries, sequences, typed scalar kind plus original numeric spelling, null, key/value locations, and source ID. It imports neither the public package nor the YAML parser. The YAML adapter translates into it. |
| Dependency direction | Public package coordinates document operations and reflection binding. Only `internal/yaml` and its adapter tests import upstream YAML packages/plugins. The adapter imports `internal/document` and translates parser errors into ConfigKit-owned internal records; document code never imports parser or public package. Adapter signatures expose no upstream types to other ConfigKit packages. Keep composition, binding, provenance, hooks, and rendering coordination in root-package private files until a demonstrated boundary needs another package. |
| Public spelling | Implement the feature roadmap's `Load`, `Definition`, `Options`, sources, path builders, metadata queries, hook helpers, version rule, result methods, and diagnostic methods as written. No functional-options framework or separate builder. |
| Input limits | Fixed initial ceilings: 8 MiB per source, 64 MiB total loaded bytes, depth 256 for input/projected trees, 64 active import edges, and 1,024 accepted sources. Allow at most 1,000,000 input nodes in aggregate and 1,000,000 nodes in each composed/projected tree; copying a snapshot does not consume the input budget again. Bound glob directory-entry visits separately at 1,000,000 per load. Include the selected profile in source/byte counts. Root depth is zero. No public limit-tuning options initially. Test limit mechanisms with smaller private test parameters. |
| Filesystem support | Initial supported targets: Linux and macOS on amd64/arm64. Test real symlink semantics on those systems. No Windows path-compatibility promise in this release. Inputs use slash-separated relative import syntax and native paths for selected file/root/base options. |
| Schema implementation | Build one reflection descriptor graph per load, reuse it throughout that load, and avoid global mutable caches initially. Recursive type definitions are permitted when actual values are acyclic. |
| Working configuration | Clone the defaults factory's result before applying input. Decode explicitly present nodes into that detached baseline according to schema descriptors; never round-trip defaults through YAML. Snapshot final values independently. |
| Error boundary | Safe library diagnostics with phase-specific codes; no raw parser/custom-hook error string or raw-value wrapper chain. `LoadError` supports `errors.As`; there is no initial `Unwrap` promise. I/O classifications are exposed as diagnostic codes. |
| Consumer integration scope | Build copied, sanitized consumer-shaped models/fixtures and executable examples in ConfigKit. Actual changes to Narratio, Notarius, WeatherReporter, and Seriatim are separate consumer migration tasks. Do not claim those apps are migrated when these local examples pass. |
| Publication scope | Prepare library source and a release runbook/checker. Do not tag, push, publish, or invent a completed release during these stages. Release-candidate validation/publication follows the adopted release procedure as a separate maintainer action. |
| Test comparisons | Use `github.com/google/go-cmp/cmp`, initially module `github.com/google/go-cmp v0.7.0`, for useful structured test comparisons. Add the requirement when first imported by tests. No production imports or runtime use; do not equate nil and empty or reorder sequences where those differences are contractual. |
The selected version is a prerelease, deliberately accepted for initial
development. Do not use `@latest`, silently change the pin, or depend on untagged
upstream changes. Reassess the available v4 release before ConfigKit publication;
a newer RC or final release requires an explicit dependency update and rerunning
the adapter, YAML-policy, diagnostics, limits, and rendering regressions. A final
upstream version is preferred when it passes those checks; if the RC is retained,
record that decision in release readiness and compatibility review.
Follow the feature's [upstream isolation contract](initial.md#yaml-dependency-and-public-api-isolation).
Translate upstream API changes internally without changing ConfigKit declarations
or consumer requirements. Never re-export upstream aliases/options/plugins, expose
parser errors through wrapping, or pass parser-created objects through `any`.
Keep ConfigKit's semantic checks independent of upstream convenience options.
The module requirement records the tested dependency; it does not lock a consuming
application's entire module graph. Do not promise compatibility with arbitrary
unverified upstream versions or add a selectable-parser framework.
### Updated extraction map
Use the source revisions and adaptation policies owned by the feature's
[foundation](initial.md#existing-implementation-foundation) and
[secret bootstrap](initial.md#secret-references-and-explicit-bootstrap) sections.
Paths below are relative to the named sibling repository. Reinspect them in
Stage 1 and record the exact revision actually copied. The maintainer is the sole
author of the relevant application code and has authorized its reuse and relicensing
as needed for ConfigKit; no separate license check is required for this extraction.
Keep source provenance for technical traceability. This map assigns reuse
work to existing stages; it does not change stage ordering or public contracts.
| Stages | Source code and tests | Extraction instruction |
| --- | --- | --- |
| 3 | Narratio `internal/config/composition.go`, `load.go`, `loader_strict_test.go`, `composition_test.go` | Extract source parsing and trailing-document regressions. Translate to the private v4 adapter and ConfigKit YAML policy; do not copy raw parser error propagation. |
| 10 | Narratio `composition.go`, `composition_test.go` | Adapt `mergeAdditiveCompositions`, claim validation and recursive merging. Preserve collision/source tests, replacing empty-mapping rejection with ConfigKit's operation table. |
| 11–12 | Narratio `pipeline_composition.go`, `pipeline_composition_test.go` | Reuse explicit import processing and duplicate/source cases. Build recursive graph/glob support and bounded containment; replace root-only directives, extension restrictions, and `os.SameFile` identity rules. |
| 13 | Narratio `mergeOverlayComposition`, `mergeOverlayNodes`, `selectPipelineProfile`, `loadPipelineProfileOverlays`, `pipeline_profiles_test.go` | Adapt overlay mechanics and independence tests. Replace null/kind rejection and declared/all-profile loading with ConfigKit's fixed selected-profile contract. |
| 14–15, 19 | Narratio `effective_pipeline.go`, composition semantic records, `pipeline_profiles_test.go`, `internal/app/config_commands_test.go` | Extract deterministic effective values, composite ownership and safe inspection cases. Add locations, retained history and generic redaction; remove YAML round trips and domain-specific projection. Do not copy diff/digest APIs. |
| 16 | WeatherReporter `internal/config/secrets.go`, secret cases in `config_test.go`; Narratio `internal/app/secrets_env.go`, `secrets_env_test.go`, `internal/fileops/read_bounded.go` | Combine WeatherReporter's read/apply separation and one-newline handling with Narratio's preserve-existing and bounded-read mechanisms. Adapt both to the agreed structured-report state machine. |
| 20 | Narratio inspection/composition/profile tests; WeatherReporter `config_test.go`, `overrides_external_test.go` | Prove side-effect-free inspection, explicit-empty domain validation, loaded/constructed consistency, and nonmutating option normalization with sanitized consumer-shaped fixtures. |
In every owning stage, record copied symbols/cases and intentional expectation
changes in the extraction inventory. Retain domain algorithms in consumer examples;
do not copy application frameworks, CLI commands, logging, digests, or provider
registries into the public library. Existing upstream types, YAML reserialization,
and partial provenance are adaptation work, not reasons to weaken the target.
### Standard-library mechanisms and custom policy
These are implementation choices for the existing contracts, not additional
public extension APIs. Apply each row in its owning stages. The Go 1.25 baseline
supplies the listed primitives; no additional runtime dependency is needed.
| Work and stages | Selected mechanism | ConfigKit work and limits |
| --- | --- | --- |
| Source reads (3, 11) | `os.Open`, `io.LimitReader`, `io.ReadAll` | Read at most `min(perSourceLimit, remainingTotalBudget) + 1` bytes; reject excess before parsing. File size is an optional early check, never the sole enforcement. Preserve read failures rather than accept partial input. |
| Source ownership (3) | `bytes.Clone`, `bytes.NewReader`, `strings.NewReader` | Reject oversized byte arguments before cloning, retaining the constructor error for `Load`. Close only ConfigKit-owned files. Keep a shared private reader-state pointer with `atomic.Bool.CompareAndSwap` to claim a reader before consumption; copying `Source` must not copy an in-use atomic or create another claim. |
| Identity and containment (11–13) | `filepath.Abs`, `EvalSymlinks`, `Rel`; `os.OpenRoot`/`Root.Open` | Canonicalize root and targets, reject a relative result that is absolute, `..`, or begins with a parent component, and open the canonical root-relative target through `Root`. Preserve declared spelling separately. Do not use string-prefix checks, `os.SameFile`, or `os.DirFS` as a confinement guarantee. |
| Glob matching (12) | `path.Match` | Use the feature's matching grammar and a small syntax check for forbidden recursive `**`, respecting escapes and character classes. Do not write another wildcard matcher or treat backslash as a native separator. Validate malformed patterns even when directories are empty or absent. |
| Glob expansion (12) | `os.File.ReadDir(n)` with positive bounded batches | Write a pattern-directed walker with explicit error propagation, canonical containment, and a budget counting every examined entry. Process returned entries and errors correctly, including EOF. Sort bounded candidates before recursion and final accepted relative matches before source traversal. Avoid `filepath.Glob`/`fs.Glob` error suppression and unbounded whole-directory reads. |
| Numeric conversion (3, 6) | `strconv.ParseInt`/`ParseUint` with base 10, `ParseFloat`, `math.IsNaN`/`IsInf` | Check ConfigKit lexical syntax first. Use the destination bit width, including `int`/`uint`; parse floats directly at 32/64 bits. Do not route wide integers through floats. Keep boolean/null recognition in simple switches; `strconv.ParseBool` accepts forms outside our policy. |
| Duration/text scalars (5–6) | `time.ParseDuration`, `Duration.String`, `encoding.TextMarshaler`/`TextUnmarshaler` | Enforce string-only input and paired codecs, canonical projection, detached reconstruction, and safe diagnostics. Date-looking text remains text. |
| Field binding (4) | `reflect.Type`, `StructField`, `StructTag.Lookup`, ordinary reflection accessors | Build our descriptor graph by inspecting declared fields and explicit inline tags. Do not use `reflect.VisibleFields` as the binding algorithm: Go promotion can hide conflicts and does not express our inline rules. No `unsafe` access to private fields. |
| Detached copies and comparison (5, 8–9) | `maps.Clone`, `slices.Clone`, `bytes.Clone` where shallow copies suffice | Recursively clone supported configuration with the descriptors; shallow clones do not detach nested objects. Compare canonical trees with a focused changed-path traversal, preserving atomic sequences and custom scalars. Neither `reflect.DeepEqual` alone nor runtime `go-cmp` supplies this audit. |
| Semantic paths (2) | `strings`, `strconv` for validated index tokens | Implement the small strict JSON Pointer codec and key/index representation. Do not use filesystem normalization or URL escaping/percent decoding. |
| Ordering and import graphs (2, 10–14) | `slices.Sort`/`SortFunc`, `cmp.Compare`, ordinary maps and slices | Define total ordering keys for observable output. Implement bounded DFS with separate active/visited maps and inclusion routes. Keep per-load state local; no global cache, `sync.Map`, or parallel import scheduler initially. |
| Safe diagnostics/output (2–3, 6, 15) | `errors.Is`/`As`, `fs.ErrNotExist`/`ErrPermission`, `fmt`, `io.WriteString`, `strconv.Quote` | Classify underlying failures and emit ConfigKit-owned records. Never forward `strconv.NumError`, duration-parser errors, or other value-bearing error text/unwrap chains. Quoting escapes controls but does not redact secrets. Check writer errors and short writes; flush buffered output if used. |
| Test support (2 onward, 21) | `testing`, `t.TempDir`, native fuzzing, `testing/iotest` | Use standard fragmented/error readers and real filesystem fixtures. Add small custom close-observers or failing writers only where needed. Use test-only `go-cmp` for structured expectations, not a second test framework. |
| Import-boundary checks (3, 20, 22) | `go/parser`, `go/ast`, `strconv.Unquote` | Inspect Go imports, including test files and platform-specific source, rather than regex-match source text. Permit YAML imports only in the adapter and its tests, and `go-cmp` imports only in test files. |
For confined access, canonicalization before `Root.Open` preserves contained
absolute symlink targets that `Root.Open` would reject if passed their original
symlink path. This is an implementation of the feature's stable-input contract,
not a promise of atomic resolution against hostile concurrent replacement.
Retain regular-file checks and safe error classification. Do not introduce custom
syscall-level confinement, path discovery, or a public filesystem-provider API.
### Secret, override, and history implementation bounds
Follow the feature contracts for [secrets](initial.md#secret-references-and-explicit-bootstrap),
[CLI overrides](initial.md#cli-overrides), and
[explanation](initial.md#historical-value-explanation). Introduce these public APIs
only in Stages 16–19. Earlier stages cover ordinary types and file/profile layers;
they must not expose an unresolved secret as a completed load or silently accept
an override option. Extend the shared descriptor/projection graph for `Secret`
in Stage 17, before ordinary struct/text handling; do not add a parallel binder.
- Bootstrap uses `os.LookupEnv`/`os.Setenv`, `os.Open` and bounded `ReadDir`/reads,
`filepath` canonicalization, and `strings.TrimSuffix` for exactly one newline.
Reuse WeatherReporter's actual code and tests, adapting policy rather than
importing its package. Validate names with a small ASCII check or its existing
regular expression. Keep a private environment/file-access seam for failure tests;
do not expose a general environment provider or public mock filesystem.
- Bootstrap ceilings: 1,024 entries, 1 MiB per eligible file, 8 MiB total eligible
bytes. Bound enumeration before sorting, read at most remaining budget plus one,
and reject excess before writes. Skipped existing files consume entry budget,
not content budget, and are not opened. These budgets are independent of YAML
inputs. Errors use `SecretsError` with `Error()` and `Diagnostics() []Diagnostic`;
no raw unwrap chain. `LoadSecretsDir` always returns its detached report alongside
an error. Diagnostics include safely quoted canonical file paths in messages; no config source
ID is fabricated for bootstrap. Reports expose no credential bytes or hashes.
- Stage 16's preflight/apply state machine owns report status assignment and
`ExistingEmpty`. On preflight failure, no `Loaded` entries are possible. On apply
failure, stop with earlier loaded/skipped statuses intact. Never log implicitly.
- Stage 17 reads each referenced name once per load, including failures. Do not
cache across loads, mutate the environment, or retain it in metadata. Application
hooks cannot inject a resolved credential through a public constructor. Store
names in projected nodes and resolved strings only in private `Secret` state.
- Override input shares the depth/node ceilings with document input; count its
nodes toward the aggregate budget. Bound string bytes to 8 MiB per record and
64 MiB combined with source bytes; at most 1,024 records, also counting their
synthetic sources toward the source ceiling. Validate and detach before applying;
no second parser or `flag` framework. Use `reflect` only to reject unsupported
values/detect cycles and standard numeric formatting to create typed scalar nodes.
- Retain structurally shared immutable document roots and audited domain deltas
for history. Cap retained unique history nodes at 4,000,000 per load, including
projected baseline and resolution snapshots; fail safely if exceeded. Explain
projects only the queried subtree for each relevant operation. Do not eagerly
store every rendered path/history combination or expose private roots.
- Test bootstrap real environment integration serially with `t.Setenv`; use the
private seam for unset names, preflight errors, write failures and deterministic
races between preflight and apply. Test load lookup using controlled names and
restore environment; no tests depend on real credentials. `go-cmp` remains test-only.
### Metadata and diagnostics layout
Use opaque `Path`, `Source`, `Metadata`, `Resolution`, and `Validation` state.
Expose detached value records for inspection, with these initial fields:
| Type | Fields |
| --- | --- |
| `Location` | `SourceID string`, `Line int`, `Column int`; zero coordinates mean unavailable/synthetic |
| `SourceInfo` | `ID string`, `Label string`, `Path string` (canonical local path or empty), `Layer Layer`, `ImportedFrom *Location` |
| `Diagnostic` | `Code Code`, `Severity Severity`, `Path Path`, `Message string`, `Primary *Location`, `Related []Location`; absent primary means no known source |
| `Origin` | `Layer Layer`, `Locations []Location`, `Operation ChangeKind`; multiple locations represent a composite origin |
| `FieldInfo` | `Explicit bool`, `Null bool`, `Origin Origin` |
| `Change` | `Path Path`, `Origin Origin`, `Previous []Origin`, `Reasons []string`, `Dependencies []Path`; safe values available separately through `Explanation` |
`Layer` is a string enum: `default`, `base`, `profile`, `cli`, `resolution`. A Go zero
baseline uses layer `default` without claiming an application deliberately chose
every zero value. `ChangeKind` has the empty value for input/default origins,
`derived_default`, and `normalization`. `Severity` is `error` or `warning`.
Define exported constants with the type prefix (for example `LayerBase`,
`ChangeDerivedDefault`, `SeverityError`).
Use stable string codes grouped by `options.*`, `source.*`, `yaml.*`, `schema.*`,
`decode.*`, `compose.*`, `import.*`, `profile.*`, `version.*`, `resolve.*`, and
`validate.*`, `secret.*`, and `override.*`. Bootstrap uses `secrets.*` codes
on a separate `SecretsError`. Include distinct missing/permission/read, duplicate, invalid-path,
limit-exceeded, hook-failed, and unrecorded-change classifications where relevant.
Application reports use their phase's `error`/`warning` code, not arbitrary code
registration. Define exact exported code constants with their owning feature;
private helpers do not need new public codes for every branch.
Sources get deterministic per-load IDs `source-1`, `source-2`, and so on in
traversal order. File traversal is followed by the selected profile, then synthetic CLI sources.
`NoSource` itself creates no source ID; its overrides can create CLI sources.
Public query slices and nested slices/pointers are detached. Default fmt/error
rendering must not reveal hidden value histories.
### Clarifications required for a coherent implementation
- `ParsePath` retains textual numeric tokens until traversal knows the container;
`Keys("0")` is a mapping key and `Index(0)` is a sequence index. Reject invalid
escapes and negative indices. The zero `Path` is the root. Empty mapping keys
remain representable. Resolve parsed numeric indices without leading zeros;
the same text may address a literal mapping key when traversing a mapping.
- Path queries on missing fields return the feature-defined empty result. Hook
diagnostics permit schema-valid missing fields/map entries; malformed paths or
unknown struct fields produce a hook diagnostic. Version hooks use the untyped
composed tree and nearest known container because decoding has not run yet.
- Defaults apply to the surviving composed/profile/CLI input; intermediate overridden
nodes do not create reset markers. An effective null suppresses defaults at
its path. Missing children of an effective mapping can retain defaults.
- Applying nonempty input to a nil pointer-to-struct creates the object and uses
zero values for omitted children. No per-entry defaults are invented. Applying
a sequence allocates a replacement; it never overlays default indices.
- Support paired text marshal/unmarshal scalar methods on value or pointer method
sets, with exact `time.Duration` handled first. Treat such scalars atomically.
Their canonical text supplies projection and detached reconstruction. Never
call YAML-specific marshal/unmarshal hooks; an otherwise ordinary struct with
such a method is bound structurally under ConfigKit's rules.
- Ignore ordinary unexported and `yaml:"-"` fields; zero them in detached ordinary
struct results rather than preserve hidden mutable aliases. Paired text scalars
are the exception: their declared scalar codec owns their internal state.
Reject unsupported schema kinds and unsupported `yaml` tag options rather than
silently reinterpret them. Support `inline` and ignore `omitempty`; defer `flow`.
- Repeated helper writes to one path compare only the last helper-assigned value
with the final projected value. Coalesce reasons/dependencies. For mixed
operations, a path record is normalization if any record normalized it; otherwise
it is a derived default. Record coverage of a container includes descendants.
- Query explicitness against surviving input contributions. A normalization of an
existing path preserves its input explicitness. A newly created path is not
explicit just because the resolver copied another explicit field; dependencies
carry its explanation. Deletions remain historical, not effective entries.
- Validator mutation is a programming error detected by before/after projection;
return `validate.mutation` and no result. Do not recover user callback panics.
Retained contexts are invalid after the hook. Calls on a closed context panic
with a fixed, value-free programming-error message and cannot mutate a published
result; this is callback misuse, not a late load diagnostic.
- The initial implementation assumes stable, application-controlled local input
trees during a load. Resolve symlinks and enforce containment, but do not claim
atomic multi-file snapshots or protection against concurrent hostile filesystem
replacement. Reject directories and non-regular file targets; arbitrary streams
are accepted only through the explicit reader API.
## Stage 1 — Establish the module and extraction baseline
Status: complete.
Completion evidence (2026-09-06):
- Added module `gitea.maximumdirect.net/eric/ConfigKit` at Go 1.25.0 and the
documented root `configkit` package, with no loader API or runtime dependency.
- Added the [source extraction inventory](../internal/extraction.md), recording the
clean Narratio revision `b68435128be1e24b94fc7ec259d6e9093669fd99` and
WeatherReporter revision `5d306a202471e10fe98923a1df27d0b0fce5c666`, assigned
symbols/cases, adaptations, and application-only behavior. No sibling checkout
is a build dependency and no fixture was needed at this boundary.
- Updated the [development baseline](../development.md), [component
inventory](../internal/overview.md), and repository status. The parser remains
planned at `go.yaml.in/yaml/v4 v4.0.0-rc.6`; its module requirement remains
deferred until the real adapter in Stage 3.
- Verified `go test ./...`, `go vet ./...`, `go build ./...`, `git diff --check`,
changed local links, and the introduced file set with Go 1.26.7 on linux/amd64.
No deviations were required. Paths, diagnostics, parsing, and all later behavior
remain assigned to subsequent stages.
Read: feature [foundation](initial.md#existing-implementation-foundation),
[consumer fit](initial.md#consumer-fit-and-intentional-migrations), all policies.
Tasks:
1. Create the module and root package documentation with the decisions above.
Keep the existing repository license. Add no application command, runtime
dependencies, framework, or placeholder exported loader.
2. Reinspect Narratio's and WeatherReporter's referenced revisions and worktrees. Create
`docs/internal/extraction.md` recording source revisions/symbols and the exact
parsing/composition/import/profile/inspection/secret and default/presence cases
assigned by the extraction map. Record preserved, adapted and application-only
behavior, including current tests replacing obsolete baseline assumptions.
Copy only minimal sanitized fixtures needed by subsequent
stages; no audio, transcripts, secrets, or entire application package trees.
3. Document the planned parser pin here; add its module requirement only when the
real adapter imports it in Stage 3 so tidy does not leave unused dependencies.
4. Update the developer guide to runnable Go baseline commands and prerequisites.
Preserve roadmap/current-contract ownership and navigation. Record
the implemented root package in the overview and retain pre-feature status.
Acceptance:
- Module path/case agrees with origin, root package builds, and baseline passes.
- Extraction notes distinguish copied/generalized code from new work and domain
logic retained in consumers. No sibling checkout is required to build/test.
- No misleading usable-loader example or unrelated files enter the module.
## Stage 2 — Implement semantic paths and safe diagnostic records
Status: complete.
Completion evidence (2026-09-06):
- Implemented immutable key/index paths, strict JSON Pointer parsing, canonical
spelling, parsed numeric-token preservation, and private traversal intent in
[`path.go`](../../path.go).
- Implemented detached metadata/provenance records and empty/unknown query behavior
in [`records.go`](../../records.go) and [`metadata.go`](../../metadata.go).
- Implemented canonical diagnostic ordering, related-location normalization,
control-safe rendering, short/error writer propagation, and defensive
`LoadError` access in [`diagnostic.go`](../../diagnostic.go). No raw values,
parser errors, source parsing, or loading surface was introduced.
- Added focused tests for escaped slash/tilde/empty/dotted/numeric keys, literal
dot/percent tokens, malformed escapes, builder immutability, detached nested
records, `errors.As`, ordering independence, control escaping, and failing/short
writers. Verified `go test ./...`, `go vet ./...`, `go build ./...`, and
`git diff --check` with Go 1.26.7 on linux/amd64. No deviations were required;
metadata population and configuration lifecycle behavior remain later work.
Read: feature [paths](initial.md#paths-and-presence),
[diagnostics](initial.md#provenance-diagnostics-and-redaction), public metadata API.
Tasks:
1. Implement `Path`, builders, parsing/stringification, private traversal helpers,
and the metadata/diagnostic value types specified above. Keep records detached.
2. Implement diagnostic accumulation, canonical ordering, safe `LoadError`,
`WriteDiagnostics`, and error-code constants needed so far. Source-less errors
sort before source errors; then source order, line/column, path, code, severity,
and safe message break ties. Canonicalize related locations deterministically.
3. Render control characters in labels/messages/paths as escaped text so source
labels cannot corrupt terminal output. Never include values or parser errors.
4. Document the implemented path and diagnostic API next to declarations.
5. Apply the standard-library path, ordering, and diagnostic mechanisms above.
Introduce test-only `go-cmp` when comparing structured records; simple scalar
assertions can remain ordinary Go checks.
Acceptance:
- Round-trip escaped slash/tilde/empty/dotted/numeric keys and indexed paths;
distinguish a malformed path from a valid missing path. Preserve literal `..`
and percent-containing keys; reject malformed tilde escapes.
- `errors.As` finds `LoadError`; mutation of returned nested records cannot alter
later results. Safe writer failures propagate, including partial output.
- Reordered independent diagnostic submissions render identically; no blanket
golden snapshot of unstable prose is required. Baseline passes.
## Stage 3 — Extract source reading and enforce YAML policy
Status: complete.
Completion evidence (2026-09-06):
- Added opaque source constructors and bounded file/byte/reader acquisition in
[`source.go`](../../source.go), including byte snapshots, regular-file checks,
deterministic I/O codes, caller-owned reader lifetime, and one atomic claim
shared by source copies. `NoSource` creates no source record.
- Pinned `go.yaml.in/yaml/v4 v4.0.0-rc.6` solely behind `internal/yaml` and added
the parser-independent ordered tree in `internal/document`. The adapter uses the
streaming loader, checks for a second document explicitly, applies a calibrated
private parser depth plugin, enforces owned depth/node limits, preserves numeric
spelling and locations, and returns only value-free internal issues.
- Generalized the strict-source/document cases recorded in the [extraction
inventory](../internal/extraction.md), created the [current configuration input
contract](../config.md), and added an AST import-boundary regression. No typed
decoder, composition, filesystem graph, or public `Load` was introduced.
- Tests cover real file failures and reopening, byte ownership, fragmented/error
readers, exact/overflow byte limits, concurrent copied-source claims, all YAML
policy categories, trailing/empty documents, duplicate locations, lexical wide
numbers, block/flow depth calibration, node limits, safe parser errors, and
adapter isolation. Verified `go test ./...`, `go vet ./...`, `go build ./...`,
`go mod tidy -diff`, and `git diff --check` with Go 1.26.7 on linux/amd64. No
deviations were required; typed binding and decoding remain later work.
Read: feature [YAML policy](initial.md#yaml-and-scalar-policy),
[source constructors](initial.md#entry-point-and-reusable-application-definition).
Start with the extraction map's parser and strict-loader cases; the old
second-document acceptance defect is already fixed in Narratio.
Tasks:
1. Adapt Narratio's file/reader loading helpers into the opaque source constructors
and bounded private reader. Add `NoSource`, byte snapshotting, source labels,
regular-file checks, deterministic I/O classification, and reader single-use
state shared by source copies. File loads reopen each time; readers are not closed.
Use the bounded-read and source-ownership recipes above, including the extra-byte
oversize check and an atomic reader claim shared by source copies.
2. Add the pinned v4 YAML adapter and ConfigKit-owned document tree. Use
`yaml.NewLoader` and load each document into `yaml.Node`; translate key/value
locations and scalar spelling before discarding parser-specific nodes. Keep
upstream imports and error translation inside the adapter. Do not use parser
typed decode for application structs or return upstream types from the adapter.
3. Enforce exactly one mapping document, duplicate keys, tag/alias/anchor/merge-key
rejection, and ConfigKit's own scalar grammar. A second successful decode is
an error, including an empty document; do not preserve Narratio's trailing-doc bug.
Leave `WithSingleDocument` disabled: in rc.6 it stops the streaming loader after
the first document instead of rejecting trailing documents. Check explicitly
for another document. Validate duplicate keys in the ConfigKit tree; rc.6 node
loading preserves duplicates despite upstream typed-decoding uniqueness options.
4. Enforce byte ceilings before parsing. Configure the v4 limit plugin privately
to bound parser nesting, and enforce ConfigKit tree depth/node ceilings during
adapter translation. Calibrate parser-depth counting against ConfigKit's root-zero
limit with block/flow boundary cases rather than assume identical counters.
Parser failures must not panic or expose input text. Add private small-limit
injection for tests. Alias limits do not replace ConfigKit's alias prohibition.
5. Create `docs/config.md` for implemented syntax/input behavior and link from
package documentation. Move the implemented definitions out of the feature
roadmap, preserving navigation for this plan.
Acceptance:
- Real file and reader tests cover missing/permission/read failures, non-regular
targets, byte snapshots, second reader use, and no caller-reader close.
Cover exact-limit and one-byte-over input, fragmented reads, data returned with
an error, and competing loads of copies of the same reader source.
- Policy tables cover duplicate keys, all forbidden YAML constructs, null spellings,
quoted literals, JSON numeric syntax, legacy booleans/numbers, timestamp-like
strings, multiple/empty documents, and root type. Numeric nodes never round via float.
- Small-limit boundary tests and malformed input tests return safe diagnostics with
available coordinates. This stage does not promise a typed public `Load` yet.
- Retain regressions for duplicate node keys, trailing/empty second documents,
lexical numbers, explicit-tag visibility, and parser-depth enforcement. Add a
focused import-boundary check that rejects direct upstream YAML imports outside
the adapter and its tests and production `go-cmp` imports, using the AST recipe
above; extend it as packages/examples appear.
## Stage 4 — Build the reusable schema binding graph
Status: complete.
Completion evidence (2026-09-06):
- Implemented a per-operation private descriptor graph in [`schema.go`](../../schema.go)
for structural roots, ordinary/named scalars, duration and paired text scalars,
pointers, structs, string-keyed maps, slices, arrays, dynamic values, and recursive
definitions. No global cache or application method execution is involved.
- Implemented declared-field YAML binding, ignored/private fields, explicit struct/
pointer/map inlining, ambiguity and tag validation, one inline map, secret-tag
retention, and schema-aware key/index path traversal. Runtime default collisions,
cyclic values, cloning, projection, and decoding remain assigned to later stages.
- Added the [current schema contract](../config.md#application-schema-binding) and
updated the component inventory and target-roadmap ownership links.
- Focused tests cover nested and inline conflicts, ignored/private fields, maps of
structs, named scalars, pointer text method sets, missing codec pairs, root and
unsupported-kind rejection, recursive descriptors, side-effect freedom, dynamic
and inline-map paths, parsed numeric indices, and array bounds. Verified
`go test ./...`, `go vet ./...`, `go build ./...`, `go mod tidy -diff`, and
`git diff --check` with Go 1.26.7 on linux/amd64. No deviations were required;
typed value operations remain later work.
Read: feature [typed decoding](initial.md#typed-decoding) and
[type/projection contract](initial.md#supported-types-and-projection).
Tasks:
1. Implement reflection descriptors for supported root structs/maps, scalar kinds,
pointers, nested structs, string-keyed maps, slices, fixed arrays, and `any`.
Use declared-field reflection and tag lookup as specified above, not Go field
promotion as a substitute for ConfigKit binding.
2. Bind exported field names, tags, inline structs/pointers and the optional inline
map. Detect ambiguous names, multiple inline maps, unsupported tag options,
conflicting defaults-map names (checked against runtime defaults in Stage 6),
unsupported kinds, and missing paired text codecs. Identify secret-tag paths.
3. Support recursive descriptors without infinite construction; actual value cycles
are checked by projection. Use descriptors for schema-aware path validation.
4. Keep reflection traversal private. Do not expose generic subtree conversion,
module registries, YAML hooks, or interfaces for arbitrary decoders.
Acceptance:
- Focused tests cover nested/inline conflicts, ignored/private fields, maps of
structs, named scalar types, pointer method sets, root rejection, recursive
acyclic models, and unsupported kinds. Descriptor building has no side effects.
- No application parser-specific hook executes. Baseline passes.
## Stage 5 — Implement canonical projection and detached copies
Status: complete.
Completion evidence (2026-09-06):
- Added descriptor-driven canonical typed projection and independently owned cloning
in [`projection.go`](../../projection.go), without YAML serialization or a second
binding model. Zero values, nil/empty distinctions, exact integers, destination-
precision finite floats, durations, text scalars, dynamic concrete values, and
recursive acyclic objects retain their configuration meaning.
- Added active-graph cycle detection, private depth/node enforcement, safe custom
codec and dynamic-type failures, zeroing of ignored state, reconstruction of text
scalars into fresh instances, and non-preservation of shared pointer identity.
- Added canonical-tree structural comparison with semantic mapping additions,
removals and replacements, cross-kind replacement, and whole-sequence atomicity.
The [current ownership contract](../config.md#typed-projection-and-ownership) and
component inventory now own the implemented behavior.
- Focused tests mutate every source alias after cloning, cover nil versus empty,
wide integers, duration/text canonicalization, recursive acyclic values and cycles,
unsupported dynamic types, non-finite floats, value-free codec failures, inline
physical members/conflicts, projection bounds, ignored state, and meaningful
comparison paths without index-wise sequence matching. Verified `go test ./...`,
`go vet ./...`, `go build ./...`, `go mod tidy -diff`, and `git diff --check` with
Go 1.26.7 on linux/amd64. No deviations were required; defaults and typed decoding
remain later work.
Read: feature [ownership](initial.md#recorded-resolution-and-validation) and
[type/projection contract](initial.md#supported-types-and-projection).
Tasks:
1. Use Stage 4 descriptors to project ordinary typed values into a ConfigKit tree
without YAML serialization. Include zero values and preserve nil/empty distinctions.
2. Clone ordinary configuration into independently owned values. Reconstruct paired
text scalars through canonical text into fresh instances; preserve type identity
and supported dynamic concrete values without retaining mutable input aliases.
Apply the shallow-versus-recursive copy distinction above and reuse descriptors.
3. Detect cyclic values, non-finite floats, invalid dynamic types, projection limit
violations, and custom codec errors safely. Do not reveal the codec error text.
4. Add structural comparison used later by resolution audit and validator-mutation
detection. Track paths of changed values, container replacements, and removals;
do not create an index-wise correspondence between replaced sequences.
Acceptance:
- Mutating input/default maps, pointers, slices, or paired text-scalar backing data
cannot change a clone or snapshot. Ignored fields do not preserve hidden aliases.
- Nil versus empty, exact wide integers, durations, canonical scalar text, recursive
acyclic objects, cycles, unsupported dynamic values, and codec failures are covered.
- Comparison reports meaningful paths and container changes; baseline passes.
## Stage 6 — Decode scalar and container values over defaults
Status: complete.
Completion evidence (2026-09-06):
- Added schema-driven typed decoding in [`decode.go`](../../decode.go), starting
from the Stage 5 detached clone and returning only private typed state, input
presence, projected baseline/final trees, and per-path default/base origins.
- Implemented recursive mapping retention, atomic sequence replacement, exact array
lengths, pointer-object zero allocation versus non-nil retention, nullable-shape
clearing, strict nested map values, inline-map decoding, and unknown-field errors.
- Added checked destination-width integers and floats, dynamic `int64`/`float64`,
named scalars, duration strings, paired text codecs, safe value-free failures, and
collection of independent decode errors at original semantic paths/coordinates.
The [current configuration contract](../config.md#defaults-and-typed-decoding) and
component inventory now own this behavior.
- Adapted Narratio's fresh-default, presence, strict nested-type, and repeated-load
cases, replacing custom YAML hooks and pointer/zero guesses with tree-derived
presence; explicit empty and null are retained and equal-valued input is explicit.
Focused coverage includes the defaults operation table, new/existing map entries,
empty mappings/sequences, nil/non-nil pointer objects, default isolation, numeric
range and coercion failures, integer durations, inline collisions, nested unknown
fields, coordinates, safe codec failures, and independent repeated results.
Verified `go test ./...`, `go vet ./...`, `go build ./...`, `go mod tidy -diff`,
and `git diff --check` with Go 1.26.7 on linux/amd64. No deviations were required;
public loading and schema-version checks remain later work.
Read: feature [defaults/null](initial.md#static-defaults-and-null) and typed decoding.
Tasks:
1. Implement schema-driven typed decoding into a Stage 5 clone of the fresh default
baseline. Allocate replacements for sequences and recurse mappings by the feature
contract; do not let parser behavior determine merging or optionality.
2. Enforce numeric syntax/range and nullability. Support dynamic `int64`/`float64`,
finite floats, exact-duration string parsing, named scalar types, paired text
codecs, arrays with exact length, and strict nested map-of-struct decoding.
Use the standard-library conversion recipes above after lexical validation;
translate failures without retaining value-bearing conversion errors publicly.
3. Reject unknown fields, tag binding conflicts, and invalid defaults consistently.
Collect independent decode errors while leaving the failed working object private.
4. Return a private decoded configuration plus its surviving input presence index,
projected baseline/final trees, and per-path default/input origins for Stage 7.
Defaults never become explicit; equal-valued input remains explicit.
Acceptance:
- A compact table proves absent/false/zero/empty/null behavior, default map entries,
`{}`, `[]`, pointer objects, new entries, dynamic null, and non-nil default isolation.
- Overflow, non-finite values, quoted-number coercions, integer durations, inline
conflicts, and unknown nested fields fail at original input paths/coordinates.
- Adapt Narratio's presence/default test cases with intentional differences noted.
Repeated loads do not alias each other. Baseline passes.
## Stage 7 — Assemble single-source loading and schema-version checks
Status: complete.
Completion evidence (2026-09-06):
- Added the public generic `Load`, `Definition`, `Options`, `Result`, version,
filesystem-context, override, validation-context, and reserved resolution
declarations in [`load.go`](../../load.go) and [`validation.go`](../../validation.go).
The active lifecycle validates options, parses one source, rejects reserved
directives, builds the schema, checks an optional application version, creates
fresh defaults, decodes, and captures detached state.
- Added absent/null/string version handling, non-string rejection, exact/nearest
primary and related input locations, safe application reports, warning retention,
and generic value-free hook failure only when no explicit error was reported.
Defaults run only after version success and exactly once for a continuing load.
- Populated effective base/default `Metadata` and chronological definition history
in [`load_metadata.go`](../../load_metadata.go), with explicit and null state,
synthetic default origins, base coordinates, equal-valued input records, and
defensive result/source/diagnostic copies. `NoSource` performs the same schema,
version, defaults, decode, and projection lifecycle without a source record.
- Added explicit `options.unavailable` gates for resolution, semantic validation,
imports, active profiles, filesystem context, overrides, and secret classification;
invalid `NoSource` combinations and `_configkit` directives fail rather than being
ignored. The [current single-source contract](../config.md#single-source-loading),
package documentation, README, and component inventory own the implemented API.
- Integration tests cover File/Bytes/Reader/NoSource, schema-version states and
warnings, related locations, raw hook-error suppression, fresh-default ordering,
metadata agreement and detachment, source lists, unavailable options, reserved
directives, and nil results for source/schema/version/decode failures. Verified
`go test ./...`, `go test -race ./...`, `go vet ./...`, `go build ./...`,
`go mod tidy -diff`, and `git diff --check` with Go 1.26.7 on linux/amd64. No
deviations were required; resolution and semantic validation remain later work.
Read: feature [lifecycle](initial.md#resolution-lifecycle),
[version hook](initial.md#schema-checking-and-secret-classification),
[results](initial.md#results-errors-and-inspection).
Tasks:
1. Introduce `Definition`, `Options`, `Load`, and `Result` at the single-source
boundary. Wire source/options checks, parsing, version checking, fresh defaults,
decode, and detached result. Accept zero or one options argument.
2. Implement `VersionRule`, `Version`, and the pre-decode `Validation` report context,
including safe error/warning messages, related input paths, absent version from
`NoSource`, and generic fallback only when no explicit error was reported.
3. Implement base/default `Metadata.Lookup`, `IsExplicit`, `History`, source listing,
and warning/error lists. Keep snapshot internals private. No effective writer
is exposed before its redaction implementation in Stage 15.
4. Reject `_configkit` when imports are disabled. Until their later stages land,
reject non-nil resolver/validator hooks and import/active-profile options with
an explicit unreleased-feature options error rather than ignore them. Document
only the supported single-source boundary. Remove each gate in its owning stage.
5. Ensure `NoSource` performs version/default/decode/projection processing with no
source records and rejects filesystem/import/active-profile settings.
Acceptance:
- Actual `Load` integration tests cover File/Bytes/Reader/NoSource and nil result on
source, schema, version, or decode failure. Defaults factory runs only after a
successful version check and once per valid load.
- String version, missing/null policy, integer rejection, warnings, actionable
safe reports, raw error suppression, and invalid options are exercised.
- Metadata reflects input versus defaults and detached ownership. Baseline passes.
## Stage 8 — Implement recorded application resolution
Status: complete.
Completion evidence (2026-09-06):
- Implemented per-load `Resolution`, `IsExplicit`, `Record`, `Default`, `Normalize`,
`Error`, and `Warning` in [`resolution.go`](../../resolution.go), and enabled the
resolver in `Load` after successful decode while leaving semantic validation gated.
- Added schema-aware path/kind/dependency checks, safe reason requirements, dynamic
and typed helper projection, final helper/path consistency checks, before/after
structural comparison, exact/ancestor record coverage, and distinct safe failures
for invalid values, targets, records, and unrecorded changes.
- Coalesced repeated path intent deterministically, retaining unique reasons and
dependencies, using the last helper value, and promoting mixed operations to
normalization. Equal-valued derived intent remains recorded; map copy/writeback,
container coverage, deletions, and new dynamic/map descendants are supported.
- Integrated resolution origins and history without changing surviving input
explicitness, retained private detached before/after snapshots, sorted records by
semantic path, and made retained callback-context use panic with fixed value-free
text. Resolver errors are suppressed unless explicitly reported through the
context; warnings remain on successful results. The
[current resolution contract](../config.md#recorded-application-resolution) owns
this behavior.
- Focused tests execute the timeout/artifact consumer shape, explicit null/zero and
absent-baseline behavior, normalization, map writeback, equal and repeated helper
intent, stable map-derived record order, subtree removal, snapshot detachment,
unrelated targets, invalid paths/kinds/values, unrecorded mutation, safe hook
failures, and closed resolution/version contexts. Verified `go test ./...`,
`go test -race ./...`, `go vet ./...`, `go build ./...`, `go mod tidy -diff`, and
`git diff --check` with Go 1.26.7 on linux/amd64. No deviations were required;
semantic validation remains later work.
Read: feature [resolution callbacks](initial.md#resolution-and-validation-callbacks).
Tasks:
1. Add per-load `Resolution`, `Default`, `Normalize`, `Record`, `Error`, and `Warning`.
Hook context queries refer to surviving input, not guesses from typed zero values.
2. Run the resolver after decoding; capture before/final projection, detect net
changes, and require record coverage. Support map-entry copy/writeback helpers,
container records, deletions, dependencies, repeated writes, and equal-valued
derived-default intent using the clarification rules above.
3. Verify final helper values against declared paths. Reject invalid paths, invalid
kinds, invalid projected values, missing records, or use after context closure.
Store reasons/dependencies and private before/after snapshots for later safe explanation.
4. Attach phase diagnostics and stop on reported errors even with nil return. A
non-nil return adds generic safe failure only if no error was already reported.
Leave callback panics as programming failures. Remove the resolver options gate.
Acceptance:
- Execute the feature's timeout and dynamic artifact examples against real code,
using their non-secret fields here; add the typed credential field in Stage 17.
- Test explicit-null/zero preservation, baseline replacement for absent input,
normalization of explicit input, map writeback, unrelated pointer/path mismatch,
unrecorded mutation, equal-value intent, repeated helper writes, and subtree removal.
- Mutating result configuration cannot change resolution snapshots/history. No
hooks or errors expose secrets. Independent map traversal yields stable records.
## Stage 9 — Add semantic validation and finish the single-source lifecycle
Status: complete.
Completion evidence (2026-09-06):
- Reused `Validation` for post-resolution application reports in
[`validation.go`](../../validation.go), with `validate.error`,
`validate.warning`, and `validate.invalid_path` classifications. Omitted declared
fields and missing string-map/dynamic keys remain valid report paths; unknown
struct fields and malformed primary or related paths fail safely.
- Attached exact or nearest input locations to primary and related reports, retained
independent errors and warnings, continued through warnings, and returned no
configuration on any semantic error. Version, decode, and resolution failures stop
before later hooks.
- Added before/after typed projection around the validator. Any net scalar,
container, or nested mutable-state change, and any newly unprojectable value,
returns `validate.mutation` without publishing the mutated object. Callback
contexts close after invocation and callback panics remain programming failures.
- Removed the semantic-validator options gate, finalized metadata only after
successful resolution/validation, and retained defensive configuration snapshot,
metadata, diagnostic, and source ownership. The
[current semantic-validation contract](../config.md#semantic-validation) owns the
completed single-source lifecycle and application/runtime-state boundary.
- Added compiling executable examples in [`example_test.go`](../../example_test.go)
for minimal loading, defaults-only operation, related-path validation, actionable
schema-version guidance, and resolution followed by validation. Focused tests cover
multiple reports, missing/dynamic paths, related/nearest locations, invalid paths,
validator mutation, hook ordering and stop points, warnings, and final value/
explicitness/origin/history agreement. Verified `go test ./...`,
`go test -race ./...`, `go vet ./...`, `go build ./...`, `go mod tidy -diff`, and
`git diff --check` with Go 1.26.7 on linux/amd64. No deviations were required;
imports and profiles remain deliberately unavailable.
Read: feature [diagnostic callbacks](initial.md#resolution-and-validation-callbacks)
and [schema reports](initial.md#schema-checking-and-secret-classification).
Tasks:
1. Reuse `Validation` for post-resolution schema-aware application reports. Validate
primary and related paths, supply nearest source locations for omitted values,
and collect multiple independent errors/warnings.
2. Detect validator mutation through projection and fail without publishing a
result. Close contexts after callbacks and remove the validator options gate.
3. Finalize all single-source metadata after resolution/validation success. Ensure
failure carries only safe diagnostics and sources, not a partially usable config.
4. Add executable examples for minimal/defaults-only loading, related-path duplicate
errors, version guidance, and resolution followed by validation. Document actual
hook lifecycle and private-field/runtime-state boundaries.
Acceptance:
- Hook-order tests prove later phases never run after prior errors; warnings continue.
- Test missing fields, dynamic keys, related locations, validator mutation, and safe
hook failure fallback. Compile and run representative examples, not API stubs.
- Compare value, explicitness, origin, and history after a complete single-source load.
Baseline passes. Imported/profile loading is still deliberately unavailable.
## Stage 10 — Implement strict document composition
Status: complete.
Completion evidence (2026-09-06):
- Adapted Narratio's additive claim validation and recursive mapping join into the
private `composeDocuments` boundary over ConfigKit-owned document trees. Present
equal or unequal scalars, nulls, sequences, mapping/value kind changes, and nested
duplicate entries return `compose.conflict`; mapping pairs alone recurse.
- Changed the extracted empty-map behavior to ConfigKit's operation table: empty
mappings add no children, compose with other mappings, and retain their container
location in composite base origins. Application defaults never enter the composer.
- Preserved source/declaration traversal in detached ordered claim records and used
it only for assembled order and exact primary/related diagnostics. Conflicts across
three or more inputs report every claimant, while shared containers aggregate all
contributing locations instead of selecting an arbitrary owner.
- Cloned every retained branch and verified mutations of a composed result cannot
affect any input. Focused tests cover disjoint keyed joins, operation categories,
ancestor and nested conflicts, equal values, nulls, sequences, empty mappings,
exact locations, all claimants, invalid boundaries, and nonmutation. The
[current composition contract](../config.md#strict-document-composition) owns the
implemented semantics; public import wiring remains deferred to subsequent work.
Read: feature [composition table](initial.md#composition-and-overlay).
Extract Narratio's additive composer and conflict tests before writing equivalent
code. Change empty-map expectations to the feature table and retain exact locations.
Tasks:
1. Implement a private strict composer over ConfigKit document trees. Recurse only
mapping pairs; treat any other present overlap as a collision, including equal
values, null, and sequences. Empty mappings contribute no children.
2. Retain both locations for collisions and aggregate composite container origins.
Preserve input order only for traversal/diagnostics, never implicit precedence.
3. Build provenance updates alongside composition; no subsequent pass may invent
an arbitrary single source for a multi-source mapping. Avoid mutating input trees.
4. Keep the composer private and test it through the smallest stable private
boundary; public file graph wiring belongs to subsequent stages.
Acceptance:
- Exercise each operation-table category plus ancestor/scalar conflicts, empty maps,
sequence collisions, equal-value duplicates, and disjoint named entries.
- Duplicate diagnostics identify both exact source locations; retained inputs and
origins survive without aliasing. No default layer participates in strict collision.
## Stage 11 — Implement explicit file-import traversal
Status: complete.
Completion evidence (2026-09-06):
- Added strict recursive `_configkit.imports` parsing with exactly one `path` or
recognized `glob`, optional booleans, unknown-shape rejection, root-only placement,
and detached directive removal. Glob declarations return the deliberate temporary
`import.unsupported` gate for the next owning work.
- Implemented declaration-order depth-first path traversal with canonical symlink
identity, relative containment checks, and canonical root-relative access through a
ConfigKit-owned `os.Root`. File inputs infer their canonical base/root; labeled
byte and reader inputs require explicit absolute base/root context and never infer
paths from URI-like labels.
- Kept active and visited indexes separate, reporting complete cycle import sites and
both diamond/alias inclusion routes. Canonical aliases repeat, while hard links at
distinct paths remain separate sources. Required missing paths, directories,
unreadable targets, escapes, and canonicalization failures remain errors; optional
suppresses only nonexistence.
- Assigned deterministic source IDs while reading each accepted source once, enforced
per-source and aggregate bytes, aggregate input nodes, source count, and active-edge
depth, and retained import-site/source records on failures. Imported fragments are
rejected when they claim the application version path or a non-mapping ancestor,
before strict composition.
- Wired composed input and multi-location origins through `Load`, version checking,
strict decoding, resolution, validation, and base metadata. Temporary-directory
tests cover nested/list-order traversal, composition conflicts, version placement,
directives, optional/required paths, explicit roots, cycles, diamonds, symlinks,
hard links, parse/decode attribution, file types, context rules, and reduced limit
boundaries. The [current path-import contract](../config.md#explicit-imports)
owns implemented behavior; glob expansion remains pending. Verified `go test ./...`,
`go test -race ./...`, `go vet ./...`, `go build ./...`, `go mod tidy -diff`, and
`git diff --check` with Go 1.26.7 on linux/amd64.
Read: feature [import graph](initial.md#explicit-imports-and-source-identity).
Extract the flat import loader and tests, then generalize its graph, identity and
containment rules. Do not treat its root-only declarations as ConfigKit's contract.
Tasks:
1. Parse the reserved root directive and `path` declarations strictly; preserve
import-site locations and strip directives before application decode. Recognize
`glob` declarations but retain an explicit temporary unsupported gate until Stage 12.
2. Resolve paths from their declaring source, canonicalize symlinks, enforce root
containment, and perform deterministic depth-first traversal. Keep active-stack
and visited indexes distinct. Track both routes for cycles and repeated imports.
Use the canonicalization/`os.Root` recipe above, including ConfigKit-owned root
handle cleanup; preserve contained relative and absolute symlink targets.
3. Read accepted sources once; enforce count/depth/aggregate budgets. Optional paths
suppress only target nonexistence. Invalid directives, directories, unreadable
files, canonicalization failures, and out-of-root targets remain errors.
4. Reject schema-version contributions from fragments before composition, even when
no main-file version exists. Ensure ancestor replacements of a nested version
path cannot smuggle a fragment version past the placement rule.
5. Wire composition and version checking into `Load`, removing the import gate for
supported path declarations. Document the temporarily limited import boundary.
Acceptance:
- Temporary-directory fixtures cover nested relative paths, list-order traversal,
symlink aliases, full-cycle routes, diamond repeats, optional versus required
missing files, out-of-root traversal, regular-file checks, and budget boundaries.
- Version placement and duplicate value diagnostics retain their owning sources.
- Bytes/reader imports require explicit base/root; URI labels never imply a path.
## Stage 12 — Complete glob imports and filesystem option handling
Status: complete.
Completion evidence (2026-09-06):
- Removed the temporary glob gate and added prevalidated Go `path.Match` semantics
for `*`, `?`, character classes, and escapes, with recursive unescaped `**`, URLs,
absolute paths, empty segments, and malformed classes rejected before any walk.
- Added a pattern-directed walker that resolves literal segments directly and reads
wildcard directories through positive bounded `File.ReadDir` batches. It charges
every examined entry, including nonmatches, canonicalizes and contains traversed
directories and final targets, sorts slash-relative matches, and bounds final
candidates against remaining source capacity before traversal.
- Reused the path-import DFS for every sorted match, preserving declaration ordering,
nested depth-first imports, canonical alias/repetition handling, hard-link path
identity, source IDs, composition, version placement, and all existing byte/node/
source/depth budgets. Optional glob declarations suppress only zero matches, never
invalid syntax, walk/permission failures, matched directories, or symlink escapes.
- Completed filesystem option validation independently of import/profile selection:
file inputs reject `BaseDir` and may select a containing root; byte/readers require
existing absolute base/root pairs whenever context is supplied; `NoSource` rejects
context while the reserved `default` profile remains inactive.
- Tests cover lexical and multi-declaration ordering, deterministic repeated loads,
overlapping matches, directories, hard links, contained absolute and escaping
symlink directories, required/optional no-match behavior, invalid patterns in empty
directories, escaped metacharacters, character classes, visible permission errors,
explicit `conf.d` imports without discovery, nonmatching-entry exhaustion, candidate
bounds, and filesystem combinations. The
[current import contract](../config.md#explicit-imports) owns the completed behavior.
Verified `go test ./...`, `go test -race ./...`, `go vet ./...`, `go build ./...`,
`go mod tidy -diff`, and `git diff --check` with Go 1.26.7 on linux/amd64.
Read: feature import syntax and source-context API.
Reuse import failure fixtures where applicable; Narratio's flat loader does not
supply generic glob traversal or its bounded filesystem behavior.
Tasks:
1. Use `path.Match` with the constrained grammar and bounded `File.ReadDir(n)`
expansion recipe above. Return sorted slash-relative matches; no recursive `**`.
Keep directory expansion pattern-directed and error-reporting. Reuse Stage 11
identity and traversal; do not add another glob engine or use `filepath.Glob`.
2. Reject invalid patterns before walking; canonicalize and contain visited
directories/targets, including symlinked directories. Optional zero matches
must not conceal a walk or permission failure. Bound accepted candidates before
accumulating unbounded source work; charge directory-entry visits to their
separate traversal budget, including entries that do not match the pattern.
3. Validate all filesystem context combinations, profile-name-independent root setup,
absolute import/URL rejection, and selected-file versus imported-file identity.
`default` means no active profile for `NoSource` checks.
4. Remove the glob gate. Exercise explicitly declared `conf.d/*.yml` without adding
any implicit directory scan or extension fallback.
Acceptance:
- Fixtures cover lexical order, multiple declarations, overlapping matches, matches
that are directories, hard links as separate paths, symlink escapes, unmatched
optional/required patterns, invalid patterns, and visible permission failures.
Cover escaped metacharacters, character classes, empty-directory pattern errors,
contained absolute symlinks, and budget exhaustion from nonmatching entries.
- Repeated runs produce identical source IDs, composition, and diagnostic ordering.
Full import contract and baseline pass.
## Stage 13 — Implement profile overlays
Status: pending.
Read: feature [profiles](initial.md#profiles), composition/overlay table.
Extract Narratio's overlay mechanics and profile tests; preserve the intentional
null/kind and profile-selection differences listed in the feature roadmap.
Tasks:
1. Implement private recursive overlay with atomic scalar/sequence/null and
cross-kind replacement. Record overridden definitions as the operation occurs.
2. Resolve exactly one constrained profile name to its fixed `.yml` path under the
root, with canonical containment. Empty/lowercase `default` means no profile;
selected missing profiles fail. No path inputs, inheritance, or extension fallback.
3. Reject profile directives and schema-version contributions before overlay.
Also reject an ancestor scalar/null replacement that would remove a declared
nested schema-version value; an unchanged nested version supplied by a profile
is still forbidden.
4. Apply profile after base graph composition/version checking and before defaults
and decoding. Remove the last optional-feature gate from `Load`.
Acceptance:
- Profile scalar, mapping sibling, full sequence, empty sequence, null, and cross-kind
replacement cases obey the operation table. Defaults remain lower precedence.
- Removed descendants disappear from effective presence/origin, remain explainable
through history, and cannot inherit stale sequence indices.
- Base YAML policy errors are not masked by profiles; effective-tree schema checks
and all profile graph/version restrictions pass. Baseline passes.
## Stage 14 — Verify layered provenance and final snapshot agreement
Status: pending.
Read: feature [presence](initial.md#paths-and-presence),
[history](initial.md#provenance-diagnostics-and-redaction), metadata API.
Use Narratio's effective ownership and inspection tests as extraction evidence.
Its source-only records need locations and complete historical operations.
Tasks:
1. Integrate default, base, profile, and resolution records into a coherent query
index; reuse prior-stage provenance rather than reconstructing sources from values.
2. Finalize histories for replaced containers, empty mappings, deleted paths, and
whole-sequence changes. Retain history lookups for removed paths without claiming
those paths exist in the effective snapshot. Canonicalize reasons/dependencies.
3. Test composition/profile/resolution interaction through one representative public
workflow and fill only uncovered cross-layer cases. Keep operation-table coverage
in its owning tests instead of duplicating it all through `Load`.
4. Update metadata reference docs with concrete composite and synthetic origin cases.
Acceptance:
- A configured value equal to its default remains explicit; derived values do not.
- Final typed projection, lookup/null state, effective origins, source order, and
histories agree after parent and sequence replacement plus resolution.
- All returned metadata/source/diagnostic nested values are detached, and successive
or concurrent independent loads cannot contaminate one another. Baseline passes.
## Stage 15 — Implement secret classification and effective rendering
Status: pending.
Read: feature [redaction](initial.md#provenance-diagnostics-and-redaction),
[secret predicate](initial.md#schema-checking-and-secret-classification), result API.
Reuse Narratio's deterministic effective-output and non-disclosure cases. Render
from the owned typed projection, without its marshal/parse reconstruction.
Tasks:
1. Combine descriptor secret tags with the application path predicate over current
and historical paths. A secret container marks descendants. Apply dependency
propagation to a fixed point so callback/map submission order is irrelevant;
cycles in dependency annotations terminate and propagate sensitivity conservatively.
2. Ensure replaced/deleted secret definitions remain classified. The predicate can
add sensitivity, never remove tagged/historical sensitivity. Do not expose raw
retained values through debug formatting, diagnostic wrappers, or public records.
3. Add `Result.WriteEffective` over the frozen canonical snapshot. Sort mapping keys
by decoded string, preserve sequence order, use two-space indentation and a final
newline, and substitute the quoted string `[REDACTED]` at secret roots. Preserve
supported nil/empty distinctions. No unredacted option or implicit logging.
4. Emit only an adapter tree made from the safe projection through the v4 dumper;
never call application YAML marshal hooks. Set indentation, sequence indentation,
line breaks, and document-marker behavior explicitly inside the adapter rather
than inherit a version preset. Use two-space indentation, noncompact sequences,
LF line breaks, and no document start/end markers for the single output document.
Finish all text-codec projection work before returning a successful load. Writer
errors may leave partial redacted output; translate upstream dump errors safely.
Acceptance:
- Canary secrets stay absent from ConfigKit-rendered errors, warnings, metadata,
effective YAML, and histories across parse/decode/hook failures and overwritten
definitions. Check ordinary and Go-syntax formatting of metadata/error wrappers
so private histories cannot leak. Typed `Result.Config`, including application
formatting of it, is the explicit trusted boundary.
- Test secret containers, dynamic paths, dependencies, cycles, custom scalar values,
snapshot mutation isolation, and writer failures. Document safe application message
responsibility and that redacted output is not promised reloadable.
- Render actual resolved values including defaults/normalization; baseline passes.
## Stage 16 — Extract explicit secrets-directory bootstrap
Status: pending.
Read: [secret bootstrap contract](initial.md#secret-references-and-explicit-bootstrap),
architecture responsibility boundary, testing policy, and WeatherReporter's
`internal/config/secrets.go` and corresponding `config_test.go` cases; also
Narratio's `internal/app/secrets_env.go`, its tests, and bounded-read helper.
Tasks:
1. Record both applications' source revisions and reused symbols/tests in
the extraction inventory. Reuse WeatherReporter's `loadSecrets`/`applySecrets`
separation and Narratio's bounded reads/preserve-existing behavior. Replace
overwrite/rollback, immediate writes, aggregate logging, and incompatible
name/newline/permission/directory policies with ConfigKit's contract. Keep this
operation separate from `Load` and do not add a directory loading option to YAML.
2. Implement bounded, deterministic enumeration and metadata validation; reject
symlinks/nonregular entries, validate names, and preserve existing variables
including empty ones without reading their contents.
3. Implement eligible-file preflight, newline/NUL checks and the apply state
machine, including the presence recheck and partial-failure report. Add the
public report, status enum, safe typed error and report renderer as specified.
4. Document startup-only process mutation, non-atomicity, and how applications
report blocked files and retain the report alongside configuration results.
Acceptance:
- Adapt WeatherReporter's preflight-failure tests and Narratio's existing-empty
environment cases. Replace overwrite/rollback expectations with skip/partial-report
checks; do not drop the underlying failure coverage. Keep bootstrap explicit,
independent of subsequent configuration validation.
- Real temporary-directory tests cover newline preservation, invalid names/types,
bounded reads/enumeration, empty files, and canonical report paths/order.
- Existing nonempty and empty variables win and are distinguishable in reports;
skipped contents are not read. Preflight failures perform no writes. Controlled
apply failure and newly set variable tests prove accurate partial statuses.
- Canary contents are absent from errors, fmt/JSON reports and rendered output.
Writer failures propagate. Run the stage baseline; no actual credentials are used.
## Stage 17 — Add environment-backed typed secret references
Status: pending.
Read: [secret reference contract](initial.md#secret-references-and-explicit-bootstrap),
shared descriptor/projection rules and lifecycle, Stage 16 report boundary.
Tasks:
1. Implement opaque `Secret`, `SecretEnv`, name/value/resolution accessors, safe
fmt behavior and name-only JSON output. Extend existing binding, cloning,
decoding, projection, comparison and historical sensitivity machinery.
2. Resolve effective names after the resolver and before validation. Cache lookup
results only for this load. Handle required versus optional references, invalid
names, absent/null/empty references, unset and empty variables with safe errors.
3. Keep credentials out of every document/history/metadata representation. Rebuild
references from names when cloning defaults, even if they came from a prior
resolved result. Add effective reference-name inspection without claiming a
bootstrap file populated the current environment.
4. Update current contracts and the pipeline example to use typed references;
retain tag/predicate redaction for sensitive derived data. Earlier lifecycle
examples using `Secret` become runnable only at this stage.
Acceptance:
- Test required and optional references in structs, maps, pointers and sequences;
only surviving names are read, once per unique name. Resolution can change a
name and validation sees its resolved value. Reject validator replacement with
a same-name unresolved reference, using a private-state audit against the lookup
cache in addition to projection. Later loads observe changed env.
- Bootstrap plus load tests show a blocked empty variable in the report and an
empty-reference failure at its config path; no fallback to directory contents.
- Secret values appear only through `Value()`, never safe fmt/JSON, diagnostics,
projected trees or history; previously resolved defaults cannot retain stale
values. `Load` never mutates environment. Run the baseline.
## Stage 18 — Apply explicit CLI overrides above profiles
Status: pending.
Read: [override contract](initial.md#cli-overrides), overlay/presence/version rules,
source ordering, and the secret-reference boundary.
Tasks:
1. Add `Override` and `Options.Overrides`. Validate paths, labels, allowed value
types, overlap/version/directive restrictions and budgets; detach input into
private nodes without YAML reparsing or scalar inference.
2. Apply records after profiles and before the existing final typed decode using
the shared overlay primitive. Create synthetic CLI sources and record explicit
presence and before/after operations, including equal-value replacements.
3. Support defaults-only loads with overrides and add a parser-neutral executable
example translating only explicitly set flags. Document general environment
overlays as an explicit non-goal, not a future option.
Acceptance:
- A lifecycle test proves defaults < base/imports < profile < CLI, followed by
recorded normalization and normal secret lookup/validation. Zero, false, empty
string/list and null remain intentional values.
- Test map overlay and whole-list replacement, invalid ancestors/index paths,
duplicate/overlapping paths, type/cycle/limit errors, unknown fields and version
protection. Input mutation after load cannot change result or history.
- CLI diagnostics and provenance identify flag names without raw arguments;
CLI secret input is an environment name. Run the baseline.
## Stage 19 — Expose safe historical-value explanation
Status: pending.
Read: [explanation contract](initial.md#historical-value-explanation), redaction
rules, and history bounds above.
Generalize Narratio's semantic-record and inspection tests. They cover effective
values and source ownership, not the full chronological history required here.
Tasks:
1. Extend retained operation snapshots where necessary to reconstruct known final
and removed paths through defaults, base contributions, profile, CLI, and net
domain changes. Use structural sharing and the history budget, not full copies
per field. Preserve original policy-validated overridden nodes.
2. Implement detached `ValueSnapshot`, `ValueChange`, `Explanation`, and reference
metadata; apply historical secret/dependency closure before any public value
projection. Add `Result.Explain` with absent/null/deleted distinctions.
3. Implement `WriteExplanation` over the safe result, with canonical pointer paths,
composite origins, ordered before/after history, reasons/dependencies and final
source. Keep bootstrap reports separate and show how commands present both.
Acceptance:
- Show the requested concurrency history 4 → 8 → 1 and effective profile file/line;
extend one test with CLI and normalization. Test equal writes, removed paths,
parent replacement, composite maps, atomic sequences and unknown paths.
- Historical input with an overridden invalid destination type stays explainable;
output never claims it was successfully typed. Default null/empty rules agree
with final decoding and metadata. Retained history obeys bounded allocation.
- Canary secrets remain absent even after replacement/deletion/dependency cycles;
returned snapshots cannot mutate internal history. Test safe fmt/JSON of public
records, escaped labels, deterministic output and failing/short writers.
- Run baseline and a representative end-to-end bootstrap/load/explain test.
## Stage 20 — Prove consumer ergonomics and migration contracts
Status: pending.
Read: feature consumer-fit table, module options, external values, and both criteria.
Include the updated consumer tests listed in the extraction map. Preserve domain
rejection of explicit empty lists without making empty lists a library schema error.
Tasks:
1. Create maintained executable examples and minimal synthetic fixtures for four
consumer shapes: Narratio-like defaults/partial imports/artifacts; Notarius-like
object bindings/string versions/dynamic options; WeatherReporter-like defaults-only
and report/module options; Seriatim-like ordered speaker/autocorrect rule files.
2. Generalize the relevant copied Narratio tests and helpers, recording source
revision and adaptations. Use actual consumer field patterns but no private data or
dependencies on sibling application modules. Keep runtime algorithms in example
application functions rather than adding library APIs for them.
3. Add a WeatherReporter-shaped startup example that renders bootstrap outcomes,
loads typed references, and explains configuration with explicitly set CLI flags.
Demonstrate both a blocked empty variable and successful lookup without real
credentials or implicit process mutation in `Load`.
Compare preserved outcomes against consumer evidence and list intentional changes:
explicit empty/null behavior, canonical objects, versions, integer representation,
strict YAML, retired general environment overrides, explicit CLI records, and
secret-reference migration. Verify imports
and profiles do not absorb application LLM/pipeline-profile semantics.
4. Convert every feature-roadmap API example into a real compiled/run example or a
focused behavioral test; document the happy path in README and `docs/consumers/`.
Verify dynamic options without YAML reserialization or generic subtree conversion.
5. Add a short consumer-fit assessment to current consumer guidance: what plumbing
disappears, what domain code remains, and what downstream migrations are still
separate tasks. Do not use line-count targets as acceptance criteria.
Acceptance:
- All maintained YAML examples load as intended; negative migration fixtures fail
with stable classifications and useful paths, not copied raw parser text.
- Simple consumers need no imports/profiles and no custom parser hooks. Complex
examples preserve domain meaning under the same conventions.
- No uncontrolled external configuration reads, LLM calls, remote services,
or sibling checkout requirements occur in default tests. Baseline passes.
- Compile consumer examples using only ConfigKit and standard-library imports.
Review exported signatures, reachable record fields, aliases, generic constraints,
callbacks, and error chains against the upstream isolation contract. Consumers
must not name or configure the YAML dependency to exercise any supported API.
## Stage 21 — Harden boundaries and verify independent loads
Status: pending.
Read: testing policy, ownership/projection, import limits, redaction guarantees.
Tasks:
1. Add bounded fuzz targets for path parsing, YAML translation, and document
composition/overlay invariants. Retain deterministic regression seeds for any
defects found; avoid a large permanent corpus of redundant cases.
2. Add parallel independent-load coverage for shared read-only definitions and
fresh defaults; exercise defensive-copy and custom-scalar paths under the race
detector. Source/reader single-use behavior must be race-safe, not silently reused.
3. Review private limit enforcement through reads, traversal, projection, and
rendering. Add only missing consequential boundary tests. Prevent unbounded
expansion through permissive aliases/providers; these remain unsupported.
4. Run default suite, vet/build, and `go test -race ./...`. Run each selected fuzz
target with `-run='^$' -fuzz=<exact-name> -fuzztime=30s -parallel=2` in its owning
package. Record exact names/commands in the developer guide. No fuzz loop in CI
is required beyond deterministic seeds unless separately configured.
Acceptance:
- Race checks and bounded fuzz runs pass on a supported native platform, with any
discovered defects fixed and retained at their narrowest durable test boundary.
- No unexplained test skips, mutable shared defaults, raw-secret render paths,
unsupported-feature gates, or stale placeholder APIs remain.
- A fresh offline baseline after dependency acquisition passes without reading any
developer-specific configuration path.
## Stage 22 — Prepare release validation and reconcile documentation
Status: pending.
Read: release policy in full and [release-readiness gaps](release-readiness.md).
Tasks:
1. Create `docs/release.md` with executable commands for the existing origin/main,
module path, source-only tags, candidate recording, branch/tag/cleanliness guards,
precise publication refs, and clean consumer module retrieval/build verification.
Include private-host access prerequisites without credentials or environment dumps.
2. Implement `scripts/release-check.sh` as a fail-fast source-candidate checker.
It must perform module hygiene (tidy in a disposable copy and diff manifests),
cached dependency verification, offline default tests, native race tests, vet,
native builds, `CGO_ENABLED=0` cross-builds for the four supported targets,
formatting/whitespace checks including untracked files, maintained examples,
and local documentation-link checks. Acquire toolchains/dependencies explicitly
before running the offline checker. Use the baseline toolchain and a newer
available supported toolchain when assessing the declared Go support range.
3. Make links to sibling consumer source files optional only when those checkouts
are absent in isolated release environments: verify local links to current
ConfigKit owners strictly, preserve recorded external revision references, and
document the external-reference applicability instead of requiring private repos.
4. List manual release gates: compatibility/migration review, secrets/unintended-file
inspection, platform execution evidence where native hosts differ, roadmap
retirement, release-note review, exact remote commit/tag guards, and post-publication
retrieval. Static-analysis baseline is vet; do not silently claim a linter was run.
5. Exercise the checker on current source and reconcile all documentation links,
API comments, `docs/config.md`, consumer examples, and component inventory.
Replace implemented feature-roadmap definitions with current contract links;
repair ADR navigation without changing accepted rationale. Keep this execution
plan until release preparation retires completed roadmap material.
Reassess the v4 pin under the dependency-update rule above. Run the import-boundary
check and review the public API for upstream leakage; preserve ConfigKit's
behavioral expectations when adapting to any upstream changes.
6. Update release-readiness evidence for the actual runbook/checker. Keep any
unvalidated release-candidate/publication gap explicit and retain its bootstrap
guard until the maintainer closes it. Do not fabricate release notes for an
unpublished candidate or mark remote verification completed. No publication is
authorized by this stage.
Acceptance:
- The checker actually passes applicable automated gates, and the runbook explicitly
owns every manual/inapplicable gate required by policy. No nonexistent commands
are presented as operational. Any missing native-platform or publication evidence
remains an explicit release gate, with its owner and closure check recorded.
- Current docs describe implemented behavior; no duplicated roadmap contracts or
stale deferred-plan statements remain. All feature areas below have completion
evidence, and both architecture success criteria are demonstrated locally.
- No tag, push, downstream migration, or release publication occurred as part of
implementation. Maintainer release preparation remains a separate procedure.
## Coverage and completion audit
Use this map to ensure completion means the whole feature roadmap, not merely a
working loader. The linked feature sections own semantics; stages own work.
| Feature area | Owning stages |
| --- | --- |
| Extraction, ordinary consumer types, package boundaries | 1, 4–6, 20 |
| YAML subset, precise numeric syntax, safe source failures | 2–3, 6 |
| Paths, presence, origins, definition history | 2, 6–8, 10, 13–14 |
| Static defaults, null, container rules, snapshot ownership | 5–7, 14, 21 |
| File/bytes/reader/defaults-only inputs and filesystem context | 3, 7, 11–12 |
| Version hook and graph-placement restrictions | 7, 11, 13 |
| Strict imports, glob ordering, identity, limits, collisions | 10–12, 21 |
| Named profiles, precedence, atomic replacement | 13–14 |
| Recorded normalization/derived defaults and hook errors | 8–9, 14 |
| Semantic validation, related diagnostics, safe messages | 2, 7–9 |
| Secret tags/predicate/dependencies and effective inspection | 5, 15, 21 |
| Module-specific options, consistency and intentional migrations | 20 |
| Runnable docs, examples, source-release checks and runbook | 1, 20, 22 |
| Secret-directory bootstrap and blocked-file provenance | 16–17 |
| Environment-backed secret references and safe lookup | 17, 19–21 |
| Explicit CLI overrides and complete precedence | 18–21 |
| Safe historical values and explanation renderer | 14–15, 19–21 |
| Deferred-feature boundaries | Every stage; final audit in 22 |
At final handoff, report completed stages, exact checks, current documentation
owners, source-extraction evidence, and remaining release/consumer migration work.
A green unit suite alone is not proof that examples, metadata safety, filesystem
behavior, documentation, or both success criteria are satisfied.
## Open Questions
No unresolved product or API question blocks the ordered stages above. The
implementation decisions explicitly select the module identity, Go baseline,
parser, limits, supported platform targets, metadata shapes, and consumer scope.
Do not reopen settled feature policies solely to preserve a legacy convention.
Two external execution facts must still be verified when their stage runs; they
are not authorization to silently weaken acceptance:
1. **Is a native Linux runner available for the declared support matrix?**
Recommended: execute filesystem/race tests on native Linux and macOS before
release; cross-builds alone cannot verify symlink/permission behavior. Stage 22
can prepare the checker on the available host while keeping missing native
evidence as an explicit release gate. Alternative: formally narrow the initial
supported platforms to those with execution evidence, with a documented scope
decision; this is more honest than asserting untested support, but less useful
for the likely server consumers. Neither option blocks the platform-independent
stages or permits claiming a skipped native check passed.
2. **Are the authoritative host and module retrieval credentials available at
publication time?** Recommended: verify the exact version from a clean consumer
module after authorized publication using the host's normal access mechanism;
keep credential setup outside ConfigKit and out of tests. Alternative: use an
explicitly configured internal Go proxy mirroring the exact immutable tag and
verify its source identity; it adds infrastructure but can suit restricted
environments. Local replacement directives are useful for development but are
not a viable substitute for release retrieval verification.