Finalize ConfigKit design and implementation plan

This commit is contained in:
2026-09-06 11:48:22 -05:00
parent 18524330fc
commit 9fc272f7a9
14 changed files with 2386 additions and 1116 deletions

View File

@@ -1,861 +0,0 @@
# ConfigKit — Initial Design and Architecture
## Status
**Design note for initial implementation.**
Audience: future maintainers of ConfigKit, including LLM coding agents assisting with implementation.
This document captures the current architectural and policy decisions for ConfigKit. The goal is to establish a sufficiently opinionated v1 design that can be implemented consistently across the application suite without prematurely becoming a general-purpose configuration framework.
---
## 1. Purpose
ConfigKit is a shared Go library for configuration loading and resolution across applications such as Seriatim, Audita, Narratio, WeatherFeeder, WeatherReporter, Distributor, and related tools.
Its purpose is to provide a consistent, blessed configuration lifecycle for applications that use YAML configuration files.
ConfigKit should solve recurring infrastructure concerns once:
- file loading;
- YAML parsing and strictness;
- deterministic imports / `conf.d` composition;
- duplicate-definition detection;
- intentional higher-precedence overlays such as profiles;
- presence tracking;
- source/provenance tracking;
- typed decoding;
- standardized diagnostics;
- effective-configuration inspection;
- support for application defaults, derived defaults, and validation.
ConfigKit is **not** intended to become a universal configuration framework.
---
## 2. Design Principles
### 2.1 YAML is syntax; ConfigKit defines semantics
The YAML parser must not implicitly define ConfigKit behavior for:
- composition;
- precedence;
- duplicate keys;
- sequence merging;
- `null`;
- imports;
- defaults;
- profile overlays.
These behaviors must be explicit ConfigKit policy.
### 2.2 Preserve information until late in the pipeline
ConfigKit should retain the parsed YAML tree and source metadata through composition and overlay processing.
Do **not** immediately reduce configuration to `map[string]any` or typed Go structs.
The implementation should preserve enough information to answer:
> Why does this effective value have this value, and where did it come from?
### 2.3 Keep application-facing Go types ordinary
Applications should normally use idiomatic types:
```go
type Config struct {
Enabled bool
Workers int
Timeout time.Duration
}
```
ConfigKit should not require pervasive `*bool`, `*int`, or `Optional[T]` fields merely to distinguish absence from explicit zero values.
Parser-level presence is metadata owned by ConfigKit.
Application-level optionality should remain explicit in the application's Go type where it is genuinely part of the application's semantics.
---
## 3. Responsibility Boundary
### ConfigKit owns
- configuration source loading;
- YAML parsing;
- YAML policy enforcement;
- import discovery;
- deterministic import ordering;
- strict composition;
- overlay application;
- duplicate-definition detection;
- source location tracking;
- presence tracking;
- definition history / provenance;
- typed decoding;
- standard diagnostics;
- effective-config rendering;
- config source reporting;
- config fingerprinting, if implemented;
- redaction mechanisms for secrets.
### Consuming applications own
- their `Config` Go types;
- static default values;
- derived/defaulted values;
- semantic validation;
- schema meaning;
- schema migrations;
- runtime initialization;
- runtime resource validation;
- hot reload and resource replacement.
---
## 4. Configuration Resolution Model
The conceptual lifecycle is:
```text
source discovery
↓
YAML parsing
↓
YAML policy validation
↓
strict base composition
↓
profile overlay
↓
static defaults / typed decode
↓
derived defaults / normalization
↓
semantic validation
↓
immutable effective configuration
```
A useful precedence model is:
```text
LOW PRECEDENCE
application defaults
↓
base configuration
├─ main file
└─ imports / conf.d ← strict composition
↓
selected profile ← intentional overlay
HIGH PRECEDENCE
```
Future layers such as environment-variable or CLI value overrides may be added later, but are not required for v1.
---
## 5. YAML Policy
YAML remains the configuration format.
ConfigKit should support a conservative YAML subset rather than arbitrary YAML semantics.
Recommended v1 policy:
- one YAML document per file;
- mappings, sequences, and scalars;
- string mapping keys;
- comments;
- unique mapping keys;
- reject unknown fields when decoding into typed configuration;
- reject custom YAML tags;
- reject YAML merge keys (`<<`);
- strongly consider rejecting anchors / aliases;
- reject ambiguous or unsupported constructs rather than guessing.
The intended model is approximately:
> JSON's data model, with comments and more pleasant human syntax.
The underlying YAML implementation must remain an internal implementation detail. Public ConfigKit APIs should not expose parser-specific node or error types.
---
## 6. Presence Semantics
ConfigKit must distinguish at least these states:
| State | Meaning |
|---|---|
| key absent | no opinion; a lower-precedence/default value may apply |
| `key: false`, `key: 0`, `key: ""` | explicitly configured value |
| `key: null` | explicitly null; not equivalent to absent |
| ordinary nonzero value | explicitly configured value |
Presence information must survive configuration resolution even after the final typed Go value is produced.
Applications should be able to determine, when necessary, whether an effective value was explicitly configured or defaulted.
Conceptual metadata API:
```go
result.Config
result.Metadata.IsExplicit("transcription.concurrency")
result.Metadata.Origin("transcription.concurrency")
result.Metadata.History("transcription.concurrency")
```
Exact API names are not yet fixed.
---
## 7. Defaults
### 7.1 Static defaults
Applications define static defaults, for example:
```go
func DefaultConfig() Config {
return Config{
Enabled: true,
Workers: 4,
Timeout: 30 * time.Second,
}
}
```
Explicit YAML values such as `false` and `0` must override those defaults.
Absent YAML values leave defaults intact.
### 7.2 Derived defaults
Defaults that depend on other resolved fields should be applied by an application-owned resolver after typed decoding.
Example:
```go
func ResolveConfig(cfg *Config, meta configkit.Metadata) error {
if !meta.IsExplicit("timeout") {
cfg.Timeout = cfg.PollInterval / 10
}
return nil
}
```
ConfigKit should provide a standard lifecycle hook for this but should not own application-specific derived-default logic.
---
## 8. `null`
`null` is **not** equivalent to absence.
Recommended v1 rule:
> `null` is an explicit value and is valid only where the application schema permits null.
For ordinary scalar fields such as:
```go
Workers int
Enabled bool
```
this should be invalid:
```yaml
workers: null
```
Nullable application semantics may be represented using suitable Go types such as pointers.
Do not use `null` in v1 to mean "reset this value to the application default."
If profile-level reset-to-default semantics are eventually required, add an explicit ConfigKit mechanism rather than overloading YAML `null`.
---
## 9. Strict Composition
Base configuration may be assembled from a main file plus imported fragments, including Debian-style `conf.d` directories.
Composition means:
> Multiple files contribute non-conflicting portions of one logical configuration.
A duplicate semantic value path across composed files is an error.
Example allowed:
```yaml
# 10-server.yaml
server:
host: localhost
```
```yaml
# 20-server.yaml
server:
port: 8080
```
These define different paths:
```text
server.host
server.port
```
Example rejected:
```yaml
# A
server:
port: 8080
```
```yaml
# B
server:
port: 9000
```
Both define:
```text
server.port
```
The diagnostic should identify both source locations.
### 9.1 Mapping behavior
Mappings compose recursively by semantic path.
### 9.2 Sequence behavior
Sequences do not implicitly compose or merge.
If two composed files define the same sequence path, that is a collision.
When configuration needs independently composable named entries, prefer mappings keyed by stable identifiers rather than lists.
---
## 10. Intentional Overlay
Overlay is a separate operation from composition.
Overlay means:
> A higher-precedence layer intentionally replaces values from a lower-precedence layer.
For mappings, overlay recursively by key.
For scalar values, the higher-precedence value wins.
For sequences, the higher-precedence sequence replaces the lower-precedence sequence in full.
Do not implement implicit sequence append, union, or index-wise merging in v1.
---
## 11. Profiles
Profiles are the first intended overlay mechanism.
Example layout:
```text
config/
├── narratio.yml
├── conf.d/
│ ├── 10-transcription.yml
│ ├── 20-artifacts.yml
│ └── 30-distribution.yml
└── profiles/
├── production.yml
└── testing.yml
```
Default invocation loads only the composed base configuration.
Example:
```text
narratio
```
Explicit profile selection:
```text
narratio --config-profile=testing
```
produces:
```text
application defaults
↓
main config + imports
↓
profiles/testing.yml
```
### 11.1 `default` profile
"Default" should mean **no profile overlay**.
Do not require a literal `profiles/default.yml`.
### 11.2 Profile names
`--config-profile` should accept a profile name, not an arbitrary path.
Profile names should use a constrained character set, e.g.:
```text
[a-zA-Z0-9_-]+
```
The selected profile should resolve deterministically relative to the configuration root.
### 11.3 Profiles should not redefine the configuration graph
Profiles should modify application configuration values, not change how ConfigKit discovers configuration.
Profile overlays should therefore not introduce new imports or redefine import behavior.
The base configuration graph is discovered and composed first; the selected profile is then applied to the resulting configuration tree.
---
## 12. Import Policy
Import behavior must be deterministic and explicitly specified.
Recommended v1 rules:
- relative imports resolve relative to the file containing the import;
- glob results are sorted deterministically;
- duplicate semantic values across imported fragments are errors;
- explicit missing imports are errors;
- an optional empty `conf.d` glob may resolve to zero files;
- import cycles are errors;
- cycle diagnostics should show the full cycle;
- define file identity carefully with respect to symlinks;
- consider constraining imports to an allowed configuration root;
- do not support remote/network imports in v1.
Remote imports create separate concerns around availability, caching, authentication, integrity, startup latency, and freshness and should remain out of scope.
---
## 13. Provenance and Definition History
Provenance is a first-class design requirement.
ConfigKit should retain, for each effective semantic value path:
- whether it was absent or explicitly configured;
- its effective value source;
- source file;
- line and column where practical;
- configuration layer;
- previous definitions overridden by higher-precedence layers.
Conceptually:
```go
type Definition struct {
Layer Layer
Source Source
Value NodeRef
}
type FieldHistory struct {
Definitions []Definition
}
```
Possible layers:
```go
const (
LayerDefault Layer = iota
LayerBase
LayerProfile
)
```
Future layers may include environment or CLI overrides.
This metadata should enable diagnostics such as:
```text
transcription.concurrency = 1
history:
application default 4
conf.d/10-transcription.yml:14 8
profiles/testing.yml:7 1
effective source:
profiles/testing.yml:7
```
---
## 14. Typed Decoding
After composition and overlays produce one effective presence-aware configuration tree, ConfigKit decodes it into the application's typed Go `Config`.
Requirements:
- unknown fields are errors;
- type mismatches are errors;
- diagnostics should preserve source location where possible;
- parser-specific errors should be converted into ConfigKit diagnostics.
Do not expose raw YAML parser types through the public API.
---
## 15. Semantic Validation
Application-specific validation remains application-owned.
ConfigKit should provide a standard validation hook and diagnostic type.
Validation should normally collect multiple independent errors rather than aborting after the first.
Example output:
```text
configuration invalid:
weather.sources.nws.timeout
must be > 0
config.yml:19:5
outputs.public
duplicate definition
config.yml:42:3
conf.d/90-output.yml:8:3
```
---
## 16. Schema Versioning
Applications should support explicit configuration schema versions.
Example:
```yaml
schema_version: narratio.v1
```
The version identifies the application's configuration schema, not the ConfigKit library version.
ConfigKit may later provide lifecycle support for application-provided migrations, but automatic migration is not required for the initial implementation.
The design should leave a clean pipeline location for schema-version checking and migration.
---
## 17. Secrets and Redaction
ConfigKit will eventually process configuration containing secrets.
Redaction must be considered before implementing:
- effective-config output;
- diagnostics;
- config diffs;
- logging;
- config fingerprinting.
Potential application annotation:
```go
APIKey string `yaml:"api_key" configkit:"secret"`
```
Exact mechanism is TBD.
Human-facing tooling must never accidentally emit raw secrets.
If fingerprints include secret values, the raw values must still never be exposed.
---
## 18. Diagnostics and Operability
ConfigKit should establish recognizable diagnostics across the application suite.
Potential standardized commands enabled by ConfigKit:
```text
<app> config validate
<app> config effective
<app> config sources
<app> config explain <path>
```
These commands do not all need to exist in the initial skeleton, but the metadata model should make them straightforward to implement later.
`config effective` must support secret redaction.
---
## 19. Non-Goals for v1
Do not include these in the initial implementation unless a concrete application requirement appears:
- hot reload;
- runtime resource replacement;
- remote/network imports;
- arbitrary configuration providers;
- TOML/JSON/HCL/CUE support;
- Vault/S3 configuration providers;
- sophisticated environment-variable overlays;
- arbitrary CLI field overrides;
- sequence append/union merge strategies;
- arbitrary custom merge policies;
- profile inheritance;
- reset-to-default operators;
- application runtime validation such as checking whether an S3 bucket exists.
ConfigKit should remain opinionated.
---
## 20. Initial Internal Architecture
A reasonable internal model is:
```text
Source
↓
Document
├─ parsed tree
└─ source locations
↓
Composer
strict combination
duplicate definitions = error
↓
Overlay
higher layer wins
mappings recurse
sequences replace
↓
Metadata / Provenance
presence
origins
definition history
↓
Decoder[T]
effective tree → typed Config
↓
Resolver[T]
application-derived defaults / normalization
↓
Validator[T]
application semantic validation
↓
Result[T]
```
Conceptual result:
```go
type Result[T any] struct {
Config T
Metadata Metadata
Sources []Source
Diagnostics []Diagnostic
Fingerprint string // optional / future
}
```
---
## 21. Possible Package Skeleton
Exact names are not settled, but an initial repository could look roughly like:
```text
configkit/
├── configkit.go
├── source.go
├── document.go
├── compose.go
├── overlay.go
├── metadata.go
├── decode.go
├── diagnostic.go
├── profile.go
├── import.go
├── schema.go
├── secret.go
└── internal/
└── yaml/
```
Public API should remain small.
A future happy path might resemble:
```go
result, err := configkit.Load[Config](
configkit.File(path),
configkit.WithImports(),
configkit.WithProfile(profile),
configkit.WithDefaults(DefaultConfig()),
configkit.WithResolver(ResolveConfig),
configkit.WithValidator(ValidateConfig),
)
```
This is illustrative only. Do not prematurely lock in option names or signatures.
---
## 22. Suggested Initial Implementation Sequence
### Stage 1 — Core document model
Implement:
- local file source;
- YAML parsing;
- conservative YAML validation;
- source locations;
- semantic path traversal;
- internal presence-aware tree representation.
### Stage 2 — Strict composition
Implement:
- recursive mapping composition;
- duplicate semantic-path detection;
- deterministic import ordering;
- useful duplicate diagnostics.
### Stage 3 — Profile overlay
Implement:
- constrained profile name resolution;
- recursive mapping overlay;
- scalar replacement;
- sequence replacement;
- provenance history across layers.
### Stage 4 — Typed decoding
Implement:
- generic `Load[T]` or equivalent;
- strict unknown-field rejection;
- typed errors converted to ConfigKit diagnostics.
### Stage 5 — Defaults and resolution hooks
Implement:
- static application defaults;
- explicit-value preservation for `false`, `0`, and empty strings;
- metadata query for explicitness;
- derived-default / resolver hook.
### Stage 6 — Semantic validation
Implement:
- application validation hook;
- multi-error diagnostics.
### Stage 7 — Introspection
Implement or prototype:
- source listing;
- effective-config rendering;
- `explain(path)`;
- secret redaction.
---
## 23. Tests That Define Policy
ConfigKit should have policy-focused tests, not merely implementation tests.
At minimum, test:
- absent bool receives default;
- explicit `false` overrides `true` default;
- absent integer receives default;
- explicit `0` overrides nonzero default;
- explicit empty string remains explicit;
- `null` differs from absent;
- duplicate key within one YAML file fails;
- duplicate semantic path across composed files fails;
- sibling mapping paths compose successfully;
- profile scalar overrides base scalar;
- profile mapping recursively overrides only specified children;
- profile sequence replaces base sequence;
- base sequence collision across composed files fails;
- profile cannot redefine import graph;
- imports are deterministically ordered;
- import cycles fail with useful diagnostics;
- unknown typed-config fields fail;
- provenance identifies effective source;
- provenance retains overridden definitions;
- defaults are distinguishable from explicit equal-valued configuration;
- secrets are redacted from introspection output.
These tests should become executable documentation for ConfigKit policy.
---
## 24. Key Decisions Summary
For v1:
1. YAML remains the configuration language.
2. ConfigKit defines a conservative YAML policy.
3. Parsed configuration remains presence-aware until after composition and overlays.
4. Application-facing Go structs remain idiomatic and strongly typed.
5. Absent, explicit zero/false/empty, and `null` are distinct.
6. Base imports use strict composition.
7. Duplicate semantic values across composed files are errors.
8. Profiles use explicit higher-precedence overlay semantics.
9. Mapping overlays recurse.
10. Sequence overlays replace.
11. "Default profile" means no profile overlay.
12. Profiles are names, not arbitrary file paths.
13. Profiles do not redefine the import graph.
14. Static defaults are application-owned.
15. Derived defaults are application-owned and presence-aware.
16. Provenance and definition history are first-class metadata.
17. Semantic validation is application-owned but follows a common lifecycle.
18. Parser implementation details remain private.
19. Secrets must be redactable.
20. Hot reload, remote providers, generalized merge strategies, and broad extensibility are non-goals until concrete requirements exist.
The core architectural objective is:
> **Build one deterministic, explainable configuration resolution model for the entire application suite.**

View File

@@ -1,3 +1,9 @@
# ConfigKit
An opinionated Go library for applications that use YAML configuration files.
ConfigKit is in pre-implementation. This repository currently contains its design
and contributor policies; no usable Go library has been implemented yet.
The [initial target design](docs/roadmap/initial.md) describes planned behavior.
See the [development guide](docs/development.md) for repository orientation.

View File

@@ -0,0 +1,37 @@
# ADR 0001: Separate Composition and Overlay
## Status
Accepted. This records a design decision, not implementation completion.
## Date
2026-09-06
## Context
Configuration fragments allow independent parts of an application to contribute
settings. Treating every file as a precedence layer makes accidental duplicates
silently depend on file order. Profiles, however, need intentional replacement.
## Decision
Use distinct strict-composition and intentional-overlay operations. Composition
rejects conflicting definitions; overlay provides explicit precedence. Keep the
operations opinionated rather than adding arbitrary per-field merge strategies.
The [architecture policy](../policy/architecture.md) owns the governing boundary;
the [target design](../roadmap/initial.md#composition-and-overlay) owns the planned
operation table until its current contract lands.
## Alternatives considered
- Last-wins loading for all files: convenient but conceals accidental conflicts.
- Strict loading for every layer: catches conflicts but cannot express profiles.
- User-defined merge policies: flexible but weakens consistency and expands the API.
## Consequences
Consumers must distinguish modular contributions from intentional overrides.
Diagnostics need both conflicting sources. File order remains useful for stable
traversal and reporting but cannot silently authorize replacement. Named mappings
are the natural representation for independently contributed entries.

View File

@@ -0,0 +1,38 @@
# ADR 0002: Own YAML Semantics Behind a Private Parser Boundary
## Status
Accepted. This records a design decision, not implementation completion.
## Date
2026-09-06
## Context
A shared library cannot provide consistent configuration semantics if each
consumer inherits parser conversions, node types, and extension behavior.
Exposing parser types also makes dependency changes part of the public API.
## Decision
ConfigKit defines its supported YAML policy and keeps parser integration private.
Application types and diagnostics must not require parser-specific public types.
Use a small internal boundary rather than a general parser/provider framework.
The [architecture policy](../policy/architecture.md) owns this constraint; the
[target design](../roadmap/initial.md#yaml-and-scalar-policy) owns planned syntax.
## Alternatives considered
- Expose parser nodes and hooks: easy integration but couples consumers to parser
behavior and permits bypassing library policy.
- Adopt the parser's entire YAML behavior: reduces initial code but leaves core
semantics outside ConfigKit's control.
- Abstract many configuration languages immediately: adds unsupported scope.
## Consequences
Parser changes require behavior-level compatibility verification. ConfigKit must
translate errors safely and enforce its own supported subset. Some existing
consumer custom decoders will need replacement with library-supported mechanisms
or application resolution. No particular parser dependency is selected here.

View File

@@ -0,0 +1,43 @@
# ADR 0003: Preserve Presence and Record Application Resolution
## Status
Accepted. This records a design decision, not implementation completion.
## Date
2026-09-06
## Context
Go zero values cannot distinguish omitted input from explicitly configured false,
zero, or empty text. Decoding immediately also loses information needed to explain
imports and overlays. Application normalization can subsequently make retained
input metadata disagree with runtime values.
## Decision
Preserve input presence and provenance separately from ordinary application Go
values. Keep application defaults and normalization application-owned, but record
resolution changes so final inspection and metadata describe the actual resolved
snapshot. Do not promise language-enforced immutability of exported Go values.
The [architecture policy](../policy/architecture.md) owns these invariants; the
[target design](../roadmap/initial.md#recorded-resolution-and-validation) owns the
planned recording mechanism and ownership semantics.
## Alternatives considered
- Infer presence from zero values: cannot preserve explicit zero-valued settings.
- Require pointers or optional wrappers everywhere: burdens application types and
confuses input presence with domain nullability.
- Retain only the composed input tree: cannot explain post-decode changes.
- Automatically diff mutations without reasons: detects changes but cannot explain
application intent or derivation dependencies.
## Consequences
Metadata and snapshot ownership are core concerns rather than optional debugging
features. Resolution hooks must participate in recording changes. Verification
must compare final values, origins, and rendering, including secret treatment.
This costs more than plain unmarshalling but removes repeated presence machinery
from consumers and supports trustworthy inspection.

View File

@@ -1,54 +1,56 @@
# Development
This is the contributor entry point for thsi application. Use the task-specific
reading guide below before making changes. Canonical architecture, contracts,
component behavior, and policies remain in their owning documents.
This is the contributor entry point for ConfigKit. Read the task-specific owners
below before making changes. The repository is in pre-implementation: it has no
Go module, packages, tests, or executable examples yet. The
[component overview](internal/overview.md) owns the implementation inventory.
## Initial Orientation
## Initial orientation
Before starting work:
1. Inspect the working tree and preserve unrelated changes.
2. Read the architecture policy for code or design work.
3. Read the policy, contract, and internal documents applicable to the task.
4. Inspect relevant implementation and tests where they exist before changing them.
1. inspect the working tree and preserve unrelated changes;
2. read the architecture policy for code or design work;
3. read the policy, contract, and internal documents listed for the task;
4. inspect the relevant implementation and tests before deciding how to change
them.
## Task-specific reading guide
Start with:
- [Architecture policy](policy/architecture.md) for system boundaries,
invariants, and non-goals;
- [Internal component overview](internal/overview.md) for the current package
and component map;
- [Documentation policy](policy/documentation.md) before changing
documentation;
- [Testing policy](policy/testing.md) before adding, rewriting, or deleting
tests.
## Task-Specific Reading Guide
| Task | Read before changing |
| Task | Read |
| --- | --- |
| Repository orientation or component responsibility | [Internal component overview](internal/overview.md) and [architecture policy](policy/architecture.md) |
| Examples or copyable assets | The owning contract for the demonstrated behavior and the related files under `examples/` |
| Architecture decisions or future work | The [documentation policy](policy/documentation.md) and relevant accepted ADRs |
| Repository orientation or component responsibility | [Architecture policy](policy/architecture.md) and [component overview](internal/overview.md) |
| Documentation changes | [Documentation policy](policy/documentation.md) and the owning document |
| Architecture decisions or future work | [Documentation policy](policy/documentation.md), [initial target design](roadmap/initial.md), and relevant ADRs listed below |
| Code and test changes | [Architecture policy](policy/architecture.md), [testing policy](policy/testing.md), and owning contracts/tests as they are introduced |
| Examples or copyable assets | The owning implemented contract and related maintained examples, once available |
| Release preparation | [Release policy](policy/release.md) and [release-readiness gaps](roadmap/release-readiness.md); the operational runbook will be added before release |
For cross-cutting changes, follow every applicable row. Internal component
documents own detailed subsystem change recipes.
Accepted decisions:
## Baseline Validation
- [Separate composition and overlay](adr/0001-separate-composition-and-overlay.md)
- [Own YAML semantics behind a private parser boundary](adr/0002-own-yaml-semantics.md)
- [Preserve presence and record application resolution](adr/0003-preserve-presence-and-resolution-history.md)
Use focused checks while iterating, then run validation proportionate to the
change and the risks described by the testing policy.
For cross-cutting changes, follow every applicable row. The implementation plan
remains deferred; `docs/roadmap/implementation.md` has not been created.
The repository-level baseline for code changes is:
## Baseline validation
```bash
go test ./...
go vet ./...
go build ./cmd/scriptorium
For the current documentation-only repository:
```sh
git diff --check
```
Documentation-only work does not require the full Go suite unless it changes
commands, examples, generated output, or another behavior that the suite
validates. Always check changed links, paths, examples, and canonical ownership.
Also inspect new untracked documents, verify changed local links and heading
fragments, and review paths, examples, ownership, and current-versus-planned
claims. The command above does not check links or untracked content.
Go validation is not runnable until a module and packages exist. The scaffolding
change must establish and verify library test, vet, and build commands here,
including their prerequisites. Do not use an application binary build as the
library baseline. Add executable-example and other checks when those artifacts
are introduced; do not document nonexistent tools as operational.
Once code exists, use focused checks while iterating and validation proportionate
to the change under the testing policy. Documentation-only work does not require
Go checks unless it changes behavior-bearing examples or commands they validate.
Release validation is a separate, stricter gate governed by release policy.

View File

@@ -1,13 +1,10 @@
# Internal Component Overview
## Purpose
ConfigKit has no implemented Go packages or components yet. This document will
inventory components and link to their current contracts and focused internal
documentation as they are introduced.
This is the inventory of this application's implemented components for contributors.
The [architecture policy](../policy/architecture.md) owns normative boundaries
and invariants; public behavior belongs in the linked contracts.
TODO: Add tables below, using the following format:
| Component | Implemented responsibility | References |
| --- | --- | --- |
| | | |
The [architecture policy](../policy/architecture.md) owns binding boundaries and
invariants. The [initial target design](../roadmap/initial.md) describes planned
behavior; it is not an inventory of implemented features. See the
[development guide](../development.md) for contributor orientation and validation.

View File

@@ -2,29 +2,119 @@
**Base Policy Version:** 1.0
**Last Updated**: 2026-09-06
**Last Updated:** 2026-09-06
## Base Policy Template
## Purpose and applicability
This document is the base template for the canonical architecture policy,. Repositories that implement this policy should:
ConfigKit is an opinionated Go library for consistent YAML configuration loading
and resolution across consuming applications. This policy governs contributions
now; it does not claim that the target features are implemented. The
[component overview](../internal/overview.md) owns implementation inventory and
[initial roadmap](../roadmap/initial.md) owns unimplemented target behavior.
1. Revise this document to include an architectural policy tailored to the implementing repository, including a description of the repository's purpose, overall intended system shape, and any architectural and/or engineering invariants that must be maintained during future development;
2. Ensure that any supporting documents referenced in the architecture policy are present on the filesystem, even if the initial content is simply `TODO`;
3. Update the `Last Updated` field to the current date; and
4. Remove this heading (`Base Policy Template`) and the associated text.
## Success criteria
As compared to other base policy documents, the Architecture Policy is the document that should be most heavily customized and tailored to the intended architecture of a particular application.
1. **Reduce application infrastructure:** Can applications delete substantial
configuration infrastructure while retaining clear ownership of their domain
rules?
2. **Unify configuration conventions:** Do consumers adopt one consistent
configuration model, with only the flexibility needed to express genuine
application requirements?
## Purpose
Both criteria must hold. ConfigKit should not preserve divergent consumer
conventions merely for compatibility, nor impose restrictions that prevent useful
domain information from being expressed. Prefer a canonical representation using
existing primitives before adding another mechanism. Extend the model when a
concrete requirement cannot be accommodated cleanly, not simply because a
consumer already implements a different convention.
This policy defines the purpose of this repository and the architectural principles that contributors
must follow during development.
Use both criteria to evaluate public API changes and consumer adoption. Shared
loading, presence tracking, source attribution, and diagnostic plumbing should
move out of applications; application types, defaults, normalization, and semantic
validation should remain recognizable application code. Replacing that plumbing
with equally repetitive callback bookkeeping does not meet the first criterion.
Offering alternate parsing, precedence, or composition conventions through escape
hatches does not meet the second.
TODO: Complete this document.
Consumer migrations are expected and may intentionally change configuration
syntax. Existing consumer code supplies implementation foundations and evidence
of needs; it does not define ConfigKit's compatibility obligations. Once ConfigKit
publishes its own contracts, changes to those contracts remain governed by the
[release policy](release.md).
Non-limiting examples of topics that may be addressed by an architectural policy include:
- Adherence to a hexagonal architecture, including whether external integrations/APIs must be encapsulated within internal adapters (strongly recommended)
- Whether a library is intended to provide a highly flexible API that supports a variety of downstream consumers, or a strongly opinionated API that is intended to enforce consistency across downstream consumers
- Whether an application should support a full DAG, or alternatively a strongly ordered sequence of discrete stages
- Whether the application should use specific external libraries to provide key functionality (e.g., FeedKit, ConfigKit, PromptKit)
- Any explicit architectural non-goals
Require representative simple and complex consumer examples to substantiate API
designs. Proposed extensions must identify the domain requirement, explain why
existing primitives cannot accommodate it cleanly, and show how the extension
preserves a coherent model. Application-specific interpretation and validation
remain appropriate. Reuse Narratio's applicable infrastructure under these
boundaries, and keep simple consumers independent of optional composition
features. The [consumer API design](../roadmap/initial.md#consumer-api-design)
describes the proposed surface; this policy does not claim it is implemented.
## Responsibility boundaries
ConfigKit owns configuration input processing and the coordination of decoding,
resolution, validation, and inspection. Applications own their configuration
types, field meanings, defaults, normalization, semantic validation, and schema
version policy. Applications also own main-file search conventions, CLI wiring,
remote acquisition, runtime initialization, and resource checks.
Library code must not depend on consuming applications or initialize their
runtime resources. It must not perform implicit network access, discover
application-specific paths, or change process-global state to load configuration.
Remote data can enter through application-supplied input without introducing a
network provider into the library.
## Public API and dependency direction
Keep one small, coherent public surface for the supported lifecycle. Simple
single-document consumers must not need import, profile, or provider machinery.
Do not expose parser-specific nodes, errors, or extension hooks through the
public API. Parser integration belongs behind a private boundary, as recorded
in [ADR 0002](../adr/0002-own-yaml-semantics.md).
Separate input access, document processing, composition, decoding, metadata, and
rendering responsibilities without prescribing an interface or package for each.
Dependencies must preserve those boundaries; package splits and abstractions
must earn their complexity. Do not impose a full application or hexagonal
framework on this library.
## Configuration integrity
Composition and intentional overlay are distinct operations. Composition must
not silently become last-wins override through file ordering. The decision and
alternatives are recorded in [ADR 0001](../adr/0001-separate-composition-and-overlay.md).
ConfigKit must define supported YAML semantics explicitly rather than inherit
parser defaults. Processing order and diagnostics must be deterministic for the
same inputs and options. Invalid input must not produce a successful usable
configuration. Do not run application hooks on invalid typed configuration.
Preserve presence and source information through processing. Ordinary Go zero
values must not be used to infer whether input was absent. Final configuration,
metadata, and effective inspection must agree, including application resolution
changes; see [ADR 0003](../adr/0003-preserve-presence-and-resolution-history.md).
Detailed syntax and operation tables belong to the target roadmap until their
current configuration contract is implemented.
## Ownership and confidentiality
Returned configuration and metadata must not share mutable state across loads.
ConfigKit must not mutate a result after returning it. Ordinary exported Go
values are not language-enforced immutable objects; document caller ownership
and the lifetime of captured metadata without promising otherwise.
Secrets must be considered throughout error handling and inspection. Library
renderers must apply secret classification, and parser errors must not be
forwarded blindly when they can include raw input. Do not log configuration
implicitly. Applications remain responsible for classifying their secrets and
for safe messages supplied by their hooks.
## Scope boundaries
Do not introduce automatic directory discovery, network providers, arbitrary
merge strategies, additional configuration languages, or application runtime
management without concrete consumer evidence and an explicit design decision.
The roadmap owns the complete deferred-feature inventory. Significant boundary
changes require an ADR and corresponding policy/contract updates; implementation
convenience alone is not a reason to broaden the public API.

View File

@@ -2,170 +2,126 @@
**Base Policy Version:** 1.0
**Last Updated**: 2026-09-06
## Base Policy Template
This document is the base template for the canonical documentation policy. Repositories that implement this policy should:
1. Review the list of documents under the `Canonical Ownership` heading, remove any entries that do not currently apply and are not expected to apply to the current project, and add any additional entries that may be required by the current project;
2. Ensure that each referenced document is present on the filesystem, even if the initial content is simply `TODO`;
3. Update the `Last Updated` field to the current date; and
4. Remove this heading (`Base Policy Template`) and the associated text.
**Last Updated:** 2026-09-06
## Purpose
This policy assigns each documentation topic to one canonical owner. Its goal is
to keep this application's documentation accurate, concise, discoverable, and resistant
to drift for users, operators, developers, integrators, and LLM coding agents.
Assign each documentation topic one canonical owner so ConfigKit's documentation
remains accurate and useful to consumers, maintainers, and implementation agents.
## Core Rules
## Core rules
### One Canonical Owner
### One canonical owner
Each authoritative fact belongs in one document. A non-owning document may give
a short, stable summary for orientation, but it must link to the canonical owner
instead of repeating volatile details.
Each authoritative fact belongs in one owning document or Go declaration's
reference documentation. Other documents may summarize and link to that owner;
they must not maintain competing definitions of commands, API contracts, YAML
semantics, defaults, paths, diagnostics, or compatibility guarantees.
Volatile details include commands, flags, configuration fields and defaults,
module keys, schemas, file names, paths, status codes, retry behavior, and
runtime guarantees. If readers could reasonably treat a statement as a
contract, maintain it only in the owning document.
### Policies, current behavior, and future work
### Current And Future Behavior
- Policies state currently binding contributor rules, including constraints on
future implementation. They do not imply that features already exist.
- Current API/configuration contracts, consumer guidance, and component
documentation describe implemented behavior only, including partial boundaries.
- ADRs record accepted decisions and rationale. Acceptance is not evidence of
implementation; execution status belongs in roadmaps.
- Roadmaps own unimplemented targets, deferred work, and execution status.
Outside `docs/roadmap/`, documentation describes implemented behavior only.
Partial features may be described only to their implemented boundary.
The initial target and implementation plan have separate owners. Do not put
staged steps or progress in `docs/roadmap/initial.md`. The future
`docs/roadmap/implementation.md` will own those details when it is created.
As behavior lands, move its normative definition into the current contract and
replace the roadmap definition with a link. Retire completed roadmap material
as part of release preparation; ADRs preserve significant decision history.
ADRs are the narrow exception: an ADR may record an accepted architectural
decision before implementation, but acceptance must not be presented as proof
that the behavior exists. The roadmap owns implementation status and sequencing
until the decision is implemented. Current architecture, user, operator,
integration, and internal documentation are updated when the behavior lands.
### Audience and examples
### Audience And Detail
Consumer documentation explains how to use the library and what consumers own.
Internal documents explain mechanisms and dependencies. Keep implementation
mechanics out of consumer guidance unless they affect a meaningful choice.
Write for the document's stated audience and include only the detail needed for
its owned topic. User and operator docs should not expose implementation detail.
Developer docs should link to user-facing and external contracts rather than
restate them.
Complete copyable artifacts belong in `examples/`; Go API examples may live in
executable Go example tests beside the relevant package. Prose should use the
smallest useful excerpt and link to the maintained artifact. Examples must be
secret-free and checked against implemented behavior. Clearly label illustrative
future syntax in roadmaps; do not present it as a runnable current example.
### Examples
### Security and privacy
Complete copyable files belong in `examples/`. Documentation may use the
smallest illustrative snippet needed to explain its owned topic, but should link
to maintained examples instead of embedding a second complete copy.
Do not include real credentials, private keys, environment dumps, sensitive input,
or private infrastructure details unless intentionally public. Document secret
handling, not secret values.
Examples must be valid, secret-free, and tested where practical. Commands and
configuration used in documentation should match the application.
## Canonical ownership
### Security And Privacy
| Topic | Owner | Boundary |
| --- | --- | --- |
| Product orientation | `README.md` | Current status, purpose, navigation, and a minimal working example once available |
| Contributor entry point | `docs/development.md` | Reading guide and actual baseline validation commands |
| Architecture rules | `docs/policy/architecture.md` | Responsibilities, dependency constraints, invariants, and scope boundaries |
| Documentation rules | `docs/policy/documentation.md` | Ownership and document lifecycle |
| Testing rules | `docs/policy/testing.md` | Risk-based sufficiency, test boundaries, and maintenance principles |
| Release rules | `docs/policy/release.md` | Compatibility, integrity, gates, immutability, and exceptions |
| Release procedure | `docs/release.md`, when operational | Exact maintainer commands, prerequisites, supported targets, and verification |
| Exported Go API | Go documentation beside declarations | Symbol contracts, errors, ownership, and lifecycle API behavior |
| YAML and resolution semantics | `docs/config.md`, as behavior lands | Syntax, composition, precedence, conversion, and metadata semantics; Go reference docs link here for shared rules |
| Consumer walkthroughs | `docs/consumers/`, when useful | Task-oriented usage and consumer responsibilities, not duplicate API reference |
| Component inventory | `docs/internal/overview.md` | Implemented components and links to their owners |
| Component mechanics | Other `docs/internal/` files, when needed | Detailed implementation behavior and change recipes |
| Architectural decisions | `docs/adr/` | Context, decisions, alternatives, consequences, and supersession history |
| Initial target | `docs/roadmap/initial.md` | Unimplemented end-state and deferred feature scope |
| Implementation plan | Future `docs/roadmap/implementation.md` | Stages, dependencies, acceptance gates, and progress |
| Release readiness gaps | `docs/roadmap/release-readiness.md` | Outstanding release prerequisites and closure evidence |
| Historical release notes | `docs/releases/` | Version-specific historical summaries required by release policy |
| Complete examples | `examples/` and executable Go examples | Maintained artifacts; prose links instead of duplicating full examples |
Documentation and examples must not contain real credentials, private keys,
private environment dumps, sensitive source material, or private infrastructure
details unless intentionally public. Document secret-handling mechanisms, not
secret values.
A listed future owner is not a requirement to create an empty file now. Create
API, consumer, example, and component documents when the corresponding behavior
exists and there is useful content to own. Link only to existing files; use plain
code paths for planned files. Release prerequisites have the explicit temporary
exception recorded in the release policy.
## Canonical Ownership
ConfigKit has no owned HTTP service, application CLI, or runtime operations
interface. Do not create those reference documents for consuming applications.
| Topic | Canonical owner | Owned content | Content owned elsewhere |
| --- | --- | --- | --- |
| Product orientation and minimal end-to-end quickstart | `README.md` | What this application is, why it is useful, one shortest successful invocation, and links onward. | Complete command reference, configuration reference, operational procedures, implementation detail. |
| Contributor entry point | `docs/development.md` | Task-oriented reading guide, minimal contributor orientation, baseline validation commands, and links to canonical docs. | Package inventory, architecture rules, subsystem behavior, detailed change recipes. |
| Application architecture policy | `docs/policy/architecture.md` | System shape, normative ownership, dependency direction, architectural boundaries, invariants, safety properties, and non-goals. | Concrete package inventory, implementation mechanics, contributor procedures, decision history, future work. |
| Documentation policy | `docs/policy/documentation.md` | Documentation ownership, audience boundaries, maintenance rules, and ADR/document lifecycle. | Application architecture or product behavior. |
| Testing policy | `docs/policy/testing.md` | Test philosophy, risk-based sufficiency, test boundaries, doubles, coverage guidance, regression-test policy, and criteria for adding, rewriting, or deleting tests. | Subsystem behavior, application contracts, subsystem-specific test inventories, and implementation plans. |
| Release policy | `docs/policy/release.md` | Versioning and compatibility rules, release integrity and immutability requirements, release-documentation policy, required gates, publication controls, verification requirements, and exception policy. | Repository-specific commands, product installation guidance, CLI version semantics, historical release summaries, and CI implementation detail. |
| Repository-specific release procedure | `docs/release.md` | Maintainer prerequisites, candidate preparation, concrete validation and publication commands, repository and registry guards, deliverable publication, consumer-facing verification, and failure recovery. | General release requirements, product installation guidance, historical release summaries, and current compatibility contracts. |
| CLI contract (if applicable) | `docs/cli.md` | Commands, arguments, flags, invocation semantics, and exit codes. | End-to-end operating procedures, configuration field definitions, runtime filesystem layout, module implementation details. |
| Configuration contract (if applicable) | `docs/config.md` | Discovery and precedence, file schema, fields, defaults, environment overrides, validation rules, and user-selectable module or validator keys. | Complete example files, CLI syntax, runtime state lifecycle, module implementation details. |
| Operations (if applicable) | `docs/operations.md` | Runtime workflows, physical filesystem and state layout, output, cache, and debug handling, resume, cleanup, permissions, recovery, and operational limits. | CLI flag syntax, configuration field definitions, logical output schemas, implementation mechanics. |
| Public HTTP contract (if applicable) | `docs/api.md` | Routes, authentication, media types, request and response schemas, status codes, pagination, caching, idempotency, rate limits, and HTTP retry semantics. | Client walkthroughs, upstream or downstream integration internals, implementation detail. |
| Consumer guidance for the public Go API (if applicable) | `docs/consumers/` | Task-oriented use of the public interface, minimal client examples, and consumer responsibilities. | HTTP wire semantics, external protocol contracts, internal implementation detail. |
| Integration contracts for external components (if applicable) | `docs/integrations/` | External file formats and protocols, upstream and downstream contracts, logical output bundle paths and schemas, media types, and compatibility behavior. | Physical runtime placement and lifecycle, internal transformations, CLI syntax, configuration defaults. |
| Inventory of implemented internal packages | `docs/internal/overview.md` | Current packages and components, their implemented responsibilities, and links to focused internal docs. | Normative architecture, contributor reading policy, external contracts. |
| Behavior and architecture of internal packages | Other files under `docs/internal/` | Implementation flow, internal collaborators and state transitions, package-local guarantees and failures, and relevant tests. | Global architecture invariants, configuration definitions and defaults, external schemas, operator procedures. |
| Architectural decision records | `docs/adr/` | Significant decisions, context, alternatives, rationale, consequences, and supersession history. | Current behavior reference, implementation status, task sequencing. |
| Future work and implementation status | `docs/roadmap/` | Proposed, accepted, deferred, or rejected work; implementation status; sequencing; and task breakdowns. | Implemented behavior reference and architectural decision rationale. |
| Release-note history | `docs/releases/` | One checked-in historical summary for each source release made under the procedure. The note at the immutable tag is that release's record. | Current commands, behavior, contracts, and compatibility definitions. |
| Complete copyable artifacts | `examples/` | Maintained configuration, inputs, and other files intended to be copied or run. | Field-by-field reference, command reference, prose explanation. |
## Boundary rules
Documents that do not exist are required only when the corresponding interface
or responsibility exists. Do not create placeholder API, consumer, integration,
or operations documents for behavior the application does not have.
Architecture owns governing constraints; ADRs explain why; contracts describe
implemented semantics; internal documentation explains how. A roadmap may link
to all of them but must not become a second current reference.
## Boundary Rules
Release policy owns guarantees; the release procedure owns commands. A release
note cannot redefine a compatibility contract or replace a current-documentation
update. The note at an immutable tag is that release's historical record.
### Orientation
This policy owns what documentation may disclose. Architecture owns security
invariants. Current configuration and Go API contracts own secret-classification
and rendering behavior when implemented; internal docs own their mechanisms.
The README owns product orientation. The developer guide routes contributors.
Architecture owns normative structure. Internal overview owns the current
concrete component map. These documents may link to one another but should not
maintain parallel package or behavior descriptions.
## Architectural decision records
### Commands, Configuration, And Operations
Use sequential filenames, such as `0001-separate-composition-and-overlay.md`,
and the lightweight Nygard structure: title, status, date, context, decision,
alternatives considered, and consequences.
CLI documentation answers how to invoke the application. Configuration
documentation answers what settings mean. Operations answers what happens to
runtime state and how to operate or recover the application. When a workflow
crosses these topics, choose the document that owns the task and link to the
other contracts.
Accepted decision content is immutable. A changed decision requires a new ADR
and a superseded status/link in the old one. Navigation links may be repaired
when specification ownership moves without rewriting historical rationale.
Rejected architectural alternatives belong in ADRs; deferred or rejected product
features belong in roadmaps. Do not create an ADR for every routine choice.
### Releases
## Maintenance and verification
The release policy defines requirements and invariants that every release must
satisfy. The repository-specific release procedure defines the commands and
workflow maintainers use to satisfy that policy. Release notes are immutable
historical summaries, not current-state contract owners: the checked-in note at
an immutable tag records that release, while current canonical documentation
must change with the behavior it describes. Do not use a release procedure or
release note to weaken policy, redefine current behavior, or defer required
documentation updates.
### Contracts And Implementation
Integration and API documents define externally observable shapes and
semantics. Internal documents explain how this application implements or consumes those
contracts. Internal docs may name a field, file, or protocol to identify a
dependency, but must link to its canonical contract for the definition.
### Security Topics
This policy owns what documentation and examples may contain. Architecture owns
application security invariants. Configuration owns credential-supply
mechanisms. Operations owns permissions and handling of sensitive runtime
artifacts. Internal docs own implementation mechanisms only.
## Architecture Decision Records
Use sequentially numbered ADR filenames such as
`0001-record-architecture-decisions.md`. Follow the lightweight Nygard format:
1. title;
2. status;
3. date;
4. context;
5. decision;
6. alternatives considered;
7. consequences.
Treat the decision content of an accepted ADR as immutable. When a decision
changes, create a new ADR and update the earlier ADR's status to superseded.
Rejected architectural alternatives belong in the ADR; rejected product ideas
belong in the roadmap.
## Maintenance
When behavior changes, update its canonical owner in the same change. If
ownership moves, remove the old definition and replace it with a link where
navigation remains useful.
Update the owning current documentation in the same change that implements or
changes behavior. When ownership moves, remove the old definition and preserve
useful navigation links.
Before completing documentation work:
- verify affected behavior and examples;
- check commands, flags, fields, defaults, schemas, and paths against their
implementation;
- keep unimplemented behavior in the roadmap, subject to the ADR exception;
- remove stale references and validate links;
- confirm that non-owning documents summarize and link rather than redefine;
- confirm that no secrets or sensitive private data were added.
- verify claims against the implementation or clearly identify future design;
- check changed local links, paths, fragments, examples, and commands;
- preserve the distinction between policy, decisions, target behavior, and status;
- check canonical ownership and remove stale or duplicate definitions;
- inspect formatting and whitespace;
- ensure no sensitive data was introduced.

View File

@@ -2,30 +2,7 @@
**Base Policy Version:** 0.1.0
**Last Updated**: 2026-09-06
## Base Policy Template
This document is the base template for the canonical release policy.
Repositories that implement this policy should:
1. Retain the substantive requirements of this policy unless a documented
repository constraint requires an exception;
2. Define the repository-specific release procedure, validation commands,
supported deliverables, and verification commands in `docs/release.md`;
3. Ensure that every document and script required by the adopted policy and
procedure is present on the filesystem;
4. Add stricter repository-specific requirements where the product, delivery
channels, or threat model require them;
5. Record every exception under `Policy Exceptions`, including its rationale
and compensating controls;
6. Update the `Last Updated` field to the current date; and
7. Remove this heading (`Base Policy Template`) and the associated text.
The base policy is a standard, not a collection of optional recommendations.
An implementing repository should customize its release mechanics without
silently weakening the common integrity, validation, documentation,
verification, or immutability guarantees below.
**Last Updated:** 2026-09-06
## Purpose
@@ -38,6 +15,26 @@ The repository-specific commands that implement these requirements belong in
`docs/release.md`. Historical release summaries belong in `docs/releases/`.
Neither document may weaken this policy.
## ConfigKit applicability
ConfigKit adopts a source-only Go module release model. The source tag and
version-matched release note are the declared deliverables. Standalone binaries,
container images, and separate binary/package-registry publication are not
required. This does not waive source compilation, module hygiene, testing, or
consumer-facing module retrieval and build verification.
The release procedure must define the module path, authoritative remote,
supported Go versions and operating-system/architecture targets, prerequisite
dependency acquisition, and the exact verification commands once established.
The default tests themselves remain deterministic and offline. Additional
publication channels require documented deliverables and verification before use.
The Go API, documented YAML/resolution semantics, and structured diagnostic
contracts all participate in compatibility review. A signature-preserving change
can still break consumers. Human diagnostic wording is stable only where a
contract expressly promises it. Minimum Go version and platform support changes
also require an explicit compatibility assessment.
## Release Model
A release consists of:
@@ -138,7 +135,7 @@ The required gates are:
- validation of maintained examples, configuration, schemas, generated
sources, and documentation commands where applicable;
- review of changed local documentation links;
- removal of obsolute roadmap documents under `docs/roadmap/` that describe
- removal of obsolete roadmap documents under `docs/roadmap/` that describe
completed work included in the release;
- review for unintended files, generated output, credentials, private data,
and unrelated changes; and
@@ -220,3 +217,22 @@ name the primary branch, authoritative remote, tag form, published
deliverables, supported build targets, release automation responsibilities,
and consumer verification path. It must link to this policy instead of
restating or redefining its requirements.
## Policy Exceptions
### Temporary bootstrap deferral
The release procedure (`docs/release.md`) and complete release-check command are
not yet present because ConfigKit has no Go module or implementation to validate.
Their creation is temporarily deferred during initial development. This exception
permits implementation work; it does not permit publishing a release without
any gate required above.
Compensating controls: no release may be published until the
[release-readiness gaps](../roadmap/release-readiness.md) are closed, the runbook
and checker exist, and the exact candidate passes all applicable gates. The
maintainer preparing the first release owns closure. Remove this temporary
exception when readiness is verified. Any other exception requires a documented repository constraint, rationale,
compensating controls, scope, and closure condition here; none is currently
adopted. Do not silently weaken the base integrity, validation, documentation,
verification, or immutability guarantees.

View File

@@ -2,17 +2,7 @@
**Base Policy Version:** 1.0
**Last Updated**: 2026-09-06
## Base Policy Template
This document is the base template for the canonical testing policy,. Repositories that implement this policy should:
1. Review the policy, remove any rules that must not apply to the current project, and add any additional rules that are required by the current project;
2. Update the `Last Updated` field to the current date; and
3. Remove this heading (`Base Policy Template`) and the associated text.
Most projects will not require any revisions to the substantive testing policy rules as written below.
**Last Updated:** 2026-09-06
## Purpose
@@ -20,6 +10,29 @@ Our tests exist to make **incorrect changes expensive and correct changes cheap*
We do not optimize for test count, line coverage, exhaustive isolation, or the fewest possible tests. We optimize for sufficient confidence in important behavior while imposing as little unnecessary friction as possible on future development.
## ConfigKit-specific priorities
Configuration semantics are compatibility contracts, even when Go signatures do
not change. Apply the risk rules below especially to presence/default behavior,
composition, provenance, and error classification. Keep detailed behavior cases
with their owning contracts and tests rather than duplicating them in this policy.
Use real temporary files for import traversal, symlinks, and source identity.
Document platform prerequisites for filesystem cases and make any unsupported
case explicit rather than silently dropping its coverage. Use labeled in-memory
input where filesystem behavior is irrelevant.
Verify agreement among final resolved values, presence, provenance, and effective
rendering. Protect against secret disclosure on both successful inspection and
failure paths, and against mutable defaults or results leaking across loads.
Use focused fuzzing for parser boundaries and semantic/filesystem path handling;
keep deterministic seed regressions in the ordinary suite. Extended fuzz runs
need a documented bounded invocation when introduced.
Baseline commands belong in the [development guide](../development.md). Release
gates belong in the [release policy](release.md) and its eventual runbook. This
policy does not imply that those tools or tests already exist.
## Every test has a cost
Testing is not an unqualified good. Every test imposes both an immediate cost and a continuing lifetime cost.
@@ -59,7 +72,8 @@ Use a **classical/Detroit-style** approach:
- Prefer package-level behavioral tests over tests coupled to private helpers or internal call sequences.
- Treat exact collaborator interactions as testable behavior only when the interaction itself is a requirement.
Examples of appropriate seams include clocks, randomness, subprocesses, remote APIs, object storage, email, and paid LLM calls.
For ConfigKit, controlled input readers and filesystem failures are useful seams.
Prefer actual parsing, composition, and decoding over mocks of their call order.
## Test execution requirements
@@ -176,13 +190,15 @@ Use a test-controlled limit and measure the behavior relative to that limit. Do
Each behavior should have a clear test owner.
- Parser tests own parsing cases.
- Validator tests own validation rules.
- Domain tests own transformations and invariants.
- Adapter tests own external integration behavior.
- Orchestrator tests own coordination and failure propagation.
- CLI tests own argument and configuration mapping.
- End-to-end tests prove that representative assembled workflows work.
- YAML boundary tests own syntax and policy enforcement.
- Composition and overlay tests own their operation semantics.
- Decoding tests own type binding, defaults, and strictness.
- Metadata and rendering tests own provenance queries and safe inspection.
- Filesystem tests own source access and import identity behavior.
- Lifecycle tests own hook ordering, failure propagation, and representative
agreement across the assembled configuration pipeline.
These are responsibility boundaries, not a required package layout.
Higher-level tests should not repeat every lower-level case. A single intentional policy change should not require unrelated edits across many test files.
@@ -215,7 +231,6 @@ Use:
- table-driven tests for meaningful behavioral categories and boundaries;
- `t.TempDir()` for real filesystem behavior;
- `httptest.Server` for realistic HTTP interactions;
- fuzz tests for parsers, normalization, path handling, and broad input spaces;
- golden files only when the complete output is intentionally stable;
- integration tests where correctness depends on component interaction; and
@@ -223,7 +238,8 @@ Use:
Avoid exact error-string assertions unless the wording is itself contractual. Prefer `errors.Is`, `errors.As`, typed errors, or structured error fields.
At CLI boundaries, prefer exit classifications, structured output, and the smallest stable semantic fragment needed to identify the error. Do not snapshot complete diagnostic wording unless it is contractual.
At the public library boundary, prefer diagnostic codes, paths, and source
locations over complete message snapshots. Assert wording only when contractual.
Golden-file updates must require an explicit local flag. CI must not update golden files automatically, and reviewers must inspect the semantic diff before accepting an update.
@@ -235,7 +251,7 @@ Coverage is a diagnostic, not a target.
Use it to find untested critical branches and unexpectedly weak packages. Do not write low-value tests solely to increase a percentage, and do not infer test quality from coverage alone.
Pure domain logic will often warrant higher coverage than CLI wiring or external adapters. Uneven coverage is acceptable when it reflects risk.
Dense configuration semantics will often warrant higher coverage than simple input wiring. Uneven coverage is acceptable when it reflects risk.
Increasing coverage is valuable only when the newly covered behavior protects a meaningful risk at an acceptable cost.
@@ -303,7 +319,7 @@ A test suite is sufficient when:
- legitimate internal changes usually do not require test edits; and
- additional tests would mostly repeat existing protection or preserve inconsequential implementation details.
Sufficiency is a risk judgment, not a coverage percentage or test count. Reassess it as the application, its users, and the consequences of failure evolve.
Sufficiency is a risk judgment, not a coverage percentage or test count. Reassess it as the library, its consumers, and the consequences of failure evolve.
The governing rule is:

View File

@@ -0,0 +1,784 @@
# ConfigKit Initial Implementation Plan
## Status and ownership
Status: ready for implementation; all stages are pending.
Audience: a `gpt-5.6-terra` coding agent implementing one stage per prompt, in
ascending order. Each stage depends on every earlier stage being complete.
Do not combine stages merely because they share files. Finish the requested
stage's acceptance checks and handoff before moving to the next requested stage.
The [feature roadmap](initial.md) owns the desired end-state and consumer intent.
This document owns execution decisions, tasks, acceptance gates, and progress.
Read [development](../development.md), all four policies
([architecture](../policy/architecture.md), [documentation](../policy/documentation.md),
[testing](../policy/testing.md), [release](../policy/release.md)), and the relevant
[ADRs](../development.md#task-specific-reading-guide) before implementation.
Both [success criteria](../policy/architecture.md#success-criteria) apply to every
stage. This plan does not authorize feature expansion or downstream application
releases.
## Execution rules and stage handoff
1. Inspect the actual worktree, prior stage evidence, relevant source, and tests.
Preserve unrelated work. This repository initially has documentation only.
2. Read the linked feature sections for the requested stage. Follow links to
current contracts if earlier work has moved a definition out of the roadmap.
3. Extract applicable Narratio code and cases before writing equivalent new
mechanisms. Record source revision, symbols, adaptations, and retained notices.
Do not import sibling applications or their internal packages into this module.
4. Implement only the stage boundary. Partial internals may be tested directly;
do not export placeholder APIs, silently ignore unsupported options, or claim
that later features work. Introduce public operations only when their stated
supported boundary is functional. No release occurs between these stages.
5. Run focused checks while iterating, then the stage baseline. Add durable tests
at the owning boundary; do not replicate every case through `Load`.
6. Update Go reference comments, implemented configuration contracts, component
inventory, and examples in the same change as behavior. Move implemented
normative text from the feature roadmap to its current owner, leaving links.
Keep this plan's tasks and completion evidence here, not in the feature roadmap.
7. Set the stage's status to complete only after its acceptance gates pass. Append
a short evidence entry under that stage: files/contract links, extraction
references, checks and outcomes, deviations, and remaining later-stage work.
Do not treat compilation against stubs as runtime acceptance.
Baseline after Stage 1: `go test ./...`, `go vet ./...`, `go build ./...`, and
`git diff --check`, plus inspection of new files, documentation links/fragments,
and formatting. These are future implementation commands; Stage 1 establishes
them in the developer guide. Tests must run without network access or credentials
after explicit dependency acquisition. Do not put network calls in default tests.
For a failed gate, fix the implementation or explicitly resolve a contract issue;
do not change expectations solely to get a pass. Record external tool/platform
limitations accurately. Do not mark a required check passed if it was skipped.
## Implementation decisions
These decisions close choices left illustrative in the feature roadmap. They
are instructions for implementation, not additional configuration conventions.
Promote their externally observable parts into current contracts when implemented.
| Area | Decision and reason |
| --- | --- |
| Module and package | Module `gitea.maximumdirect.net/eric/ConfigKit`, matching the existing origin's case; root package `configkit`. No `cmd` application. |
| Go baseline | `go 1.25.0`; use a currently supported patched toolchain at or above that minimum for development, and verify minimum-version compatibility separately. The existing Narratio/Seriatim baselines and local 1.25.5 toolchain support this language baseline. No toolchain directive forcing a newer consumer toolchain. |
| Parser | Pin `go.yaml.in/yaml/v3 v3.0.5` initially, inside `internal/yaml`. It preserves the v3 node-oriented extraction seam while using the YAML organization's maintained module. Do not inherit its scalar coercions. The [upstream v3 reference](https://pkg.go.dev/go.yaml.in/yaml/v3@v3.0.5) documents this version and provenance. |
| Internal representation | A ConfigKit-owned tree in `internal/document`: ordered mapping entries, sequences, typed scalar kind plus original numeric spelling, null, key/value locations, and source ID. It imports neither the public package nor the YAML parser. The YAML adapter translates into it. |
| Dependency direction | Public package coordinates document operations and reflection binding. Private parser imports `internal/document`; document code never imports parser or public package. Keep composition, binding, provenance, hooks, and rendering coordination in root-package private files until a demonstrated boundary needs another package. |
| Public spelling | Implement the feature roadmap's `Load`, `Definition`, `Options`, sources, path builders, metadata queries, hook helpers, version rule, result methods, and diagnostic methods as written. No functional-options framework or separate builder. |
| Input limits | Fixed initial ceilings: 8 MiB per source, 64 MiB total loaded bytes, depth 256 for input/projected trees, 64 active import edges, and 1,024 accepted sources. Allow at most 1,000,000 input nodes in aggregate and 1,000,000 nodes in each composed/projected tree; copying a snapshot does not consume the input budget again. Bound glob directory-entry visits separately at 1,000,000 per load. Include the selected profile in source/byte counts. Root depth is zero. No public limit-tuning options initially. Test limit mechanisms with smaller private test parameters. |
| Filesystem support | Initial supported targets: Linux and macOS on amd64/arm64. Test real symlink semantics on those systems. No Windows path-compatibility promise in this release. Inputs use slash-separated relative import syntax and native paths for selected file/root/base options. |
| Schema implementation | Build one reflection descriptor graph per load, reuse it throughout that load, and avoid global mutable caches initially. Recursive type definitions are permitted when actual values are acyclic. |
| Working configuration | Clone the defaults factory's result before applying input. Decode explicitly present nodes into that detached baseline according to schema descriptors; never round-trip defaults through YAML. Snapshot final values independently. |
| Error boundary | Safe library diagnostics with phase-specific codes; no raw parser/custom-hook error string or raw-value wrapper chain. `LoadError` supports `errors.As`; there is no initial `Unwrap` promise. I/O classifications are exposed as diagnostic codes. |
| Consumer integration scope | Build copied, sanitized consumer-shaped models/fixtures and executable examples in ConfigKit. Actual changes to Narratio, Notarius, WeatherReporter, and Seriatim are separate consumer migration tasks. Do not claim those apps are migrated when these local examples pass. |
| Publication scope | Prepare library source and a release runbook/checker. Do not tag, push, publish, or invent a completed release during these stages. Release-candidate validation/publication follows the adopted release procedure as a separate maintainer action. |
The parser dependency choice is based on its own published documentation, not a
promise that v3.0.5 will always be the newest patch. A later verified dependency
fix can change the pin with focused YAML regression checks and normal review.
Do not silently switch parser families or use `@latest` during implementation.
### Metadata and diagnostics layout
Use opaque `Path`, `Source`, `Metadata`, `Resolution`, and `Validation` state.
Expose detached value records for inspection, with these initial fields:
| Type | Fields |
| --- | --- |
| `Location` | `SourceID string`, `Line int`, `Column int`; zero coordinates mean unavailable/synthetic |
| `SourceInfo` | `ID string`, `Label string`, `Path string` (canonical local path or empty), `Layer Layer`, `ImportedFrom *Location` |
| `Diagnostic` | `Code Code`, `Severity Severity`, `Path Path`, `Message string`, `Primary *Location`, `Related []Location`; absent primary means no known source |
| `Origin` | `Layer Layer`, `Locations []Location`, `Operation ChangeKind`; multiple locations represent a composite origin |
| `FieldInfo` | `Explicit bool`, `Null bool`, `Origin Origin` |
| `Change` | `Path Path`, `Origin Origin`, `Previous []Origin`, `Reasons []string`, `Dependencies []Path`; no public historical values |
`Layer` is a string enum: `default`, `base`, `profile`, `resolution`. A Go zero
baseline uses layer `default` without claiming an application deliberately chose
every zero value. `ChangeKind` has the empty value for input/default origins,
`derived_default`, and `normalization`. `Severity` is `error` or `warning`.
Define exported constants with the type prefix (for example `LayerBase`,
`ChangeDerivedDefault`, `SeverityError`).
Use stable string codes grouped by `options.*`, `source.*`, `yaml.*`, `schema.*`,
`decode.*`, `compose.*`, `import.*`, `profile.*`, `version.*`, `resolve.*`, and
`validate.*`. Include distinct missing/permission/read, duplicate, invalid-path,
limit-exceeded, hook-failed, and unrecorded-change classifications where relevant.
Application reports use their phase's `error`/`warning` code, not arbitrary code
registration. Define exact exported code constants with their owning feature;
private helpers do not need new public codes for every branch.
Sources get deterministic per-load IDs `source-1`, `source-2`, and so on in
traversal order. The selected profile is last. `NoSource` creates no source ID.
Public query slices and nested slices/pointers are detached. Default fmt/error
rendering must not reveal hidden value histories.
### Clarifications required for a coherent implementation
- `ParsePath` retains textual numeric tokens until traversal knows the container;
`Keys("0")` is a mapping key and `Index(0)` is a sequence index. Reject invalid
escapes and negative indices. The zero `Path` is the root. Empty mapping keys
remain representable. Resolve parsed numeric indices without leading zeros;
the same text may address a literal mapping key when traversing a mapping.
- Path queries on missing fields return the feature-defined empty result. Hook
diagnostics permit schema-valid missing fields/map entries; malformed paths or
unknown struct fields produce a hook diagnostic. Version hooks use the untyped
composed tree and nearest known container because decoding has not run yet.
- Defaults apply to the surviving composed/profile input; intermediate overridden
nodes do not create reset markers. An effective null suppresses defaults at
its path. Missing children of an effective mapping can retain defaults.
- Applying nonempty input to a nil pointer-to-struct creates the object and uses
zero values for omitted children. No per-entry defaults are invented. Applying
a sequence allocates a replacement; it never overlays default indices.
- Support paired text marshal/unmarshal scalar methods on value or pointer method
sets, with exact `time.Duration` handled first. Treat such scalars atomically.
Their canonical text supplies projection and detached reconstruction. Never
call YAML-specific marshal/unmarshal hooks; an otherwise ordinary struct with
such a method is bound structurally under ConfigKit's rules.
- Ignore ordinary unexported and `yaml:"-"` fields; zero them in detached ordinary
struct results rather than preserve hidden mutable aliases. Paired text scalars
are the exception: their declared scalar codec owns their internal state.
Reject unsupported schema kinds and unsupported `yaml` tag options rather than
silently reinterpret them. Support `inline` and ignore `omitempty`; defer `flow`.
- Repeated helper writes to one path compare only the last helper-assigned value
with the final projected value. Coalesce reasons/dependencies. For mixed
operations, a path record is normalization if any record normalized it; otherwise
it is a derived default. Record coverage of a container includes descendants.
- Query explicitness against surviving input contributions. A normalization of an
existing path preserves its input explicitness. A newly created path is not
explicit just because the resolver copied another explicit field; dependencies
carry its explanation. Deletions remain historical, not effective entries.
- Validator mutation is a programming error detected by before/after projection;
return `validate.mutation` and no result. Do not recover user callback panics.
Retained contexts are invalid after the hook. Calls on a closed context panic
with a fixed, value-free programming-error message and cannot mutate a published
result; this is callback misuse, not a late load diagnostic.
- The initial implementation assumes stable, application-controlled local input
trees during a load. Resolve symlinks and enforce containment, but do not claim
atomic multi-file snapshots or protection against concurrent hostile filesystem
replacement. Reject directories and non-regular file targets; arbitrary streams
are accepted only through the explicit reader API.
## Stage 1 — Establish the module and extraction baseline
Status: pending.
Read: feature [foundation](initial.md#existing-implementation-foundation),
[consumer fit](initial.md#consumer-fit-and-intentional-migrations), all policies.
Tasks:
1. Create the module and root package documentation with the decisions above.
Keep the existing repository license. Add no application command, runtime
dependencies, framework, or placeholder exported loader.
2. Reinspect Narratio's referenced revision and current worktree. Create
`docs/internal/extraction.md` recording source revisions/symbols and the exact
loading/presence/default/diagnostic cases to extract. Include applicable notices
with copied material. Copy only minimal sanitized fixtures needed by subsequent
stages; no audio, transcripts, secrets, or entire application package trees.
3. Document the planned parser pin here; add its module requirement only when the
real adapter imports it in Stage 3 so tidy does not leave unused dependencies.
4. Update the developer guide to runnable Go baseline commands and prerequisites.
Correct its stale claim that this plan is deferred. Update documentation-policy
navigation to this now-existing plan without changing ownership rules. Record
the implemented root package in the overview and retain pre-feature status.
Acceptance:
- Module path/case agrees with origin, root package builds, and baseline passes.
- Extraction notes distinguish copied/generalized code from new work and domain
logic retained in consumers. No sibling checkout is required to build/test.
- No misleading usable-loader example or unrelated files enter the module.
## Stage 2 — Implement semantic paths and safe diagnostic records
Status: pending.
Read: feature [paths](initial.md#paths-and-presence),
[diagnostics](initial.md#provenance-diagnostics-and-redaction), public metadata API.
Tasks:
1. Implement `Path`, builders, parsing/stringification, private traversal helpers,
and the metadata/diagnostic value types specified above. Keep records detached.
2. Implement diagnostic accumulation, canonical ordering, safe `LoadError`,
`WriteDiagnostics`, and error-code constants needed so far. Source-less errors
sort before source errors; then source order, line/column, path, code, severity,
and safe message break ties. Canonicalize related locations deterministically.
3. Render control characters in labels/messages/paths as escaped text so source
labels cannot corrupt terminal output. Never include values or parser errors.
4. Document the implemented path and diagnostic API next to declarations.
Acceptance:
- Round-trip escaped slash/tilde/empty/dotted/numeric keys and indexed paths;
distinguish a malformed path from a valid missing path.
- `errors.As` finds `LoadError`; mutation of returned nested records cannot alter
later results. Safe writer failures propagate, including partial output.
- Reordered independent diagnostic submissions render identically; no blanket
golden snapshot of unstable prose is required. Baseline passes.
## Stage 3 — Extract source reading and enforce YAML policy
Status: pending.
Read: feature [YAML policy](initial.md#yaml-and-scalar-policy),
[source constructors](initial.md#entry-point-and-reusable-application-definition).
Tasks:
1. Adapt Narratio's file/reader loading helpers into the opaque source constructors
and bounded private reader. Add `NoSource`, byte snapshotting, source labels,
regular-file checks, deterministic I/O classification, and reader single-use
state shared by source copies. File loads reopen each time; readers are not closed.
2. Add the pinned YAML adapter and ConfigKit-owned document tree. Decode into parser
nodes only; translate key/value locations and scalar spelling before discarding
parser-specific nodes. Do not use parser typed decode for application structs.
3. Enforce exactly one mapping document, duplicate keys, tag/alias/anchor/merge-key
rejection, and ConfigKit's own scalar grammar. A second successful decode is
an error, including an empty document; do not preserve Narratio's trailing-doc bug.
4. Enforce byte ceilings before parsing and tree depth/node ceilings during adapter
translation. The parser can reject stricter malformed/nesting inputs first, but
none may panic or expose input text. Add private small-limit injection for tests.
5. Create `docs/config.md` for implemented syntax/input behavior and link from
package documentation. Move the implemented definitions out of the feature
roadmap, preserving navigation for this plan.
Acceptance:
- Real file and reader tests cover missing/permission/read failures, non-regular
targets, byte snapshots, second reader use, and no caller-reader close.
- Policy tables cover duplicate keys, all forbidden YAML constructs, null spellings,
quoted literals, JSON numeric syntax, legacy booleans/numbers, timestamp-like
strings, multiple/empty documents, and root type. Numeric nodes never round via float.
- Small-limit boundary tests and malformed input tests return safe diagnostics with
available coordinates. This stage does not promise a typed public `Load` yet.
## Stage 4 — Build the reusable schema binding graph
Status: pending.
Read: feature [typed decoding](initial.md#typed-decoding) and
[type/projection contract](initial.md#supported-types-and-projection).
Tasks:
1. Implement reflection descriptors for supported root structs/maps, scalar kinds,
pointers, nested structs, string-keyed maps, slices, fixed arrays, and `any`.
2. Bind exported field names, tags, inline structs/pointers and the optional inline
map. Detect ambiguous names, multiple inline maps, unsupported tag options,
conflicting defaults-map names (checked against runtime defaults in Stage 6),
unsupported kinds, and missing paired text codecs. Identify secret-tag paths.
3. Support recursive descriptors without infinite construction; actual value cycles
are checked by projection. Use descriptors for schema-aware path validation.
4. Keep reflection traversal private. Do not expose generic subtree conversion,
module registries, YAML hooks, or interfaces for arbitrary decoders.
Acceptance:
- Focused tests cover nested/inline conflicts, ignored/private fields, maps of
structs, named scalar types, pointer method sets, root rejection, recursive
acyclic models, and unsupported kinds. Descriptor building has no side effects.
- No application parser-specific hook executes. Baseline passes.
## Stage 5 — Implement canonical projection and detached copies
Status: pending.
Read: feature [ownership](initial.md#recorded-resolution-and-validation) and
[type/projection contract](initial.md#supported-types-and-projection).
Tasks:
1. Use Stage 4 descriptors to project ordinary typed values into a ConfigKit tree
without YAML serialization. Include zero values and preserve nil/empty distinctions.
2. Clone ordinary configuration into independently owned values. Reconstruct paired
text scalars through canonical text into fresh instances; preserve type identity
and supported dynamic concrete values without retaining mutable input aliases.
3. Detect cyclic values, non-finite floats, invalid dynamic types, projection limit
violations, and custom codec errors safely. Do not reveal the codec error text.
4. Add structural comparison used later by resolution audit and validator-mutation
detection. Track paths of changed values, container replacements, and removals;
do not create an index-wise correspondence between replaced sequences.
Acceptance:
- Mutating input/default maps, pointers, slices, or paired text-scalar backing data
cannot change a clone or snapshot. Ignored fields do not preserve hidden aliases.
- Nil versus empty, exact wide integers, durations, canonical scalar text, recursive
acyclic objects, cycles, unsupported dynamic values, and codec failures are covered.
- Comparison reports meaningful paths and container changes; baseline passes.
## Stage 6 — Decode scalar and container values over defaults
Status: pending.
Read: feature [defaults/null](initial.md#static-defaults-and-null) and typed decoding.
Tasks:
1. Implement schema-driven typed decoding into a Stage 5 clone of the fresh default
baseline. Allocate replacements for sequences and recurse mappings by the feature
contract; do not let parser behavior determine merging or optionality.
2. Enforce numeric syntax/range and nullability. Support dynamic `int64`/`float64`,
finite floats, exact-duration string parsing, named scalar types, paired text
codecs, arrays with exact length, and strict nested map-of-struct decoding.
3. Reject unknown fields, tag binding conflicts, and invalid defaults consistently.
Collect independent decode errors while leaving the failed working object private.
4. Return a private decoded configuration plus its surviving input presence index,
projected baseline/final trees, and per-path default/input origins for Stage 7.
Defaults never become explicit; equal-valued input remains explicit.
Acceptance:
- A compact table proves absent/false/zero/empty/null behavior, default map entries,
`{}`, `[]`, pointer objects, new entries, dynamic null, and non-nil default isolation.
- Overflow, non-finite values, quoted-number coercions, integer durations, inline
conflicts, and unknown nested fields fail at original input paths/coordinates.
- Adapt Narratio's presence/default test cases with intentional differences noted.
Repeated loads do not alias each other. Baseline passes.
## Stage 7 — Assemble single-source loading and schema-version checks
Status: pending.
Read: feature [lifecycle](initial.md#resolution-lifecycle),
[version hook](initial.md#schema-checking-and-secret-classification),
[results](initial.md#results-errors-and-inspection).
Tasks:
1. Introduce `Definition`, `Options`, `Load`, and `Result` at the single-source
boundary. Wire source/options checks, parsing, version checking, fresh defaults,
decode, and detached result. Accept zero or one options argument.
2. Implement `VersionRule`, `Version`, and the pre-decode `Validation` report context,
including safe error/warning messages, related input paths, absent version from
`NoSource`, and generic fallback only when no explicit error was reported.
3. Implement base/default `Metadata.Lookup`, `IsExplicit`, `History`, source listing,
and warning/error lists. Keep snapshot internals private. No effective writer
is exposed before its redaction implementation in Stage 15.
4. Reject `_configkit` when imports are disabled. Until their later stages land,
reject non-nil resolver/validator hooks and import/active-profile options with
an explicit unreleased-feature options error rather than ignore them. Document
only the supported single-source boundary. Remove each gate in its owning stage.
5. Ensure `NoSource` performs version/default/decode/projection processing with no
source records and rejects filesystem/import/active-profile settings.
Acceptance:
- Actual `Load` integration tests cover File/Bytes/Reader/NoSource and nil result on
source, schema, version, or decode failure. Defaults factory runs only after a
successful version check and once per valid load.
- String version, missing/null policy, integer rejection, warnings, actionable
safe reports, raw error suppression, and invalid options are exercised.
- Metadata reflects input versus defaults and detached ownership. Baseline passes.
## Stage 8 — Implement recorded application resolution
Status: pending.
Read: feature [resolution callbacks](initial.md#resolution-and-validation-callbacks).
Tasks:
1. Add per-load `Resolution`, `Default`, `Normalize`, `Record`, `Error`, and `Warning`.
Hook context queries refer to surviving input, not guesses from typed zero values.
2. Run the resolver after decoding; capture before/final projection, detect net
changes, and require record coverage. Support map-entry copy/writeback helpers,
container records, deletions, dependencies, repeated writes, and equal-valued
derived-default intent using the clarification rules above.
3. Verify final helper values against declared paths. Reject invalid paths, invalid
kinds, invalid projected values, missing records, or use after context closure.
Store reasons/dependencies without raw public history values.
4. Attach phase diagnostics and stop on reported errors even with nil return. A
non-nil return adds generic safe failure only if no error was already reported.
Leave callback panics as programming failures. Remove the resolver options gate.
Acceptance:
- Execute the feature's timeout and dynamic artifact examples against real code.
- Test explicit-null/zero preservation, baseline replacement for absent input,
normalization of explicit input, map writeback, unrelated pointer/path mismatch,
unrecorded mutation, equal-value intent, repeated helper writes, and subtree removal.
- Mutating result configuration cannot change resolution snapshots/history. No
hooks or errors expose secrets. Independent map traversal yields stable records.
## Stage 9 — Add semantic validation and finish the single-source lifecycle
Status: pending.
Read: feature [diagnostic callbacks](initial.md#resolution-and-validation-callbacks)
and [schema reports](initial.md#schema-checking-and-secret-classification).
Tasks:
1. Reuse `Validation` for post-resolution schema-aware application reports. Validate
primary and related paths, supply nearest source locations for omitted values,
and collect multiple independent errors/warnings.
2. Detect validator mutation through projection and fail without publishing a
result. Close contexts after callbacks and remove the validator options gate.
3. Finalize all single-source metadata after resolution/validation success. Ensure
failure carries only safe diagnostics and sources, not a partially usable config.
4. Add executable examples for minimal/defaults-only loading, related-path duplicate
errors, version guidance, and resolution followed by validation. Document actual
hook lifecycle and private-field/runtime-state boundaries.
Acceptance:
- Hook-order tests prove later phases never run after prior errors; warnings continue.
- Test missing fields, dynamic keys, related locations, validator mutation, and safe
hook failure fallback. Compile and run representative examples, not API stubs.
- Compare value, explicitness, origin, and history after a complete single-source load.
Baseline passes. Imported/profile loading is still deliberately unavailable.
## Stage 10 — Implement strict document composition
Status: pending.
Read: feature [composition table](initial.md#composition-and-overlay).
Tasks:
1. Implement a private strict composer over ConfigKit document trees. Recurse only
mapping pairs; treat any other present overlap as a collision, including equal
values, null, and sequences. Empty mappings contribute no children.
2. Retain both locations for collisions and aggregate composite container origins.
Preserve input order only for traversal/diagnostics, never implicit precedence.
3. Build provenance updates alongside composition; no subsequent pass may invent
an arbitrary single source for a multi-source mapping. Avoid mutating input trees.
4. Keep the composer private and test it through the smallest stable private
boundary; public file graph wiring belongs to subsequent stages.
Acceptance:
- Exercise each operation-table category plus ancestor/scalar conflicts, empty maps,
sequence collisions, equal-value duplicates, and disjoint named entries.
- Duplicate diagnostics identify both exact source locations; retained inputs and
origins survive without aliasing. No default layer participates in strict collision.
## Stage 11 — Implement explicit file-import traversal
Status: pending.
Read: feature [import graph](initial.md#explicit-imports-and-source-identity).
Tasks:
1. Parse the reserved root directive and `path` declarations strictly; preserve
import-site locations and strip directives before application decode. Recognize
`glob` declarations but retain an explicit temporary unsupported gate until Stage 12.
2. Resolve paths from their declaring source, canonicalize symlinks, enforce root
containment, and perform deterministic depth-first traversal. Keep active-stack
and visited indexes distinct. Track both routes for cycles and repeated imports.
3. Read accepted sources once; enforce count/depth/aggregate budgets. Optional paths
suppress only target nonexistence. Invalid directives, directories, unreadable
files, canonicalization failures, and out-of-root targets remain errors.
4. Reject schema-version contributions from fragments before composition, even when
no main-file version exists. Ensure ancestor replacements of a nested version
path cannot smuggle a fragment version past the placement rule.
5. Wire composition and version checking into `Load`, removing the import gate for
supported path declarations. Document the temporarily limited import boundary.
Acceptance:
- Temporary-directory fixtures cover nested relative paths, list-order traversal,
symlink aliases, full-cycle routes, diamond repeats, optional versus required
missing files, out-of-root traversal, regular-file checks, and budget boundaries.
- Version placement and duplicate value diagnostics retain their owning sources.
- Bytes/reader imports require explicit base/root; URI labels never imply a path.
## Stage 12 — Complete glob imports and filesystem option handling
Status: pending.
Read: feature import syntax and source-context API.
Tasks:
1. Implement segment glob matching with sorted slash-relative results and no `**`.
Use an error-reporting directory walker: do not use a convenience glob API that
silently swallows permission/I/O errors. Reuse Stage 11 identity and traversal.
2. Reject invalid patterns before walking; canonicalize and contain visited
directories/targets, including symlinked directories. Optional zero matches
must not conceal a walk or permission failure. Bound accepted candidates before
accumulating unbounded source work; charge directory-entry visits to their
separate traversal budget, including entries that do not match the pattern.
3. Validate all filesystem context combinations, profile-name-independent root setup,
absolute import/URL rejection, and selected-file versus imported-file identity.
`default` means no active profile for `NoSource` checks.
4. Remove the glob gate. Exercise explicitly declared `conf.d/*.yml` without adding
any implicit directory scan or extension fallback.
Acceptance:
- Fixtures cover lexical order, multiple declarations, overlapping matches, matches
that are directories, hard links as separate paths, symlink escapes, unmatched
optional/required patterns, invalid patterns, and visible permission failures.
- Repeated runs produce identical source IDs, composition, and diagnostic ordering.
Full import contract and baseline pass.
## Stage 13 — Implement profile overlays and wire final precedence
Status: pending.
Read: feature [profiles](initial.md#profiles), composition/overlay table.
Tasks:
1. Implement private recursive overlay with atomic scalar/sequence/null and
cross-kind replacement. Record overridden definitions as the operation occurs.
2. Resolve exactly one constrained profile name to its fixed `.yml` path under the
root, with canonical containment. Empty/lowercase `default` means no profile;
selected missing profiles fail. No path inputs, inheritance, or extension fallback.
3. Reject profile directives and schema-version contributions before overlay.
Also reject an ancestor scalar/null replacement that would remove a declared
nested schema-version value; an unchanged nested version supplied by a profile
is still forbidden.
4. Apply profile after base graph composition/version checking and before defaults
and decoding. Remove the last optional-feature gate from `Load`.
Acceptance:
- Profile scalar, mapping sibling, full sequence, empty sequence, null, and cross-kind
replacement cases obey the operation table. Defaults remain lower precedence.
- Removed descendants disappear from effective presence/origin, remain explainable
through history, and cannot inherit stale sequence indices.
- Base YAML policy errors are not masked by profiles; effective-tree schema checks
and all profile graph/version restrictions pass. Baseline passes.
## Stage 14 — Verify layered provenance and final snapshot agreement
Status: pending.
Read: feature [presence](initial.md#paths-and-presence),
[history](initial.md#provenance-diagnostics-and-redaction), metadata API.
Tasks:
1. Integrate default, base, profile, and resolution records into a coherent query
index; reuse prior-stage provenance rather than reconstructing sources from values.
2. Finalize histories for replaced containers, empty mappings, deleted paths, and
whole-sequence changes. Retain history lookups for removed paths without claiming
those paths exist in the effective snapshot. Canonicalize reasons/dependencies.
3. Test composition/profile/resolution interaction through one representative public
workflow and fill only uncovered cross-layer cases. Keep operation-table coverage
in its owning tests instead of duplicating it all through `Load`.
4. Update metadata reference docs with concrete composite and synthetic origin cases.
Acceptance:
- A configured value equal to its default remains explicit; derived values do not.
- Final typed projection, lookup/null state, effective origins, source order, and
histories agree after parent and sequence replacement plus resolution.
- All returned metadata/source/diagnostic nested values are detached, and successive
or concurrent independent loads cannot contaminate one another. Baseline passes.
## Stage 15 — Implement secret classification and effective rendering
Status: pending.
Read: feature [redaction](initial.md#provenance-diagnostics-and-redaction),
[secret predicate](initial.md#schema-checking-and-secret-classification), result API.
Tasks:
1. Combine descriptor secret tags with the application path predicate over current
and historical paths. A secret container marks descendants. Apply dependency
propagation to a fixed point so callback/map submission order is irrelevant;
cycles in dependency annotations terminate and propagate sensitivity conservatively.
2. Ensure replaced/deleted secret definitions remain classified. The predicate can
add sensitivity, never remove tagged/historical sensitivity. Do not expose raw
retained values through debug formatting, diagnostic wrappers, or public records.
3. Add `Result.WriteEffective` over the frozen canonical snapshot. Sort mapping keys
by decoded string, preserve sequence order, use two-space indentation and a final
newline, and substitute the quoted string `[REDACTED]` at secret roots. Preserve
supported nil/empty distinctions. No unredacted option or implicit logging.
4. Marshal only an adapter tree made from the safe projection; never call application
YAML marshal hooks. Finish all text-codec projection work before returning a
successful load. Writer errors may leave partial redacted output.
Acceptance:
- Canary secrets stay absent from ConfigKit-rendered errors, warnings, metadata,
effective YAML, and histories across parse/decode/hook failures and overwritten
definitions. Check ordinary and Go-syntax formatting of metadata/error wrappers
so private histories cannot leak. Typed `Result.Config`, including application
formatting of it, is the explicit trusted boundary.
- Test secret containers, dynamic paths, dependencies, cycles, custom scalar values,
snapshot mutation isolation, and writer failures. Document safe application message
responsibility and that redacted output is not promised reloadable.
- Render actual resolved values including defaults/normalization; baseline passes.
## Stage 16 — Prove consumer ergonomics and migration contracts
Status: pending.
Read: feature consumer-fit table, module options, external values, and both criteria.
Tasks:
1. Create maintained executable examples and minimal synthetic fixtures for four
consumer shapes: Narratio-like defaults/partial imports/artifacts; Notarius-like
object bindings/string versions/dynamic options; WeatherReporter-like defaults-only
and report/module options; Seriatim-like ordered speaker/autocorrect rule files.
2. Generalize the relevant copied Narratio tests and helpers, recording source
revision and notices. Use actual consumer field patterns but no private data or
dependencies on sibling application modules. Keep runtime algorithms in example
application functions rather than adding library APIs for them.
3. Compare preserved outcomes against consumer evidence and list intentional changes:
explicit empty/null behavior, canonical objects, versions, integer representation,
strict YAML, and retired environment/CLI configuration overrides. Verify imports
and profiles do not absorb application LLM/pipeline-profile semantics.
4. Convert every feature-roadmap API example into a real compiled/run example or a
focused behavioral test; document the happy path in README and `docs/consumers/`.
Verify dynamic options without YAML reserialization or generic subtree conversion.
5. Add a short consumer-fit assessment to current consumer guidance: what plumbing
disappears, what domain code remains, and what downstream migrations are still
separate tasks. Do not use line-count targets as acceptance criteria.
Acceptance:
- All maintained YAML examples load as intended; negative migration fixtures fail
with stable classifications and useful paths, not copied raw parser text.
- Simple consumers need no imports/profiles and no custom parser hooks. Complex
examples preserve domain meaning under the same conventions.
- No external configuration reads, environment mutation, LLM calls, remote services,
or sibling checkout requirements occur in default tests. Baseline passes.
## Stage 17 — Harden boundaries and verify independent loads
Status: pending.
Read: testing policy, ownership/projection, import limits, redaction guarantees.
Tasks:
1. Add bounded fuzz targets for path parsing, YAML translation, and document
composition/overlay invariants. Retain deterministic regression seeds for any
defects found; avoid a large permanent corpus of redundant cases.
2. Add parallel independent-load coverage for shared read-only definitions and
fresh defaults; exercise defensive-copy and custom-scalar paths under the race
detector. Source/reader single-use behavior must be race-safe, not silently reused.
3. Review private limit enforcement through reads, traversal, projection, and
rendering. Add only missing consequential boundary tests. Prevent unbounded
expansion through permissive aliases/providers; these remain unsupported.
4. Run default suite, vet/build, and `go test -race ./...`. Run each selected fuzz
target with `-run='^$' -fuzz=<exact-name> -fuzztime=30s -parallel=2` in its owning
package. Record exact names/commands in the developer guide. No fuzz loop in CI
is required beyond deterministic seeds unless separately configured.
Acceptance:
- Race checks and bounded fuzz runs pass on a supported native platform, with any
discovered defects fixed and retained at their narrowest durable test boundary.
- No unexplained test skips, mutable shared defaults, raw-secret render paths,
unsupported-feature gates, or stale placeholder APIs remain.
- A fresh offline baseline after dependency acquisition passes without reading any
developer-specific configuration path.
## Stage 18 — Prepare release validation and reconcile documentation
Status: pending.
Read: release policy in full and [release-readiness gaps](release-readiness.md).
Tasks:
1. Create `docs/release.md` with executable commands for the existing origin/main,
module path, source-only tags, candidate recording, branch/tag/cleanliness guards,
precise publication refs, and clean consumer module retrieval/build verification.
Include private-host access prerequisites without credentials or environment dumps.
2. Implement `scripts/release-check.sh` as a fail-fast source-candidate checker.
It must perform module hygiene (tidy in a disposable copy and diff manifests),
cached dependency verification, offline default tests, native race tests, vet,
native builds, `CGO_ENABLED=0` cross-builds for the four supported targets,
formatting/whitespace checks including untracked files, maintained examples,
and local documentation-link checks. Acquire toolchains/dependencies explicitly
before running the offline checker. Use the baseline toolchain and a newer
available supported toolchain when assessing the declared Go support range.
3. Make links to sibling consumer source files optional only when those checkouts
are absent in isolated release environments: verify local links to current
ConfigKit owners strictly, preserve recorded external revision references, and
document the external-reference applicability instead of requiring private repos.
4. List manual release gates: compatibility/migration review, secrets/unintended-file
inspection, platform execution evidence where native hosts differ, roadmap
retirement, release-note review, exact remote commit/tag guards, and post-publication
retrieval. Static-analysis baseline is vet; do not silently claim a linter was run.
5. Exercise the checker on current source and reconcile all documentation links,
API comments, `docs/config.md`, consumer examples, and component inventory.
Replace implemented feature-roadmap definitions with current contract links;
repair ADR navigation without changing accepted rationale. Keep this execution
plan until release preparation retires completed roadmap material.
6. Update release-readiness evidence for the actual runbook/checker. Keep any
unvalidated release-candidate/publication gap explicit and retain its bootstrap
guard until the maintainer closes it. Do not fabricate release notes for an
unpublished candidate or mark remote verification completed. No publication is
authorized by this stage.
Acceptance:
- The checker actually passes applicable automated gates, and the runbook explicitly
owns every manual/inapplicable gate required by policy. No nonexistent commands
are presented as operational. Any missing native-platform or publication evidence
remains an explicit release gate, with its owner and closure check recorded.
- Current docs describe implemented behavior; no duplicated roadmap contracts or
stale deferred-plan statements remain. All feature areas below have completion
evidence, and both architecture success criteria are demonstrated locally.
- No tag, push, downstream migration, or release publication occurred as part of
implementation. Maintainer release preparation remains a separate procedure.
## Coverage and completion audit
Use this map to ensure completion means the whole feature roadmap, not merely a
working loader. The linked feature sections own semantics; stages own work.
| Feature area | Owning stages |
| --- | --- |
| Extraction, ordinary consumer types, package boundaries | 1, 4–6, 16 |
| YAML subset, precise numeric syntax, safe source failures | 2–3, 6 |
| Paths, presence, origins, definition history | 2, 6–8, 10, 13–14 |
| Static defaults, null, container rules, snapshot ownership | 5–7, 14, 17 |
| File/bytes/reader/defaults-only inputs and filesystem context | 3, 7, 11–12 |
| Version hook and graph-placement restrictions | 7, 11, 13 |
| Strict imports, glob ordering, identity, limits, collisions | 10–12, 17 |
| Named profiles, precedence, atomic replacement | 13–14 |
| Recorded normalization/derived defaults and hook errors | 8–9, 14 |
| Semantic validation, related diagnostics, safe messages | 2, 7–9 |
| Secret tags/predicate/dependencies and effective inspection | 5, 15, 17 |
| Module-specific options, consistency and intentional migrations | 16 |
| Runnable docs, examples, source-release checks and runbook | 1, 16, 18 |
| Deferred-feature boundaries | Every stage; final audit in 18 |
At final handoff, report completed stages, exact checks, current documentation
owners, source-extraction evidence, and remaining release/consumer migration work.
A green unit suite alone is not proof that examples, metadata safety, filesystem
behavior, documentation, or both success criteria are satisfied.
## Open Questions
No unresolved product or API question blocks the ordered stages above. The
implementation decisions explicitly select the module identity, Go baseline,
parser, limits, supported platform targets, metadata shapes, and consumer scope.
Do not reopen settled feature policies solely to preserve a legacy convention.
Two external execution facts must still be verified when their stage runs; they
are not authorization to silently weaken acceptance:
1. **Is a native Linux runner available for the declared support matrix?**
Recommended: execute filesystem/race tests on native Linux and macOS before
release; cross-builds alone cannot verify symlink/permission behavior. Stage 18
can prepare the checker on the available host while keeping missing native
evidence as an explicit release gate. Alternative: formally narrow the initial
supported platforms to those with execution evidence, with a documented scope
decision; this is more honest than asserting untested support, but less useful
for the likely server consumers. Neither option blocks the platform-independent
stages or permits claiming a skipped native check passed.
2. **Are the authoritative host and module retrieval credentials available at
publication time?** Recommended: verify the exact version from a clean consumer
module after authorized publication using the host's normal access mechanism;
keep credential setup outside ConfigKit and out of tests. Alternative: use an
explicitly configured internal Go proxy mirroring the exact immutable tag and
verify its source identity; it adds infrastructure but can suit restricted
environments. Local replacement directives are useful for development but are
not a viable substitute for release retrieval verification.

1122
docs/roadmap/initial.md Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,24 @@
# Release Readiness
## Status and ownership
ConfigKit is not release-ready. This document tracks outstanding release
prerequisites under the [temporary bootstrap exception](../policy/release.md#temporary-bootstrap-deferral).
It is not the library implementation plan and does not assign implementation
stages. The maintainer preparing the first release owns closure.
## Outstanding prerequisites
| Gap | Evidence required to close |
| --- | --- |
| No Go module or declared support matrix | Established module path, supported Go versions and platform targets documented in the release procedure |
| No release runbook | `docs/release.md` exists with the prerequisites, commands, publication guards, and verification required by release policy |
| No complete release checker | A documented command exists and exercises every applicable automated candidate gate; manual gates and inapplicable checks are explicit |
| No consumer verification procedure | Runbook verifies retrieval of the exact released module version and a representative consumer build without local replacement directives |
| No validated release candidate | Version-matched release note, current contracts/examples, and complete checks against the exact committed candidate |
No release can proceed while these prerequisites remain open. Close items only
on evidence from actual files and checks, not proposed commands. Record closure
evidence here while this roadmap remains active. Once all gaps are closed,
remove the bootstrap exception and retire this document in favor of the
operational runbook and normal release records.