Finalize the ConfigKit roadmap and implementation plan
This commit is contained in:
@@ -77,11 +77,12 @@ execution steps belong in the [implementation plan](implementation.md).
|
||||
|
||||
| ConfigKit owns | Applications own |
|
||||
| --- | --- |
|
||||
| Local file and labeled bytes/reader inputs | Main-file search paths, environment conventions, and CLI wiring |
|
||||
| YAML policy, explicit import traversal, composition, and profiles | Configuration types and field meanings |
|
||||
| Local file and labeled bytes/reader inputs | Main-file search paths, bootstrap timing, and CLI parsing/wiring |
|
||||
| YAML policy, explicit import traversal, composition, profiles, and explicit CLI overrides | Configuration types and field meanings |
|
||||
| Presence, source locations, resolution records, and safe diagnostics | Static defaults, derived defaults, and normalization rules |
|
||||
| Secret-reference resolution and explicit directory bootstrap | Credential use and process startup coordination |
|
||||
| Strict typed decoding and lifecycle coordination | Semantic validation and schema-version policy |
|
||||
| Source reporting and redacted effective output | Remote acquisition, migrations, runtime initialization, and resource checks |
|
||||
| Source reporting, safe history/explanation, and redacted effective output | Remote acquisition, migrations, runtime initialization, and resource checks |
|
||||
|
||||
Applications may fetch remote data and pass labeled bytes to ConfigKit. This
|
||||
provides useful provenance without making ConfigKit responsible for network
|
||||
@@ -99,7 +100,7 @@ contracts govern the result; reuse does not require preserving incidental
|
||||
application structure or incompatible behavior.
|
||||
|
||||
The references below were inspected on 2026-09-06 at Narratio commit
|
||||
`a7ec1955874fda61f3e3f9e3cc33acd345530952`. Paths refer to the sibling checkout
|
||||
`b68435128be1e24b94fc7ec259d6e9093669fd99`. Paths refer to the sibling checkout
|
||||
`../narratio` relative to the ConfigKit repository root; the links require that
|
||||
checkout layout. They are extraction references, not a runtime or build dependency.
|
||||
Recheck them against the source revision actually used for extraction.
|
||||
@@ -107,12 +108,22 @@ Recheck them against the source revision actually used for extraction.
|
||||
| Foundation | Source references | Intended treatment |
|
||||
| --- | --- | --- |
|
||||
| File and labeled reader/bytes loading; strict field checking | [load.go](../../../narratio/internal/config/load.go): `decodeStrictYAML`, `decodeStrictYAMLFromReader`, `LoadSessionBytesWithOptions` | Extract reusable input handling and generalize decoding behind ConfigKit's private parser boundary |
|
||||
| Node parsing, strict composition, overlays, and canonical rendering | [composition.go](../../../narratio/internal/config/composition.go): `parseCompositionDocument`, `buildCompositionNode`, `mergeAdditiveCompositions`, `mergeOverlayComposition`, `canonicalYAML` | Extract algorithms and tests; replace upstream node kinds, path spelling, scalar semantics, and incompatible merge rules |
|
||||
| Explicit imports and profile selection | [pipeline_composition.go](../../../narratio/internal/config/pipeline_composition.go): `loadPipelineCompositionSources`, `loadPipelineImports`, `loadPipelineProfileOverlays`, `selectPipelineProfile` | Reuse traversal/selection mechanics with ConfigKit's recursive graph, containment, directive syntax, and fixed selected-profile convention |
|
||||
| Effective value/source inspection | [effective_pipeline.go](../../../narratio/internal/config/effective_pipeline.go): `EffectivePipelineValues`, `EffectivePipelineSources`, `MarshalEffectivePipeline`; [config_commands.go](../../../narratio/internal/app/config_commands.go) | Reuse deterministic projection, ownership, and side-effect-free inspection cases; retain application commands and generated-domain attribution in Narratio |
|
||||
| Explicit field presence | [normalize.go](../../../narratio/internal/config/normalize.go): `NormalizeConfig.UnmarshalYAML` | Generalize the requirement into library metadata; replace the application-specific parser hook |
|
||||
| Default application | [load.go](../../../narratio/internal/config/load.go): `applyPipelineDefaults` and related helpers; [defaults.go](../../../narratio/internal/config/defaults.go) | Reuse applicable mechanics and behavioral cases while keeping actual default values and domain rules application-owned |
|
||||
| Resolution and source attribution | [load.go](../../../narratio/internal/config/load.go): `Resolve`, `mergeCampaignSession`, `selectStableInput`; [config.go](../../../narratio/internal/config/config.go): `ResolvedInputFile`, `SessionSource` | Adapt source tracking into generic provenance; retain campaign/session fallback and domain types in Narratio |
|
||||
| Application-supplied remote bytes | [config_loader.go](../../../narratio/internal/app/config_loader.go) | Use as evidence for labeled input support; keep fetching, storage clients, and remote metadata acquisition outside ConfigKit |
|
||||
| Existing behavioral coverage | [normalize_test.go](../../../narratio/internal/config/normalize_test.go), [load_validate_test.go](../../../narratio/internal/config/load_validate_test.go), [campaign_config_test.go](../../../narratio/internal/config/campaign_config_test.go) | Extract relevant cases and minimal fixtures; preserve domain-specific assertions in consumer tests |
|
||||
|
||||
Additional extraction coverage includes [composition tests](../../../narratio/internal/config/composition_test.go),
|
||||
[import tests](../../../narratio/internal/config/pipeline_composition_test.go),
|
||||
[profile tests](../../../narratio/internal/config/pipeline_profiles_test.go),
|
||||
[strict-loader tests](../../../narratio/internal/config/loader_strict_test.go), and
|
||||
[inspection tests](../../../narratio/internal/app/config_commands_test.go).
|
||||
These are reusable behavior evidence, not an instruction to preserve every assertion.
|
||||
|
||||
### Compatibility and adaptation
|
||||
|
||||
Extraction must distinguish preserved behavior, intentionally changed behavior,
|
||||
@@ -126,13 +137,30 @@ Known adaptation points include:
|
||||
- Some Narratio defaulting helpers use an empty string to trigger a default.
|
||||
Adapt these to ConfigKit's [presence and defaults rules](#static-defaults-and-null)
|
||||
rather than losing explicitly configured empty values.
|
||||
- `decodeStrictYAMLFromReader` currently accepts a successfully decoded second
|
||||
document. ConfigKit must enforce its [single-document policy](#yaml-and-scalar-policy).
|
||||
- `decodeStrictYAMLFromReader` now rejects trailing documents. Preserve and
|
||||
generalize its strict-loader regressions under ConfigKit's single-document policy.
|
||||
- The custom normalization decoder and pointer-based presence conventions need
|
||||
to give way to generic metadata while preserving genuine domain nullability.
|
||||
- Existing source labels and campaign/session attribution provide useful
|
||||
foundations but do not supply full per-path locations, overlay history, or
|
||||
recorded post-decode changes.
|
||||
- Existing composition nodes and effective-field ownership are substantial
|
||||
foundations, but do not retain the full line/column and historical-value records
|
||||
ConfigKit requires. Keep application-generated family/campaign attribution as
|
||||
domain resolution with recorded dependencies.
|
||||
- Narratio rejects overlapping empty mappings, null overlays, and cross-kind
|
||||
overlays. ConfigKit's operation table deliberately permits those cases. Adapt
|
||||
the implementation and expectations together.
|
||||
- Narratio declares import paths and profiles in a root `composition` envelope,
|
||||
supports a declared default profile, and loads every declared overlay. ConfigKit
|
||||
uses `_configkit.imports`, recursive imports/globs, and only the selected fixed
|
||||
profile path; `default` means no overlay. Do not preserve an alternate convention.
|
||||
- Narratio uses `os.SameFile` to reject filesystem aliases. ConfigKit uses canonical
|
||||
path identity, treats hard links as distinct paths, and enforces root containment.
|
||||
- Narratio's owned tree still embeds upstream YAML kinds/tags and dotted/indexed
|
||||
paths. Its final binding and effective inspection reserialize/reparse YAML.
|
||||
Reuse algorithms without importing those dependencies into ConfigKit's document
|
||||
model or replacing direct descriptor-driven decoding and projection.
|
||||
- Narratio's configuration diffs and digests are useful consumer evidence, but
|
||||
remain outside ConfigKit's initial scope. Their implementation does not supply
|
||||
full historical-value explanation or authorize a new public API.
|
||||
|
||||
Each intentional behavior change needs focused tests and an explicit migration
|
||||
account for the affected consumer. Do not copy an old expectation merely because
|
||||
@@ -143,17 +171,20 @@ public API remains practical for single-document applications.
|
||||
|
||||
### Capabilities requiring broader implementation
|
||||
|
||||
The inspected loader coordinates specific pipeline, campaign, and session
|
||||
sources and application fallback rules. It is not the generic import graph and
|
||||
strict recursive composer described by this target. Extract applicable pieces,
|
||||
but expect new or substantially revised code for generic composition, profile
|
||||
overlays, schema-aware defaults/decoding, comprehensive provenance, recorded
|
||||
resolution, and safe structured diagnostics and rendering.
|
||||
The inspected Narratio loader supplies explicit flat imports, strict composition,
|
||||
profile overlays, source ownership, and effective inspection in addition to its
|
||||
pipeline/campaign/session domain coordination. Prefer extracting those foundations
|
||||
rather than rebuilding equivalent mechanisms.
|
||||
|
||||
Automatic `conf.d` discovery remains deferred. The existing application source
|
||||
relationships do not authorize implicit discovery or define ConfigKit's import
|
||||
syntax. The [implementation plan](implementation.md) owns concrete extraction
|
||||
tasks and their ordering.
|
||||
ConfigKit still requires new or substantially revised work for recursive import
|
||||
graphs and globs, bounded canonical containment, its conservative YAML policy,
|
||||
schema-aware defaults/direct decoding, comprehensive location and value history,
|
||||
recorded domain resolution, generic secret classification, and safe structured
|
||||
reporting. Existing provenance is a starting point, not full explainability.
|
||||
|
||||
Automatic `conf.d` discovery remains deferred. Existing application relationships
|
||||
and declared profile paths do not redefine ConfigKit's import/profile conventions.
|
||||
The [implementation plan](implementation.md) owns extraction tasks and ordering.
|
||||
|
||||
## Consumer fit and intentional migrations
|
||||
|
||||
@@ -166,9 +197,16 @@ on 2026-09-06 and must be rechecked at migration time.
|
||||
| --- | --- | --- |
|
||||
| Narratio: [configuration loader](../../../narratio/internal/config/load.go) | Shared loading, presence, defaults coordination, and source attribution | Replace ad hoc presence/empty-value defaults; retain campaign/session domain resolution |
|
||||
| Notarius: [file model](../../../notarius/internal/core/config/file_config.go), [environment handling](../../../notarius/internal/core/config/env.go) | Strict nested models, named pipelines, presence-aware values, and diagnostics | Use canonical object bindings/references and string schema versions; adapt dynamic integer consumers to `int64`; migrate configuration-field environment overrides to files/profiles; keep catalog resolution and runtime invocation choices application-owned |
|
||||
| WeatherReporter: [loader](../../../weatherreporter/internal/config/load.go), [module options](../../../weatherreporter/internal/config/reports.go) | Defaults-only loading, durations, report maps, list presence, and typed module options | Replace string shorthand with object entries and private presence flags with metadata; migrate configuration-field CLI overrides to files/profiles; keep secrets-directory loading and process-environment mutation outside ConfigKit |
|
||||
| WeatherReporter: [loader](../../../weatherreporter/internal/config/load.go), [module options](../../../weatherreporter/internal/config/reports.go) | Defaults-only loading, durations, report maps, list presence, and typed module options | Replace string shorthand with object entries and private presence flags with metadata; translate explicitly set configuration flags into ConfigKit overrides; reuse secrets-directory loading through explicit ConfigKit bootstrap |
|
||||
| Seriatim: [configuration surfaces](../../../seriatim/docs/config.md), [speaker rules](../../../seriatim/internal/speaker/map.go), [autocorrect rules](../../../seriatim/internal/autocorrect/autocorrect.go) | Simple typed loading of ordered YAML rule files, normalization, and cross-rule diagnostics | Adopt strict YAML rejection deliberately; retain matching/replacement algorithms, resource checks, and CLI/environment runtime settings; no central application YAML file is required |
|
||||
|
||||
WeatherReporter's updated [configuration tests](../../../weatherreporter/internal/config/config_test.go)
|
||||
and [external constructed-config tests](../../../weatherreporter/internal/config/overrides_external_test.go)
|
||||
cover explicit empty lists, consistent loaded/constructed behavior, and normalization
|
||||
without mutating source options. Preserve these domain outcomes through ConfigKit's
|
||||
presence, detached ownership, and application validation. ConfigKit permits an empty
|
||||
list structurally; a particular consumer may reject it semantically.
|
||||
|
||||
The initial model standardizes syntax and resolution while preserving application
|
||||
field meanings. Module shorthand and integer schema versions are syntax migration
|
||||
issues; module-specific option schemas are genuine domain variation addressed
|
||||
@@ -177,24 +215,184 @@ not substitutes for application-owned LLM or pipeline profile concepts.
|
||||
|
||||
## External values and invocation state
|
||||
|
||||
Environment and CLI configuration-field overlays remain outside the initial
|
||||
release. Consumers adopting the complete ConfigKit lifecycle must move such
|
||||
settings into base files or supported profile overlays, or defer migration of
|
||||
that configuration surface. Do not silently discard supported consumer flags;
|
||||
consumer migration must document their replacement or retirement.
|
||||
Configuration-field CLI flags use the explicit [override API](#cli-overrides).
|
||||
Applications own parsing flags and selecting sources/profiles. Separate invocation
|
||||
state, such as an input artifact or execution lane, remains application-owned.
|
||||
General environment-variable configuration overlays are an explicit non-goal,
|
||||
not deferred work. Environment access is reserved for named secret references.
|
||||
|
||||
Application CLI arguments may select a configuration source/profile or describe
|
||||
separate invocation state, such as an input artifact, output destination, or
|
||||
selected execution lane. Environment variables may supply runtime credentials
|
||||
outside the configuration snapshot. Such state remains application-owned and
|
||||
must not be reported as part of ConfigKit's effective configuration.
|
||||
Do not inject hidden overrides through captured environment/CLI values in default
|
||||
factories, resolver hooks, or validators. Domain resolution follows the declared
|
||||
input layers. Mutating `Result.Config` cannot update a validated snapshot.
|
||||
|
||||
Do not inject configuration overrides through captured environment/CLI values in
|
||||
default factories, resolver hooks, or validators. Resolution is for domain
|
||||
derivation and normalization, not an undocumented precedence layer. Mutating
|
||||
`Result.Config` after loading cannot create a newly validated ConfigKit snapshot
|
||||
or update its provenance. A future override mechanism would need an explicit
|
||||
precedence, presence, validation, and origin contract before adoption.
|
||||
## Secret references and explicit bootstrap
|
||||
|
||||
Secret-bearing configuration fields use ConfigKit's opaque `Secret` type. Their
|
||||
YAML value is an environment-variable name, for example `api_key: OPENROUTER_API_KEY`.
|
||||
Names are public configuration; resolved credentials are not configuration-tree
|
||||
values. Literal-secret input, interpolation, fallback literal values, dotenv
|
||||
parsing, and implicit directory discovery are unsupported. Names must match
|
||||
`[A-Za-z_][A-Za-z0-9_]*`. A string that happens to be a valid name is always a
|
||||
reference; ConfigKit cannot detect credentials mistakenly written in other fields.
|
||||
|
||||
```go
|
||||
func SecretEnv(name string) Secret
|
||||
func (s Secret) EnvName() string
|
||||
func (s Secret) Value() string
|
||||
func (s Secret) IsResolved() bool
|
||||
```
|
||||
|
||||
`SecretEnv` creates an unresolved reference; invalid names fail the containing
|
||||
load. The zero value is an absent reference, never a literal secret. A non-pointer
|
||||
`Secret` is required: absent, null, empty/invalid name, unset environment variable,
|
||||
or an empty environment value fails with a safe `secret.*` diagnostic. Use
|
||||
`*Secret` for optional credentials: nil is allowed, but every non-nil reference
|
||||
must resolve to a nonempty value. Applications validate conditional requirements.
|
||||
Only typed fields/maps/slices establish secret-reference semantics; dynamic `any`
|
||||
strings are ordinary values and must migrate to typed secret slots when needed.
|
||||
|
||||
Recognize `Secret` before ordinary struct/text binding. Projection, comparison,
|
||||
cloning for defaults, and resolution auditing operate on the name only. Resolve
|
||||
the surviving references after recorded application resolution and before semantic
|
||||
validation, using one `os.LookupEnv` result per unique name per load. A copied
|
||||
previous result used as defaults is re-resolved; it cannot carry stale credentials.
|
||||
Validation can use `Value()`; resolution works with names. In addition to ordinary
|
||||
projection auditing, reject validator changes to a secret's private resolved state
|
||||
by comparing it with this load's lookup result. A same-name unresolved replacement
|
||||
must not evade the read-only validator contract. `Value()` returns empty
|
||||
on unresolved references; `IsResolved()` distinguishes that state. Private credential
|
||||
storage is excluded from every tree, historical snapshot, diagnostic, default
|
||||
format, JSON/text serialization, and inspection path. Provide safe formatting for
|
||||
all fmt verbs and name-only JSON marshaling; expose no credential decoder/codec.
|
||||
Only the deliberately named `Value()` accessor returns a credential. Applications
|
||||
own its use and must not embed it in hook messages. Secret references themselves
|
||||
need no `configkit:"secret"` tag: the existing tag/predicate classifies additional
|
||||
sensitive or derived values and can intentionally hide a reference name.
|
||||
|
||||
The separate, opt-in bootstrap operation below loads a directory into the process
|
||||
environment. `Load` never calls it or changes process-global state. Applications
|
||||
choose a directory and call bootstrap once during startup before concurrent work.
|
||||
Bootstrap paths are native application-selected paths, independent of import roots.
|
||||
An empty or missing directory path is an error; an existing empty directory succeeds.
|
||||
|
||||
```go
|
||||
func LoadSecretsDir(directory string) (*SecretsReport, error)
|
||||
func (r *SecretsReport) Entries() []SecretFileResult
|
||||
func (r *SecretsReport) WriteTo(output io.Writer) (int64, error)
|
||||
type SecretsError struct { /* private safe diagnostic storage */ }
|
||||
func (e *SecretsError) Error() string
|
||||
func (e *SecretsError) Diagnostics() []Diagnostic
|
||||
type SecretFileResult struct {
|
||||
Name, Path string
|
||||
Status SecretFileStatus
|
||||
ExistingEmpty bool
|
||||
}
|
||||
type SecretFileStatus string
|
||||
// Constants use values "loaded", "skipped_existing", "not_applied", "failed":
|
||||
// SecretFileLoaded, SecretFileSkippedExisting, SecretFileNotApplied, SecretFileFailed.
|
||||
```
|
||||
|
||||
Extract applicable code and tests directly from WeatherReporter's
|
||||
[secrets loader](../../../weatherreporter/internal/config/secrets.go), inspected
|
||||
at revision `5d306a202471e10fe98923a1df27d0b0fce5c666`. Reuse its filename checks,
|
||||
file-type rejection, single terminal LF/CRLF removal, and separation of `loadSecrets`
|
||||
from `applySecrets`. WeatherReporter now preloads file contents, validates its
|
||||
configuration, then applies environment writes with attempted rollback. ConfigKit
|
||||
preserves existing variables, reports partial apply failure without rollback, and
|
||||
keeps bootstrap separate from configuration loading. Record extraction symbols,
|
||||
revision and adaptations during implementation.
|
||||
|
||||
Narratio's [secrets loader](../../../narratio/internal/app/secrets_env.go) at the
|
||||
revision recorded above is a complementary foundation: `loadSecretsFromConfigWithDependencies`
|
||||
preserves existing variables including empty ones, and `readAPIKeyFile` uses
|
||||
[bounded reads](../../../narratio/internal/fileops/read_bounded.go). Its
|
||||
[tests](../../../narratio/internal/app/secrets_env_test.go) provide reusable failure
|
||||
and environment-preservation cases. Adapt its immediate writes and aggregate/log
|
||||
reporting to ConfigKit's preflight and per-file structured report. Narratio skips
|
||||
invalid names, removes all trailing CR/LF characters, rejects a symlink at the
|
||||
selected directory path, and enforces restrictive permission bits. Those policies
|
||||
are not inherited: use ConfigKit's name/type/newline/directory rules and fixed
|
||||
bounds. ConfigKit does not add a permission-mode gate in the initial API;
|
||||
applications own deployment permissions. Extract bounded-read mechanisms without
|
||||
copying Narratio's application dependency framework or logger.
|
||||
|
||||
Process only immediate entries, sorted by filename. Reject invalid names,
|
||||
symlinks, directories, and nonregular files, including entries whose names are
|
||||
already set. Allow a symlink in the explicitly selected directory path by resolving
|
||||
it to a canonical directory first. Validate all entry metadata before mutation.
|
||||
`os.LookupEnv` presence wins, including an empty existing variable: skip reading
|
||||
that file's contents and report `SkippedExisting` with `ExistingEmpty` set
|
||||
accurately. Read eligible files with bounds, remove exactly one terminal CRLF or
|
||||
LF, preserve all other whitespace, and reject NUL bytes. Empty file values are
|
||||
allowed at bootstrap, but fail if subsequently used by a required reference.
|
||||
Preflight all eligible reads before any `os.Setenv`. No writes occur on preflight
|
||||
failure. At apply time recheck presence before each write and skip newly set names.
|
||||
Concurrent mutation is unsupported; this is not an atomic compare-and-set facility.
|
||||
|
||||
Return a non-nil report even on failure. Successful writes are `Loaded`, blocked
|
||||
files are `SkippedExisting`, the failed entry is `Failed`, and eligible but unwritten
|
||||
entries are `NotApplied`. Directory-level failures have a safe error without a
|
||||
fabricated entry. Stop on the first deterministic failure; reports include all
|
||||
entries enumerated so far, with unprocessed entries `NotApplied`. An apply failure
|
||||
preserves earlier successful writes and reports partial progress; do not attempt
|
||||
rollback that could overwrite unrelated process changes. Errors are `*SecretsError`
|
||||
with safe `secrets.*` diagnostic codes and no raw unwrap chain. Errors and report
|
||||
rendering contain names, paths, statuses, and safe classifications, never contents.
|
||||
|
||||
Report entries and diagnostic slices are defensive copies. Bootstrap diagnostics
|
||||
include safely quoted canonical paths in messages with no invented configuration
|
||||
source ID or source location.
|
||||
Report formatting escapes control characters and propagates short writes/errors.
|
||||
|
||||
The report is the authoritative bootstrap provenance: a blocked file explicitly
|
||||
identifies the existing process variable as the winner, even when empty. Keep it
|
||||
alongside the load result and render it before configuration explanation in an
|
||||
application command. Secret-resolution metadata records the variable name and
|
||||
`resolved`, `unset`, or `empty` lookup outcome (failures in diagnostics). It does
|
||||
not claim which external actor populated the process environment. Reports are
|
||||
separate snapshots, not accepted as evidence of the environment's later state;
|
||||
there is no implicit global report or stale filename attribution in `Load`.
|
||||
|
||||
## CLI overrides
|
||||
|
||||
Precedence is **application defaults < composed base < selected profile < explicit
|
||||
CLI overrides**. Domain normalization follows those inputs and is recorded.
|
||||
|
||||
```go
|
||||
type Override struct {
|
||||
Path Path
|
||||
Value any
|
||||
Flag string
|
||||
}
|
||||
// Options[T].Overrides is []Override.
|
||||
```
|
||||
|
||||
Applications provide records only for flags explicitly set by the caller; parser
|
||||
defaults are not overrides. `Flag` is a required, non-secret display name, such as
|
||||
`--workers`, never the raw argument or value. ConfigKit has no CLI parser dependency.
|
||||
Values are detached and converted to ConfigKit-owned nodes before composition:
|
||||
accept built-in string, bool, integer and unsigned integer widths, finite floats,
|
||||
`nil`, `[]any`, and `map[string]any` recursively. Strings remain strings; do not
|
||||
parse YAML fragments or coerce textual booleans/numbers. Durations and secret
|
||||
references use strings. Reject other types, cycles, nonfinite numbers, and limits
|
||||
violations with safe `override.*` diagnostics.
|
||||
|
||||
Validate every path/record before applying any. Reject root paths, sequence-index
|
||||
traversal, duplicate paths, and ancestor/descendant overlaps within the override
|
||||
list. Numeric tokens are mapping keys only here. Mapping-key paths may create
|
||||
missing mapping ancestors, but cannot traverse an existing scalar/null/sequence.
|
||||
The target path overlays with the same rules as profiles: mappings recursively
|
||||
merge; sequences and all other replacements are atomic. Use a whole sequence
|
||||
value to replace a list. Preserve supplied list order; mapping entries use canonical
|
||||
key order. Unknown struct fields and effective type mismatches fail normal strict
|
||||
decoding. Never allow `_configkit` directives or writes to the configured schema
|
||||
version path, including ancestor replacement. Overrides also work with `NoSource()`.
|
||||
|
||||
Each record contributes a synthetic CLI source after file/profile sources, in
|
||||
record order, labeled with `Flag`, with no file coordinates. Explicit false, zero,
|
||||
empty string, null, and empty list remain present. Snapshot caller-owned values
|
||||
before applying them. History includes replaced inputs even for equal-value writes.
|
||||
Neither callers nor hooks may mutate supplied data concurrently with loading.
|
||||
|
||||
## Resolution lifecycle
|
||||
|
||||
@@ -204,13 +402,15 @@ application selects source and loading options
|
||||
→ discover explicit imports and strictly compose the base
|
||||
→ application schema-version check, when configured
|
||||
→ apply the selected profile
|
||||
→ apply explicit CLI overrides to the node tree
|
||||
→ apply static defaults and strictly decode
|
||||
→ apply recorded application resolution and normalization
|
||||
→ resolve surviving secret references from the environment
|
||||
→ collect application semantic validation diagnostics
|
||||
→ return independently owned configuration and metadata
|
||||
```
|
||||
|
||||
Value precedence is defaults < composed base < selected profile. Resolution is
|
||||
Value precedence is defaults < composed base < selected profile < CLI overrides. Resolution is
|
||||
an application transformation after those layers, not another user override
|
||||
source. Future migration support belongs after base composition/version
|
||||
inspection and before profile application and typed decoding; migrations are
|
||||
@@ -258,6 +458,10 @@ for the root. For example, `/transcription/concurrency` names a nested field;
|
||||
using the containing node's kind, distinguishing mapping keys from list indices.
|
||||
Use YAML field names, not Go field names.
|
||||
|
||||
Semantic paths are not filesystem paths or URLs. Empty keys, literal `.`/`..`
|
||||
keys, and percent sequences retain their meaning; do not clean path components
|
||||
or percent-decode them. Only JSON Pointer escape processing applies.
|
||||
|
||||
Distinguish absent, explicitly null, explicitly zero/false/empty, and other
|
||||
explicit values. An explicit value equal to its default remains explicit.
|
||||
Presence queries describe the surviving input contribution after overlays;
|
||||
@@ -322,7 +526,9 @@ disabled, the presence of `_configkit` is an error, not an ignored directive.
|
||||
paths and network URLs are rejected.
|
||||
- Expand declarations in list order, sorting each glob's matches
|
||||
lexicographically by slash-separated relative path, with no locale sorting.
|
||||
Use segment glob syntax (`*`, `?`, character classes), with no recursive `**`.
|
||||
Use the slash-based [Go `path.Match` grammar](https://pkg.go.dev/path#Match)
|
||||
(`*`, `?`, character classes, and backslash escapes), with recursive `**`
|
||||
syntax forbidden. Backslashes escape pattern characters, not directory separators.
|
||||
- Traverse depth-first: process a file's own contribution, then its declarations
|
||||
in order. Ordering stabilizes source lists and diagnostics; it never permits
|
||||
duplicate definitions to override one another.
|
||||
@@ -333,6 +539,8 @@ disabled, the presence of `_configkit` is an error, not an ignored directive.
|
||||
explicitly supply another root. Resolve symlinks and cleaned absolute paths
|
||||
for identity and containment. Imports and profiles must stay inside that
|
||||
canonical root. Retain declared paths and import-site locations for messages.
|
||||
Relative and absolute symlink targets are permitted when their canonical target
|
||||
is contained; the spelling of a symlink target alone does not determine containment.
|
||||
- An active-stack repeat is a cycle; report the complete cycle and import sites.
|
||||
A previously visited canonical file outside the active stack is a repeated
|
||||
import error, including diamond graphs and overlapping glob matches. Report
|
||||
@@ -341,6 +549,9 @@ disabled, the presence of `_configkit` is an error, not an ignored directive.
|
||||
- Read each accepted source once per load. Do not promise an atomic snapshot of
|
||||
a filesystem being modified concurrently. Bound input bytes, node nesting,
|
||||
import depth, and source count with documented limits and structured errors.
|
||||
Excess input is an error, never an implicitly truncated document. Glob traversal
|
||||
counts examined directory entries, including nonmatches, against its work budget;
|
||||
only accepted matches contribute to the sorted source order.
|
||||
|
||||
Bytes/reader sources require a diagnostic label. Imports and profiles for these
|
||||
sources require an explicit filesystem base directory and root; never infer a
|
||||
@@ -428,10 +639,10 @@ Strictness must be enforced by ConfigKit across the effective tree:
|
||||
let them bypass key, null, or type policy. Custom structural decoders are
|
||||
deferred; unsupported type shapes fail with a clear schema diagnostic.
|
||||
|
||||
Keep parser nodes and errors internal. Exact package choice and Go API spelling
|
||||
are implementation decisions, to be exercised against consumer types before
|
||||
stabilization. The field-binding model must also support defaults, provenance,
|
||||
secret classification, and rendering consistently.
|
||||
Keep parser nodes and errors internal under the
|
||||
[upstream isolation contract](#yaml-dependency-and-public-api-isolation).
|
||||
The field-binding model must also support defaults, provenance, secret
|
||||
classification, and rendering consistently.
|
||||
|
||||
## Recorded resolution and validation
|
||||
|
||||
@@ -509,8 +720,8 @@ for inspection and is not promised to be reloadable. ConfigKit renderers must
|
||||
not expose raw secrets; returning typed configuration to the application is an
|
||||
intentional trusted boundary. ConfigKit does not log configuration on its own.
|
||||
|
||||
Retain sufficient history for later full `explain` tooling, but defer historical
|
||||
value rendering, config diffs, and fingerprints. A future fingerprint design
|
||||
Provide full safe historical-value explanation in the initial release, as defined
|
||||
[below](#historical-value-explanation). Defer config diffs and fingerprints. A future fingerprint design
|
||||
must specify canonicalization, secret handling, and disclosure risks before
|
||||
claiming identity or reproducibility guarantees.
|
||||
|
||||
@@ -518,7 +729,7 @@ claiming identity or reproducibility guarantees.
|
||||
|
||||
Applications may declare a schema-version path (for example `/schema_version`)
|
||||
and a pre-decode checker. The main document owns this value; fragments omit it,
|
||||
and profiles cannot supply or change it. Check fragment placement before
|
||||
and profiles/CLI overrides cannot supply or change it. Check fragment placement before
|
||||
composition so a version supplied only by an import is also rejected. ConfigKit enforces that placement when
|
||||
the application enables the mechanism. The application decides whether a version
|
||||
is required and which values it accepts. ConfigKit does not require one universal
|
||||
@@ -553,6 +764,7 @@ type Options[T any] struct {
|
||||
Imports bool
|
||||
Profile string
|
||||
Filesystem *FilesystemContext
|
||||
Overrides []Override
|
||||
}
|
||||
|
||||
type FilesystemContext struct {
|
||||
@@ -593,7 +805,8 @@ both absolute `BaseDir` and `Root` when imports or profiles are used. A base mus
|
||||
be inside its root. These options do not change application field path meanings.
|
||||
|
||||
`NoSource()` explicitly selects defaults-only loading. It contributes no document
|
||||
or source entry and marks no fields explicit; it still runs schema checking with
|
||||
or file source entry and marks no fields explicit by itself; CLI overrides still
|
||||
contribute synthetic sources and explicit values; it still runs schema checking with
|
||||
an absent version, defaults, resolution, validation, and snapshot/inspection.
|
||||
An application requiring an input version can therefore reject this mode.
|
||||
Imports, an active profile, or filesystem context with `NoSource()` are options
|
||||
@@ -674,8 +887,8 @@ errors. Path spelling uses YAML names throughout.
|
||||
`FieldInfo` reports explicitness, null state, and effective `Origin`. An origin
|
||||
identifies defaults, input, or application resolution and associated source
|
||||
locations; it may be composite. `Change` reports layer/operation, prior origins,
|
||||
safe reasons, dependency paths, and related locations, without raw historical
|
||||
values. These are library-owned metadata types, never parser nodes. Queries
|
||||
safe reasons, dependency paths, and related locations. Safe historical values
|
||||
are available through `Explain`, rather than unrestricted raw-value references. These are library-owned metadata types, never parser nodes. Queries
|
||||
return copies or immutable views. Unknown paths yield no lookup/history and
|
||||
false explicitness. Callback mutation/validation paths are checked more strictly:
|
||||
misspelled struct fields are errors, while valid missing map entries are allowed.
|
||||
@@ -751,7 +964,7 @@ type PipelineConfig struct {
|
||||
PollInterval time.Duration `yaml:"poll_interval"`
|
||||
Timeout time.Duration `yaml:"timeout"`
|
||||
Artifacts map[string]Artifact `yaml:"artifacts"`
|
||||
APIKey string `yaml:"api_key" configkit:"secret"`
|
||||
APIKey configkit.Secret `yaml:"api_key"`
|
||||
}
|
||||
|
||||
func resolvePipeline(cfg *PipelineConfig, r *configkit.Resolution) error {
|
||||
@@ -970,7 +1183,74 @@ func InspectPipeline(path, profile string, output io.Writer) error {
|
||||
|
||||
Applications can use `Metadata().Lookup`, `Metadata().History`, and `Sources()`
|
||||
for structured inspection without implementing value rendering or importing a
|
||||
parser. A complete historical-value `explain` command remains deferred.
|
||||
parser. The explanation API below supplies safe values and formatting for application-owned
|
||||
`config explain` commands.
|
||||
|
||||
### Historical-value explanation
|
||||
|
||||
```go
|
||||
func (r *Result[T]) Explain(path Path) (Explanation, bool)
|
||||
func (r *Result[T]) WriteExplanation(output io.Writer, path Path) error
|
||||
|
||||
type Explanation struct {
|
||||
Path Path
|
||||
Effective ValueSnapshot
|
||||
Origin Origin
|
||||
History []ValueChange
|
||||
SecretReference *SecretReferenceInfo
|
||||
}
|
||||
type ValueSnapshot struct {
|
||||
Present bool
|
||||
Redacted bool
|
||||
YAML string
|
||||
}
|
||||
type ValueChange struct {
|
||||
Before, After ValueSnapshot
|
||||
Origin Origin
|
||||
Reasons []string
|
||||
Dependencies []Path
|
||||
}
|
||||
type SecretReferenceInfo struct { Name string } // successful resolved lookup
|
||||
```
|
||||
|
||||
Snapshots contain detached deterministic YAML text, never `any`, parser objects,
|
||||
or mutable internal trees. `Present=false` is absence/deletion and has empty YAML;
|
||||
null is present with `null` YAML. Redacted snapshots use the quoted marker
|
||||
`[REDACTED]`; `Redacted` also marks containers containing redacted descendants.
|
||||
Apply the same historical sensitivity/dependency closure as effective rendering
|
||||
before exposing any snapshots. A `Secret` projects only its environment name;
|
||||
its resolved value is never captured, even privately as history. For classified
|
||||
reference paths omit `SecretReference` as well as masking the projected name.
|
||||
|
||||
Retain private persistent node snapshots or operation deltas with structural
|
||||
sharing; do not copy the entire tree for every changed leaf. Explain a known final
|
||||
or historical path, including removed paths; unknown paths return false, and the
|
||||
writer returns a safe unknown-path error without output. Malformed query paths
|
||||
also fail safely. Render the canonical JSON-pointer path, effective value/source,
|
||||
and chronological history: initial application-default (or Go-zero baseline),
|
||||
base contributions in traversal order, profile, CLI, and recorded domain changes.
|
||||
Include before/after values for changed or explicitly redefined paths, equal-value
|
||||
writes, removals, reasons/dependencies, and composite origins. Inputs overridden
|
||||
before decoding retain policy-validated node values even when they would not fit
|
||||
the final Go type. Defaults/final/domain snapshots use canonical typed projection;
|
||||
label those representations rather than claiming every historical input was typed.
|
||||
|
||||
Container queries reconstruct the value at each relevant operation; descendant
|
||||
queries include ancestor replacement/removal and whole-sequence replacements,
|
||||
without implying element-wise sequence merging. Omitted fields/default retention
|
||||
must agree with the presence/default contract; do not fabricate a write for every
|
||||
import that leaves a path alone. Resolution history records the audited net change
|
||||
per path, not every intermediate Go assignment. Lookup of a secret reference
|
||||
annotates its effective name as resolved from the process environment; bootstrap
|
||||
file outcomes come from the separate report and are never guessed.
|
||||
|
||||
`WriteExplanation` is a deterministic human renderer over this safe structured
|
||||
result. Escape control characters in paths, labels, names, and reasons, propagate
|
||||
writer errors/short writes, and permit partial output only on writer failure.
|
||||
No unredacted switch or bundled command is provided. For example an application
|
||||
can call it for `/transcription/concurrency` to show default `4`, an explicitly
|
||||
imported file's `8`, then a testing profile's `1`, with file line numbers and the
|
||||
profile as effective source. This does not require automatic `conf.d` discovery.
|
||||
|
||||
### Module-specific options
|
||||
|
||||
@@ -1102,20 +1382,76 @@ These are responsibilities, not a mandate for public interfaces or a file per
|
||||
concept. Start with one public package and a private YAML adapter; add package
|
||||
boundaries only when dependencies and ownership warrant them.
|
||||
|
||||
Use standard-library filesystem, I/O, scalar conversion, reflection, ordering,
|
||||
and synchronization primitives. ConfigKit owns the policy that coordinates them:
|
||||
field binding, presence/default behavior, strict composition, overlays, provenance,
|
||||
resolution auditing, and secret propagation. One descriptor graph and document
|
||||
model support those operations; do not add competing binding or copying models.
|
||||
Convenience functions must not broaden accepted scalar syntax, silently truncate
|
||||
input, lose nested ownership, or substitute Go field promotion for YAML inlining.
|
||||
|
||||
The [implementation plan](implementation.md#standard-library-mechanisms-and-custom-policy)
|
||||
owns the selected primitives and their integration checks. Test comparison uses
|
||||
`go-cmp` only in tests, never as the runtime resolution comparator. Additional
|
||||
dependencies remain case-by-case decisions based on code saved after accounting
|
||||
for adaptation, validation, and recurring maintenance.
|
||||
|
||||
### YAML dependency and public API isolation
|
||||
|
||||
Use the YAML organization's `go.yaml.in/yaml/v4` package behind `internal/yaml`.
|
||||
V4 is upstream's active development line; v3 receives security fixes only.
|
||||
This choice aligns a new library with ongoing upstream maintenance while keeping
|
||||
ConfigKit responsible for its own configuration model.
|
||||
See [upstream's version policy](https://github.com/yaml/go-yaml#version-intentions).
|
||||
The [implementation decisions](implementation.md#implementation-decisions) own
|
||||
the exact selected release-candidate pin and upgrade validation requirements.
|
||||
|
||||
The adapter owns all direct upstream imports, including parser plugins, loading,
|
||||
and emission. Its boundary uses ConfigKit-owned document and diagnostic records.
|
||||
No upstream nodes, type aliases, options, plugin interfaces, error types, or
|
||||
generic constraints form part of ConfigKit's public declarations or callbacks.
|
||||
ConfigKit must not return parser-created values hidden inside `any`, metadata,
|
||||
or an error's unwrap chain. Applications use ordinary Go types, ConfigKit types,
|
||||
and the supported standard-library text interfaces; they need no YAML-package
|
||||
imports or upstream version presets to use ConfigKit.
|
||||
|
||||
The existing `yaml` field-tag spelling is a ConfigKit-owned binding contract,
|
||||
not delegation to upstream reflection or marshal/unmarshal hooks. ConfigKit also
|
||||
owns scalar interpretation, duplicate detection, document count, source-coordinate
|
||||
conventions, diagnostic classifications, and effective-output formatting. Upstream
|
||||
options may help enforce these contracts internally but cannot redefine them.
|
||||
|
||||
An upstream breaking API change must be absorbed in the adapter or other private
|
||||
implementation, preserving downstream Go source compatibility and documented
|
||||
configuration behavior. Parser upgrades do not independently authorize changes
|
||||
to ConfigKit's public contracts. If an upstream version cannot preserve those
|
||||
contracts, retain the compatible dependency or adapt the implementation; any
|
||||
intentional ConfigKit contract change follows ConfigKit's own release policy.
|
||||
|
||||
This boundary isolates API coupling, not every dependency risk: the YAML module
|
||||
remains a transitive build dependency, and parser defects or behavior changes
|
||||
still require ConfigKit regression checks. It does not require a public parser
|
||||
interface, configurable backends, or simultaneous v3/v4 implementations.
|
||||
|
||||
## Explicit non-goals
|
||||
|
||||
General environment-variable configuration overlays are not on the roadmap.
|
||||
Secret references and explicit bootstrap are the supported environment convention.
|
||||
Literal-secret configuration and implicit secret-directory discovery are unsupported.
|
||||
|
||||
## Deferred scope
|
||||
|
||||
The initial target excludes:
|
||||
|
||||
- automatic `conf.d` discovery;
|
||||
- remote imports, arbitrary providers, and additional configuration formats;
|
||||
- environment or CLI field overlays;
|
||||
- custom merge strategies, sequence merging, reset operators, and profile inheritance;
|
||||
- parser-specific or custom structural decoding extension APIs;
|
||||
- generic dynamic-subtree conversion and module-registry APIs;
|
||||
- automatic schema migrations;
|
||||
- hot reload, runtime resource replacement, and remote resource validation;
|
||||
- a bundled CLI framework or commands;
|
||||
- full historical value rendering, config diffs, and fingerprints.
|
||||
- config diffs and fingerprints.
|
||||
|
||||
Applications can build their own validate/effective/sources commands on the
|
||||
library. Broadening these boundaries requires concrete consumer evidence and
|
||||
|
||||
Reference in New Issue
Block a user