141 lines
8.2 KiB
Markdown
141 lines
8.2 KiB
Markdown
# Configuration Internals
|
|
|
|
This document describes the maintainer-facing configuration boundary in
|
|
**internal/core/config**. The [Configuration](../config.md) reference owns the
|
|
file format, fields, defaults, precedence contract, and selectable keys. The
|
|
[CLI reference](../cli.md) owns command syntax; this document does not redefine
|
|
either interface.
|
|
|
|
## Boundary
|
|
|
|
The configuration package turns a selected ConfigKit-composed YAML document
|
|
into a validated, independently owned configuration. It then resolves a
|
|
requested pipeline against a module catalog before the framework prepares or
|
|
runs anything.
|
|
|
|
| Boundary | Inputs | Outputs | Does not own |
|
|
| --- | --- | --- | --- |
|
|
| Loading | Selected file path, profile, overrides, and invocation directory | **LoadedConfig** with ConfigKit inspection metadata and a populated **Config** | Choosing the file path or reporting a command result. |
|
|
| Validation | **Config** | Structural configuration errors with pipeline, lane, or binding context | Module availability, capabilities, or construction. |
|
|
| Resolution | Valid **Config**, selected pipeline and lanes, runtime reference changes, LLM override, and module catalog | **EffectiveConfig** with a **ResolvedPipeline** | Materializing reference bytes, preparing modules, execution, or filesystem state. |
|
|
| Inspection | **LoadedConfig** and an optional JSON Pointer | Redacted effective YAML or explanation text | Pipeline selection, module preparation, or provider work. |
|
|
|
|
The CLI discovers a configuration file, invokes this package, and supplies the
|
|
result to the framework. Configuration never reads an input file, constructs a
|
|
module, or creates output, cache, or debug paths. Those responsibilities remain
|
|
at their respective [CLI](cli.md), [pipeline](pipeline.md), and
|
|
[run-state](state.md) boundaries.
|
|
|
|
## Loading And Validation
|
|
|
|
ConfigKit acquires the selected root, imports, optional selected profile, and
|
|
explicit CLI overrides in precedence order. Its document definition supplies
|
|
typed defaults, source-aware path normalization, and structural validation
|
|
before conversion to the independently owned runtime **Config**. Only
|
|
`NOTARIUS_CONFIG` participates in discovery; retired field-level environment
|
|
variables do not participate in loading. Version 5 decoding rejects scalar
|
|
module bindings and scalar external references. ConfigKit retains redacted,
|
|
historical inspection metadata while **LoadedConfig** keeps that implementation
|
|
detail private behind effective and explanation writers.
|
|
|
|
**Config.Validate** checks configuration-only invariants before resolution. It
|
|
rejects incompatible profile sources, invalid state-surface values, unsupported
|
|
concurrency settings, malformed bindings and references, invalid retries, and
|
|
invalid pipeline, step, or lane structure. PromptKit local-backend validation
|
|
accepts only an absolute HTTP or HTTPS endpoint with a host and no user
|
|
information, query, or fragment, and rejects a negative local concurrency
|
|
limit. Its errors retain the closest known pipeline, lane, and binding context.
|
|
It deliberately does not require modules to be registered: that requires a
|
|
catalog and belongs to resolution.
|
|
|
|
The exact user-selectable values and validation rules are defined in
|
|
[Configuration](../config.md). Keep document schema, source-aware normalization,
|
|
runtime conversion, and that reference aligned in the same change.
|
|
|
|
## Effective Resolution
|
|
|
|
**Config.Resolve** derives an absent extraction-worker value in an owned copy
|
|
and validates the configuration. A programmatically supplied worker value is
|
|
always explicit. It normalizes the requested pipeline ID, copies the selected
|
|
profile, and passes the non-empty command-level LLM profile override, requested
|
|
lane selection, and reference changes to the framework resolver.
|
|
|
|
After module and validator selection, the resolver applies the effective
|
|
profile policy to LLM-backed bindings only: command override, binding profile,
|
|
pipeline profile, then the prompt default. Deterministic bindings remain
|
|
profile-free, and no second inheritance decision occurs during execution. The
|
|
public field definitions and precedence are owned by
|
|
[Configuration](../config.md#pipelines).
|
|
|
|
The resolver retains configured `validation_policy` overrides and derives one
|
|
detached concrete terminal policy for the chunk producer and every lane's
|
|
extract, merge, and normalize producers. That field-by-field inheritance is
|
|
complete before preparation, and the effective values contribute to pipeline
|
|
and checkpoint identity; execution does not interpret configuration defaults.
|
|
|
|
The framework resolver supplies defaults, selects lanes, resolves validator
|
|
chains, checks registered module and artifact compatibility, validates module
|
|
options, and returns the fixed ordered pipeline shape. Positive validator retry
|
|
budgets require an LLM-backed selected validator; deterministic validators are
|
|
rejected during resolution. Eligible LLM-backed producer specifications also
|
|
contribute their declared correction protocol to the resolved metadata. The
|
|
resulting **EffectiveConfig** retains the selected ID, requested selection and reference
|
|
changes, a clone of the input configuration, and the resolved pipeline.
|
|
Callers may therefore retain or modify their input slices and maps without
|
|
changing the resolved result, and later consumers cannot mutate the original
|
|
configuration through the effective value. This ownership includes the nested
|
|
PromptKit local-backend value.
|
|
|
|
Resolution failures stop before module construction and source parsing. They
|
|
include an error path for an unconfigured pipeline, missing module, missing
|
|
capability, incompatible artifact variant, invalid option, invalid reference,
|
|
or invalid lane selection. CLI code maps these valid-invocation failures to the
|
|
runtime error class described in the [CLI reference](../cli.md#output-streams-and-exit-statuses).
|
|
|
|
## Resolved Identity And Redaction
|
|
|
|
The framework assigns the resolved pipeline a deterministic SHA-256 digest
|
|
after defaults, lane selection, module bindings, reference bindings, validator
|
|
chains, selected correction protocols, effective LLM profiles, and artifact
|
|
schema identity have been resolved. The digest excludes
|
|
its own stored value. It identifies resolved composition rather than raw YAML
|
|
bytes, a debug payload, or all runtime state. The CLI records it as invocation
|
|
provenance before execution; cache and checkpoint identity have additional
|
|
owners in [Run State Internals](state.md).
|
|
|
|
Configuration publication uses ConfigKit's redacted effective YAML. Resolved
|
|
pipeline publication uses **RedactedResolvedPipelinePayload**, which copies
|
|
bindings and nested option containers, replaces credential-shaped values with
|
|
**[REDACTED]**, and omits materialized reference content while retaining safe
|
|
provenance. This redaction is deliberately narrow and does not authorize
|
|
recording arbitrary environment values or provider requests.
|
|
|
|
## Invariants To Preserve
|
|
|
|
- ConfigKit composition applies defaults, imports, an optional profile, and
|
|
explicit overrides in one direction with retained origin metadata.
|
|
- A configuration is structurally valid before it is resolved, and a resolved
|
|
pipeline is compatible with the supplied catalog before preparation begins.
|
|
- Whitespace-normalized identifiers are unique wherever they identify a
|
|
pipeline, step, lane, worker, or reference slot.
|
|
- Runtime conversion and resolved-pipeline publication return detached data.
|
|
- The resolved digest changes when resolved composition changes and never
|
|
includes itself.
|
|
|
|
## Focused Tests
|
|
|
|
- **internal/core/config/document_test.go** and
|
|
**operational_document_test.go** cover ConfigKit document loading,
|
|
normalization, imports, profiles, and overrides.
|
|
- **internal/core/config/validation_contract_test.go** covers configuration
|
|
invariants and contextual failures.
|
|
- **internal/core/config/effective_config_contract_test.go** covers defaults,
|
|
selections, overrides, resolution context, digest changes, and ownership.
|
|
- **internal/core/config/loaded_config_test.go** covers safe effective and
|
|
explanation rendering; **redaction_test.go** covers resolved-pipeline
|
|
redaction and reference-content exclusion.
|
|
|
|
Run **go test ./internal/core/config** after changing this boundary. Changes to
|
|
the handoff or resolved-composition semantics also need the focused framework
|
|
pipeline tests.
|