145 lines
8.1 KiB
Markdown
145 lines
8.1 KiB
Markdown
# Configuration Internals
|
|
|
|
This document describes the maintainer-facing configuration boundary in
|
|
**internal/core/config**. The [Configuration](../config.md) reference owns the
|
|
file format, fields, defaults, precedence contract, and selectable keys. The
|
|
[CLI reference](../cli.md) owns command syntax; this document does not redefine
|
|
either interface.
|
|
|
|
## Boundary
|
|
|
|
The configuration package turns a selected YAML file and supported environment
|
|
values into a validated, independently owned configuration. It then resolves a
|
|
requested pipeline against a module catalog before the framework prepares or
|
|
runs anything.
|
|
|
|
| Boundary | Inputs | Outputs | Does not own |
|
|
| --- | --- | --- | --- |
|
|
| Loading | Selected file path and environment lookup | Parsed file model and a populated **Config** | Choosing the file path or reporting a command result. |
|
|
| Validation | **Config** | Structural configuration errors with pipeline, lane, or binding context | Module availability, capabilities, or construction. |
|
|
| Resolution | Valid **Config**, selected pipeline and lanes, runtime reference changes, LLM override, and module catalog | **EffectiveConfig** with a **ResolvedPipeline** | Materializing reference bytes, preparing modules, execution, or filesystem state. |
|
|
| Summary | **Config** or **EffectiveConfig** | Detached redacted payload suitable for debug summaries | Redacting arbitrary process state or provider traffic. |
|
|
|
|
The CLI discovers a configuration file, invokes this package, and supplies the
|
|
result to the framework. Configuration never reads an input file, constructs a
|
|
module, or creates output, cache, or debug paths. Those responsibilities remain
|
|
at their respective [CLI](cli.md), [pipeline](pipeline.md), and
|
|
[run-state](state.md) boundaries.
|
|
|
|
## Loading And Validation
|
|
|
|
The CLI loads configuration in this order:
|
|
|
|
1. parse the selected YAML file strictly into the file model;
|
|
2. start from **Default**;
|
|
3. apply the file model; and
|
|
4. apply the supported environment overrides.
|
|
|
|
This establishes the public precedence order without giving environment input a
|
|
second file schema. Loading and application reject malformed YAML, unsupported
|
|
file versions, unknown fields, invalid values, and identifiers that are empty
|
|
or collide after whitespace normalization. The file application also makes the
|
|
effective extraction-worker default follow the effective LLM limit. A present
|
|
PromptKit local-backend object requires and trims its endpoint, defaults its
|
|
omitted concurrency limit to zero, and is copied so the parsed file model
|
|
cannot alias the populated **Config**. A pipeline `llm_profile` is
|
|
presence-aware: omission remains empty, while a present blank value is
|
|
rejected and a non-empty file value is trimmed before it reaches **Config**.
|
|
|
|
**Config.Validate** checks configuration-only invariants before resolution. It
|
|
rejects incompatible profile sources, invalid state-surface values, unsupported
|
|
concurrency settings, malformed bindings and references, invalid retries, and
|
|
invalid pipeline, step, or lane structure. PromptKit local-backend validation
|
|
accepts only an absolute HTTP or HTTPS endpoint with a host and no user
|
|
information, query, or fragment, and rejects a negative local concurrency
|
|
limit. Its errors retain the closest known pipeline, lane, and binding context.
|
|
It deliberately does not require modules to be registered: that requires a
|
|
catalog and belongs to resolution.
|
|
|
|
The exact user-selectable values and validation rules are defined in
|
|
[Configuration](../config.md). Keep additions to the file model, an
|
|
environment override, its validation, and that reference in the same change.
|
|
|
|
## Effective Resolution
|
|
|
|
**Config.Resolve** first recomputes derived concurrency defaults and validates
|
|
the configuration. It normalizes the requested pipeline ID, copies the selected
|
|
profile, and passes the non-empty command-level LLM profile override, requested
|
|
lane selection, and reference changes to the framework resolver.
|
|
|
|
After module and validator selection, the resolver applies the effective
|
|
profile policy to LLM-backed bindings only: command override, binding profile,
|
|
pipeline profile, then the prompt default. Deterministic bindings remain
|
|
profile-free, and no second inheritance decision occurs during execution. The
|
|
public field definitions and precedence are owned by
|
|
[Configuration](../config.md#pipelines).
|
|
|
|
The framework resolver supplies defaults, selects lanes, resolves validator
|
|
chains, checks registered module and artifact compatibility, validates module
|
|
options, and returns the fixed ordered pipeline shape. The resulting
|
|
**EffectiveConfig** retains the selected ID, requested selection and reference
|
|
changes, a clone of the input configuration, and the resolved pipeline.
|
|
Callers may therefore retain or modify their input slices and maps without
|
|
changing the resolved result, and later consumers cannot mutate the original
|
|
configuration through the effective value. This ownership includes the nested
|
|
PromptKit local-backend value.
|
|
|
|
Resolution failures stop before module construction and source parsing. They
|
|
include an error path for an unconfigured pipeline, missing module, missing
|
|
capability, incompatible artifact variant, invalid option, invalid reference,
|
|
or invalid lane selection. CLI code maps these valid-invocation failures to the
|
|
runtime error class described in the [CLI reference](../cli.md#output-streams-and-exit-statuses).
|
|
|
|
## Resolved Identity And Redaction
|
|
|
|
The framework assigns the resolved pipeline a deterministic SHA-256 digest
|
|
after defaults, lane selection, module bindings, reference bindings, validator
|
|
chains, effective LLM profiles, and artifact schema identity have been
|
|
resolved. The digest excludes
|
|
its own stored value. It identifies resolved composition rather than raw YAML
|
|
bytes, a debug payload, or all runtime state. The CLI records it as invocation
|
|
provenance before execution; cache and checkpoint identity have additional
|
|
owners in [Run State Internals](state.md).
|
|
|
|
Configuration summaries must use **Redacted**, **RedactedSummaryPayload**, or
|
|
**RedactedResolvedPipelinePayload**, never a direct configuration marshal.
|
|
Those methods copy every binding and nested option container, replace values
|
|
whose key is credential-shaped with **[REDACTED]**, and omit materialized
|
|
reference content while retaining safe binding and reference provenance. The
|
|
payload must not alias the source configuration or resolved pipeline.
|
|
PromptKit's local endpoint and concurrency limit are preserved as non-secret
|
|
configuration metadata in the independently owned summary; the object contains
|
|
no credential value. This redaction is deliberately narrow: it protects
|
|
configuration summaries and does not authorize recording arbitrary environment
|
|
values or provider requests.
|
|
|
|
## Invariants To Preserve
|
|
|
|
- Defaults, YAML values, and environment values are applied in one direction;
|
|
later sources may override only their supported operational settings.
|
|
- A configuration is structurally valid before it is resolved, and a resolved
|
|
pipeline is compatible with the supplied catalog before preparation begins.
|
|
- Whitespace-normalized identifiers are unique wherever they identify a
|
|
pipeline, step, lane, worker, or reference slot.
|
|
- Resolution and summary generation return detached data. Redaction must cover
|
|
every configured and resolved binding, including nested validator bindings.
|
|
- The resolved digest changes when resolved composition changes and never
|
|
includes itself.
|
|
|
|
## Focused Tests
|
|
|
|
- **internal/core/config/file_config_contract_test.go** covers strict file
|
|
parsing, normalization, file application, and structural rejection.
|
|
- **internal/core/config/env_contract_test.go** covers supported operational
|
|
overrides and their precedence.
|
|
- **internal/core/config/validation_contract_test.go** covers configuration
|
|
invariants and contextual failures.
|
|
- **internal/core/config/effective_config_contract_test.go** covers defaults,
|
|
selections, overrides, resolution context, digest changes, and ownership.
|
|
- **internal/core/config/redaction_test.go** covers recursive credential
|
|
redaction, reference-content exclusion, and non-aliasing payloads.
|
|
|
|
Run **go test ./internal/core/config** after changing this boundary. Changes to
|
|
the handoff or resolved-composition semantics also need the focused framework
|
|
pipeline tests.
|