Configuration Internals

Audience: developers and LLM coding agents changing internal/config.

Purpose

internal/config owns YAML config structs, config file loading, defaulting, validation, size/duration parsing, SSH/S3 normalization helpers, and the credential environment resolver.

Inputs And Outputs

Inputs are YAML files, YAML scalar values, process environment lookup functions, optional secrets directories, and Config values. Outputs are defaulted Config values, validation errors, parsed byte sizes and durations, normalized backend options, loaded secret environments, secret conflict metadata, and resolved credentials.

Boundaries

The package does not open storage backends, authenticate HTTP requests, start servers, publish destinations, or execute transforms. Runtime execution support is wired by internal/app.

The canonical user-facing config reference is docs/config.md.

Config Fields Used

The package defines all user-visible config fields: server.http, secrets, pipelines, source and destination backend fields, validation policy, destination workflow, publish policy, transform policy, path mapping, links, and retention policy.

Adapters Used

No external storage adapters are used directly. The package exposes normalized config and credential values consumed by app-level adapter construction.

State And Manifest Behavior

The package does not parse source manifests or destination state. It validates config values that later affect manifest validation and destination state, such as workflow, publish/transform combinations, links, retention policy, backend roots, S3 prefix shape, and HTTP upload source settings. Legacy destination policy YAML fields are rejected by strict decoding because they are not user-visible config fields.

Skip And Resume Behavior

The package has no runtime skip or resume behavior. Publish planning later applies workflow values and per-run force options to destination catalog outcomes.

Failure Behavior

LoadFile wraps file open, YAML parse, and validation failures with config path context. YAML decoding rejects unknown fields. Validation collects all detected field errors into a single error value.

Secret loading fails for unreadable secrets directories, invalid secret filenames, unreadable secret files, and missing or empty required credential values. Secret conflicts are returned as warnings metadata, not secret values.

Tests To Inspect

internal/config/load_test.go
internal/config/validate_test.go
internal/config/secrets_test.go
internal/config/backend_view_test.go
internal/app/runtime_test.go
example configs under examples/

Architectural Invariants

Defaults are applied before validation.
Unknown YAML fields are rejected.
http_upload is source-only config.
Credential-consuming runtime code must use the config-owned environment resolver.
Secret values are never printed by config warnings.
New user-visible config behavior must update docs/config.md and tests.

3.0 KiB Raw Blame History