# Configuration Internals

## Purpose

`internal/config` defines YAML-backed configuration structs, defaulting, and validation for distributor pipelines.

## Inputs and outputs

Input is a YAML file containing optional `server`, optional `secrets`, and required `pipelines`. Output is a `Config` value with defaults applied and validation completed. Load failures include the config path and whether the failure occurred during file loading, YAML parsing, or validation.

## Loading flow

`LoadFile` opens the requested path, decodes YAML with known-field checking enabled, applies defaults, and validates the result. The app uses `DefaultConfigPath` when the CLI does not supply a config path.

Known-field checking rejects misspelled or unknown YAML keys before defaults and validation run.

`LoadFile` does not read secret files. App entrypoints load the configured
secrets directory after config validation and before credential-consuming work.

## Defaults

Defaults are applied in `ApplyDefaults`:

- HTTP server `bind` defaults to `127.0.0.1:8080`;
- HTTP server `staging_root` defaults to `/var/spool/distributor`;
- HTTP server `max_upload_size` defaults to `20MB`;
- HTTP server `queue_size` defaults to `16`;
- HTTP server `max_concurrency` defaults to `1`;
- HTTP server `retention` defaults to `24h`;
- `http_upload` source `staging_path` defaults to `<server.http.staging_root>/<pipeline id>`;
- `http_upload` source `max_upload_size` defaults to `server.http.max_upload_size`;
- pipeline validation defaults `on_digest_mismatch` to `fail`;
- SSH backend `port` defaults to `22`;
- SSH backend `host_key_policy` defaults to `accept-new`;
- destination publish policy defaults to source output only;
- Markdown-to-HTML mode defaults to `sidecar` when a transform block is present and mode is omitted;
- destination path mapping defaults to `preserve_relative`;
- destination link primary policy defaults to `auto` when a `links` block is present;
- `transfer.on_destination_same` defaults to `skip`;
- `transfer.on_destination_older` defaults to `replace`;
- `transfer.on_destination_newer` defaults to `skip`;
- `transfer.on_conflict` defaults to `fail`.

## Validation responsibilities

Validation requires positive HTTP server limits and retention, at least one pipeline, slug-like unique pipeline ids, one source per pipeline, at least one destination, slug-like unique destination ids within each pipeline, backend-specific required fields, valid validation policy, valid publish and transform combinations, valid destination path mapping mode, valid destination link config, and valid transfer actions.

HTTP upload sources require `token_env`, a staging path after defaults, and a positive maximum upload size. Literal token fields are not part of the YAML schema. The `http_upload` backend is accepted only for sources and rejected for destinations.

Upload size values are parsed from strings with `B`, `KB`, `MB`, or `GB` suffixes using 1024 multipliers. Retention values are parsed with `time.ParseDuration`. Explicit zero values fail validation; omitted values receive defaults before validation.

Transfer validation accepts `replace` for `on_destination_newer` and `on_conflict`, but publish planning honors those destructive actions only when the current run explicitly requests force.

`ValidatePublishTransformPolicy` is shared with publish planning so destination policy combinations are checked consistently. Publishing HTML requires an enabled Markdown-to-HTML transform in `sidecar` or `index` mode. Enabled Markdown-to-HTML config is rejected when `publish.html` is false. `input` is accepted only for enabled `index` mode. A publish policy must select source output, HTML output, or both.

Destination path mapping accepts `preserve_relative` and `fixed`. The app layer applies the mapping when it selects destination bundle paths; config owns only YAML shape, defaulting, and validation.

Destination links are optional. When a `links` block is present, `base_url` is required, must use `http` or `https`, and must not include a query string or fragment. `primary` accepts `auto`, `html`, and `source`.

## Executable support boundary

Config validation accepts `local`, `ssh`, `s3`, and source-only `http_upload` backend shapes. Runtime `run`, `validate`, and `inspect` workflows open `local`, `ssh`, and `s3` through `internal/app`. Runtime `serve` workflows execute `http_upload` sources through the app upload coordinator and HTTP server.

SSH config uses structured fields: `host`, optional `user`, optional `port`, `path`, optional `ssh_key_file`, optional `known_hosts`, and optional `host_key_policy`. `host_key_policy` accepts YAML booleans and strings and normalizes `true`/`strict`, `accept-new`, and `false`/`off`.

S3 config requires `endpoint` and `bucket`, normalizes optional `prefix`, defaults `region` to `us-east-1`, and defaults omitted `force_path_style` to `true` while preserving explicit `false`.

HTTP upload config is source-only. Config owns its YAML shape, defaulting, size and duration parsing, and validation. The config package does not authenticate requests, stage uploads, or execute HTTP upload sources. The app layer resolves `token_env` through the config-owned environment resolver before starting the HTTP server.

## Secrets and credential resolution

`secrets.directory` points to a directory of credential files. `LoadSecretEnvironment` reads regular files and symlinks to regular files, rejects invalid filenames, trims exactly one trailing LF or CRLF, and returns an `Environment` resolver plus conflict metadata.

The resolver checks the real process environment first and loaded secret values second. Differing process/secret conflicts are reported by variable name only. The resolver does not mutate `os.Environ`; default SDK credential chains continue to see only real process environment values.

Credential-consuming backend wiring should resolve explicit credential environment variable references through `Environment.ResolveCredentials` or the same resolver pattern instead of calling `os.Getenv` directly.

The user-facing configuration reference is `docs/config.md`; this file documents package behavior for maintainers.

## Failure behavior

Load errors wrap the underlying file, YAML, or validation error with context. Validation collects all detected field errors into one error value instead of stopping at the first invalid field.

Unsupported backend names fail validation. Accepted backend names without runtime execution support fail later during app backend opening.

## Tests

Before changing config behavior, inspect:

- `internal/config/load_test.go`
- `internal/config/validate_test.go`
- example-loading coverage in `internal/config`
- user-facing examples under `examples/`

## Invariants

- Defaults are applied before validation.
- Unknown YAML fields are rejected.
- `docs/config.md` remains the canonical user-facing config reference.
- Runtime backend execution support is not inferred from config validation support.
- New user-visible config behavior must be covered by tests and docs in the same change.