Add secrets directory credential resolver

docs/config.md

@@ -72,6 +72,7 @@ Sidecar generation writes `report.html` for `report.md`. It does not mutate the
 
 Top level:
 
+- `secrets.directory`: optional credential secrets directory.
 - `pipelines`: required non-empty list.
 
 Pipeline:
@@ -168,7 +169,18 @@ Defaults are applied after YAML decoding and before validation:
 
 ## Secrets
 
-Do not put literal secrets in config files. S3 credentials may name environment variables:
+Do not put literal secrets in config files. `secrets.directory` lets deployments provide credential values as files:
+
+```yaml
+secrets:
+  directory: /run/secrets/distributor
+```
+
+Each regular file in the directory becomes an internal credential environment value named by the filename. Valid filenames must match `[A-Za-z_][A-Za-z0-9_]*`. Directories are ignored, and symlinks to regular files are followed. Exactly one trailing LF or CRLF is trimmed from each file; other whitespace is preserved.
+
+The resolver checks the real process environment first, then the secrets directory. If both define the same variable with different values, `run` prints a warning with the variable name and uses the real environment value. Secret values are not printed. The process environment is not modified, so SDK default credential chains see only real environment variables.
+
+S3 credentials may name environment variables:
 
 - `credentials.access_key_id_env`
 - `credentials.secret_access_key_env`

docs/internal/config.md

@@ -6,7 +6,7 @@
 
 ## Inputs and outputs
 
-Input is a YAML file containing `pipelines`. Output is a `Config` value with defaults applied and validation completed. Load failures include the config path and whether the failure occurred during file loading, YAML parsing, or validation.
+Input is a YAML file containing optional `secrets` and required `pipelines`. Output is a `Config` value with defaults applied and validation completed. Load failures include the config path and whether the failure occurred during file loading, YAML parsing, or validation.
 
 ## Loading flow
 
@@ -14,6 +14,8 @@ Input is a YAML file containing `pipelines`. Output is a `Config` value with def
 
 Known-field checking rejects misspelled or unknown YAML keys before defaults and validation run.
 
+`LoadFile` does not read secret files. `Run` loads the configured secrets directory after config validation and before backend construction.
+
 ## Defaults
 
 Defaults are applied in `ApplyDefaults`:
@@ -39,6 +41,14 @@ Config validation accepts `local`, `ssh`, and `s3` backend shapes so config file
 
 SSH config uses structured fields: `host`, optional `user`, optional `port`, `path`, optional `ssh_key_file`, optional `known_hosts`, and optional `host_key_policy`. `host_key_policy` accepts YAML booleans and strings and normalizes `true`/`strict`, `accept-new`, and `false`/`off`.
 
+## Secrets and credential resolution
+
+`secrets.directory` points to a directory of credential files. `LoadSecretEnvironment` reads regular files and symlinks to regular files, rejects invalid filenames, trims exactly one trailing LF or CRLF, and returns an `Environment` resolver plus conflict metadata.
+
+The resolver checks the real process environment first and loaded secret values second. Differing process/secret conflicts are reported by variable name only. The resolver does not mutate `os.Environ`; default SDK credential chains continue to see only real process environment values.
+
+Future credential-consuming backend code should resolve explicit credential environment variable references through `Environment.ResolveCredentials` or the same resolver pattern instead of calling `os.Getenv` directly.
+
 The user-facing configuration reference is `docs/config.md`; this file documents package behavior for maintainers.
 
 ## Failure behavior

docs/operations.md

@@ -96,6 +96,19 @@ The default host key policy is `accept-new`. New host keys are written to `known
 
 Recovery boundaries are the same as local storage: replacement deletes only managed output paths recorded in `.distributor.json` plus the state file, and failed writes are cleaned up where practical. Distributor never performs broad recursive remote deletion.
 
+## Secrets Directory
+
+Configure `secrets.directory` when credential values should come from mounted files, such as deployment secrets:
+
+```yaml
+secrets:
+  directory: /run/secrets/distributor
+```
+
+The directory is loaded during `run` before any source or destination backend is opened. If the directory is missing, unreadable, or contains an invalid secret filename, the run fails before publication work starts.
+
+Real process environment values take precedence over files with the same name. If the values differ and stdout is enabled, `run` prints a warning naming the ignored secret file variable without printing either value. The process environment is not changed.
+
 ## Caveats
 
 S3 execution, external notification adapters, and force overwrite behavior are unavailable.

docs/policy/architecture.md

@@ -166,7 +166,7 @@ For example, one destination may publish source files only as a long-term archiv
 
 ## Backend Abstraction
 
-Sources and destinations use the same storage abstraction. Current runtime execution uses the local filesystem backend. Additional storage backends should be peer implementations behind the same interface, and any backend-specific execution limitation must be documented.
+Sources and destinations use the same storage abstraction. Current runtime execution uses the local filesystem and SSH/SFTP backends. Additional storage backends should be peer implementations behind the same interface, and any backend-specific execution limitation must be documented.
 
 Application logic must interact with storage through internal backend interfaces. Backend-specific behavior belongs in adapter packages. Pipeline, bundle, state, publish, and transform packages must not import service-specific or filesystem adapter implementation details.
 
@@ -194,6 +194,7 @@ Use this current layout unless the project has a documented reason to differ:
 - `internal/state`: `.distributor.json` parsing, validation, comparison, and output metadata.
 - `internal/storage`: backend interfaces, shared path/resource types, backend registry, and storage errors.
 - `internal/adapters/local`: local filesystem backend.
+- `internal/adapters/ssh`: SSH/SFTP backend.
 - `internal/transform`: transform interfaces, registry, planning, and shared transform models.
 - `internal/transform/markdown`: Markdown-to-HTML implementation.
 - `internal/publish`: destination planning, reconciliation, safety checks, and publish execution.

docs/policy/development.md

@@ -13,6 +13,7 @@ Use it with `docs/policy/architecture.md` and `docs/policy/documentation.md`.
 - `internal/state`: destination `.distributor.json` parsing, validation, and comparison.
 - `internal/storage`: backend interface, registry, logical path rules, typed errors, and shared storage helpers.
 - `internal/adapters/local`: local filesystem backend.
+- `internal/adapters/ssh`: SSH/SFTP backend.
 - `internal/storage/fake`: in-memory backend for tests.
 - `internal/publish`: destination inspection, output planning, reconciliation, execution, and managed cleanup.
 - `internal/transform`: transform interface and registry.
@@ -82,6 +83,8 @@ The project currently depends on:
 
 - `gopkg.in/yaml.v3` for YAML configuration loading.
 - `github.com/yuin/goldmark` for Markdown rendering.
+- `golang.org/x/crypto/ssh`, `golang.org/x/crypto/ssh/agent`, and `golang.org/x/crypto/ssh/knownhosts` for native SSH support.
+- `github.com/pkg/sftp` for native SFTP support.
 
 Add external dependencies only when they materially improve correctness,
 security, interoperability, or implementation complexity. Avoid dependencies
@@ -104,6 +107,12 @@ Config validation may accept fields for backends that are not executable yet,
 but user-facing docs and examples must clearly state execution support. At the
 time of this policy, local and SSH backends are executable.
 
+Credential-consuming code must use the config-owned environment resolver for
+explicit credential environment variable references. Do not call `os.Getenv`
+directly for backend credentials, because `secrets.directory` values are
+intentionally available through the resolver without mutating the process
+environment.
+
 ## CLI Changes
 
 The CLI is hand-written with the Go standard library. Do not introduce a CLI

docs/troubleshooting.md

@@ -48,6 +48,69 @@ go run ./cmd/distributor run --config <config-path> --dry-run
 
 Safe fix: use `local` or `ssh` for executable workflows. See [configuration](config.md).
 
+## `load secrets directory ... no such file or directory`
+
+Likely cause: `secrets.directory` points to a missing directory.
+
+Diagnostic:
+
+```sh
+ls -ld <secrets-directory>
+```
+
+Safe fix: create or mount the directory before running, or remove `secrets.directory` if no credential files are needed.
+
+## `load secrets directory ... permission denied`
+
+Likely cause: the service user cannot read the configured secrets directory.
+
+Diagnostic:
+
+```sh
+ls -ld <secrets-directory>
+namei -l <secrets-directory>
+```
+
+Safe fix: adjust the directory path or deployment permissions so the service user can read the directory. Distributor does not enforce owner, group, or mode policy beyond OS read access.
+
+## `secret filename ... is invalid`
+
+Likely cause: a regular file in `secrets.directory` does not match `[A-Za-z_][A-Za-z0-9_]*`.
+
+Diagnostic:
+
+```sh
+find <secrets-directory> -maxdepth 1 -type f -printf '%f\n'
+```
+
+Safe fix: rename the file to a valid credential environment variable name, or remove it from the secrets directory.
+
+## `credential environment variable ... is not set`
+
+Likely cause: a backend credential field references an environment variable that is absent from both the real process environment and the configured secrets directory.
+
+Diagnostic:
+
+```sh
+printenv <variable-name>
+ls -l <secrets-directory>/<variable-name>
+```
+
+Safe fix: set the real environment variable or create a readable secrets-directory file with the same name.
+
+## `secret ... ignored because the real environment already has that variable`
+
+Likely cause: the real process environment and secrets directory both define the variable with different values.
+
+Diagnostic:
+
+```sh
+printenv <variable-name>
+ls -l <secrets-directory>/<variable-name>
+```
+
+Safe fix: remove one source of the credential or make the deployment intentionally prefer the real environment value. Distributor does not print either value.
+
 ## `host is required for ssh backend`
 
 Likely cause: SSH config is missing the structured `host` field, or an old URI-based SSH config is still in use.