Add HTTP upload server and serve command

docs/internal/app.md

@@ -34,6 +34,11 @@ same destination fan-out path as normal runs.
 source. They share source backend construction with run workflows and never open
 destination backends.
 
+`Serve` is the CLI-facing HTTP upload server entrypoint. It loads config,
+loads the configured secrets directory, resolves upload bearer tokens for
+configured `http_upload` sources, creates an `UploadCoordinator`, binds
+`server.http.bind`, and serves the upload API until its context is cancelled.
+
 ## Run Reports
 
 `RunReport` is the structured result model for run workflows. It includes
@@ -116,6 +121,32 @@ Completed records retain the final run report or error text until
 their committed staged bundle directories. The coordinator is memory-only and
 does not persist queue state, status records, or run reports.
 
+## HTTP Upload Server
+
+The HTTP upload server is app-layer transport wiring around
+`UploadCoordinator`. It owns request authentication, route dispatch, HTTP status
+mapping, and JSON response projection. Bundle staging and publication remain in
+the coordinator and staged-source run path.
+
+Server startup resolves every configured `http_upload` source `token_env`
+through the config-owned environment resolver after `secrets.directory` has
+been loaded. Startup fails when a token is missing, empty, or duplicates another
+upload pipeline token. Error messages identify environment variable names and
+pipeline ids, but not token values.
+
+Routes:
+
+- `GET /healthz`: returns `200` after config, secrets, tokens, coordinator, and route setup succeed.
+- `POST /upload`: accepts authenticated tar and tar.gz archives and returns an accepted run id.
+- `GET /runs/<run_id>`: returns the current in-memory upload status record or `404`.
+
+The upload token maps to exactly one configured pipeline. Producers do not
+submit pipeline ids, and submitted `pipeline` or `pipeline_id` query values are
+rejected. Full queues are rejected before the request body is read. Oversized
+uploads, unsupported content types, invalid bearer tokens, full queues, and
+unknown status records are mapped to stable HTTP status codes without returning
+secret token values.
+
 ## Coordination
 
 `PipelineRunCoordinator` wraps `RunPipeline` with in-memory admission control.
@@ -171,6 +202,8 @@ Run helpers are grouped by responsibility:
 - `run_notify.go`: notification event projection and action filtering.
 - `run_coordinator.go`: in-memory run admission, run IDs, status records, and duplicate-run errors.
 - `upload_coordinator.go`: in-memory upload admission, queueing, status tracking, staging handoff, and staged-source execution.
+- `upload_http.go`: HTTP upload authentication, routes, JSON response projection, and HTTP error mapping.
+- `serve.go`: config/secrets loading and HTTP server startup.
 - `backends.go`: app-level backend factory wiring.
 - `transforms.go`: app-level transform registry wiring.
 - `source_select.go`: configured-source selection shared by `validate` and `inspect`.

docs/internal/config.md

@@ -14,7 +14,8 @@ Input is a YAML file containing optional `server`, optional `secrets`, and requi
 
 Known-field checking rejects misspelled or unknown YAML keys before defaults and validation run.
 
-`LoadFile` does not read secret files. `Run` loads the configured secrets directory after config validation and before backend construction.
+`LoadFile` does not read secret files. App entrypoints load the configured
+secrets directory after config validation and before credential-consuming work.
 
 ## Defaults
 
@@ -58,13 +59,13 @@ Destination links are optional. When a `links` block is present, `base_url` is r
 
 ## Executable support boundary
 
-Config validation accepts `local`, `ssh`, `s3`, and source-only `http_upload` backend shapes. Runtime execution opens `local`, `ssh`, and `s3` through `internal/app`.
+Config validation accepts `local`, `ssh`, `s3`, and source-only `http_upload` backend shapes. Runtime `run`, `validate`, and `inspect` workflows open `local`, `ssh`, and `s3` through `internal/app`. Runtime `serve` workflows execute `http_upload` sources through the app upload coordinator and HTTP server.
 
 SSH config uses structured fields: `host`, optional `user`, optional `port`, `path`, optional `ssh_key_file`, optional `known_hosts`, and optional `host_key_policy`. `host_key_policy` accepts YAML booleans and strings and normalizes `true`/`strict`, `accept-new`, and `false`/`off`.
 
 S3 config requires `endpoint` and `bucket`, normalizes optional `prefix`, defaults `region` to `us-east-1`, and defaults omitted `force_path_style` to `true` while preserving explicit `false`.
 
-HTTP upload config is source-only. Config owns its YAML shape, defaulting, size and duration parsing, and validation. The config package does not resolve `token_env`, authenticate requests, stage uploads, or execute HTTP upload sources.
+HTTP upload config is source-only. Config owns its YAML shape, defaulting, size and duration parsing, and validation. The config package does not authenticate requests, stage uploads, or execute HTTP upload sources. The app layer resolves `token_env` through the config-owned environment resolver before starting the HTTP server.
 
 ## Secrets and credential resolution