eric/distributor
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 7 Wiki Activity

Refresh app and HTTP boundary documentation

Browse Source
This commit is contained in:
Eric Rakestraw
2026-06-03 11:57:37 +00:00
parent 00677148e2
commit 22ce15c707
2 changed files with 188 additions and 232 deletions
Download Patch File Download Diff File Expand all files Collapse all files

240
docs/roadmap/implementation.md
View File

@@ -1,197 +1,91 @@
# HTTP API Stabilization Roadmap
# HTTP API Boundary Roadmap
## Summary
## Purpose
Prepare `distributor` for a later HTTP API without implementing the HTTP
server in this roadmap. The goal is to make the existing run workflow callable
as a single-pipeline, structured, concurrency-safe application use case.
This roadmap records the accepted boundary for a future HTTP API. The current
application exposes CLI commands and internal app-layer run contracts; it does
not implement an HTTP server, HTTP routes, a `serve` command, app-level
authentication, or in-app TLS.
Chosen defaults:
Implemented internal run contracts are documented in `docs/internal/app.md`.
This file is the canonical home for future HTTP boundary decisions until an
HTTP implementation roadmap replaces it.
- Scope: stabilization only; HTTP routes and `serve` command are deferred to a
later roadmap.
- Future HTTP trigger mode: asynchronous start with run status.
- Security boundary: private bind or reverse proxy/mTLS outside the app; no
app-level auth or in-app TLS in v1.
- Trigger input: pipeline ID only.
## Accepted Direction
## Future HTTP Boundary Contract
The future HTTP API should trigger configured distributor pipelines through the
existing app-layer single-pipeline run path and in-memory coordinator.
This roadmap records the target boundary for a later HTTP implementation. No
HTTP server, routes, `serve` command, app-level authentication, or in-app TLS is
implemented as part of this roadmap.
The HTTP API is intentionally narrow:
Future trigger behavior:
- The trigger endpoint accepts exactly one application input: pipeline ID.
- The trigger is asynchronous. A successful admission starts a run and returns a
run ID rather than waiting for publication to finish.
- Run status is exposed through a later status endpoint keyed by run ID. Status
records should expose the run ID, pipeline ID, current status, timestamps, and
- A trigger request accepts only a pipeline ID as application input.
- A trigger request starts work asynchronously and returns a run ID after
admission.
- Run status is read through a separate status endpoint keyed by run ID.
- Status records expose run ID, pipeline ID, current status, timestamps, and
completed report or error details when available.
- Duplicate in-flight runs for the same pipeline ID map to `409 Conflict`.
The application remains a bundle distribution tool. The HTTP API must not turn
`distributor` into a workflow engine, CMS, report generator, or public web
authoring service.
## Error Mapping
Future transport code should map app-layer errors without changing app-layer
error ownership:
- Unknown pipeline IDs map to `404 Not Found`.
- Duplicate in-flight runs for the same pipeline ID map to `409 Conflict`.
- Validation, config, source, destination, publish, transform, and notification
errors map to transport errors according to their app-layer context.
Future runtime and security boundaries:
Duplicate runs must not be queued. Run state remains in memory unless a later
roadmap explicitly adds durable run storage.
- Request context guards admission. Once admitted, the actual run is tied to the
server or coordinator lifetime context, not to the client request lifetime.
- The server defaults to private binding, such as `127.0.0.1`.
- Operators should expose the server through a reverse proxy, private network,
or external mTLS when transport security or remote access is required.
- The first HTTP implementation does not include bearer-token authentication,
in-app TLS configuration, or public-network exposure unless a later roadmap
explicitly changes that decision.
## Context And Lifetime
## Implementation Rules
The request context guards admission. Once a run is admitted, execution is tied
to the server or coordinator lifetime context rather than to the client request
lifetime.
- Implement stages in order; each stage should be one prompt or commit unless
trivially small.
- Before each stage, read `docs/policy/architecture.md`,
`docs/policy/development.md`, and `docs/policy/documentation.md`.
- Preserve existing CLI behavior, text output, JSON output, config semantics,
backend behavior, manifest schema, and destination state schema.
- Do not introduce external dependencies, a CLI framework, a generic workflow
engine, a plugin system, durable run storage, or broad adapter abstractions.
- Do not document the HTTP API as implemented outside `docs/roadmap/`.
This split allows a client disconnect or request timeout to stop waiting for
admission without canceling a run that has already been accepted.
## Stages
## Security Boundary
### Stage 1: Structured Run Core
The first HTTP server should default to private binding, such as `127.0.0.1`.
Operators should expose it through a reverse proxy, private network, or external
mTLS when transport security or remote access is required.
Refactor `internal/app` so the run workflow first produces a structured run
report, then projects that report to text or JSON output.
The first HTTP implementation should not include:
Required behavior:
- bearer-token authentication;
- in-app TLS configuration;
- public-network exposure defaults.
- Keep `app.Run` as the CLI-facing entrypoint.
- Move stdout writes out of the core planning/execution loop where practical.
- Preserve the existing `run --format text` and `run --format json` output
byte-for-byte except where tests already allow map ordering.
- Preserve partial-result behavior: destination-scoped failures produce a
structured report plus an aggregated error.
A later roadmap must explicitly change this security decision before any of
those features are added.
Suggested internal shape:
## Non-Goals
- Introduce an app-owned `RunReport` model containing dry-run state, pipeline
summaries, action records, summary counters, warnings, and output errors.
- Keep JSON tags compatible with the current `run` JSON output.
- Keep failure aggregation in `internal/app`; do not move it into `publish`,
`config`, or storage adapters.
The future HTTP API should not add:
Tests:
- public CLI flags for selecting one pipeline during `distributor run`;
- a CLI framework;
- a generic workflow engine;
- plugin execution;
- durable run storage;
- app-level authentication;
- in-app TLS.
- Add or update `internal/app` tests proving structured reports include
warnings, actions, outputs, summary counters, and partial failures.
- Add CLI regression coverage for existing text and JSON output.
- Run `go test ./internal/app ./internal/cli`.
## Verification Expectations
### Stage 2: Single-Pipeline Run Entry Point
Any later HTTP implementation should preserve existing CLI behavior and keep
the app-layer run contracts tested. At minimum, it should cover:
Add an app-layer entrypoint for running exactly one configured pipeline by ID.
Required behavior:
- Add an exported internal app function such as
`RunPipeline(ctx, RunPipelineOptions) (RunReport, error)`.
- `RunPipelineOptions` must include config path, pipeline ID, dry-run, force,
and optional notifier.
- Invalid pipeline IDs must return a typed or predicate-detectable error
suitable for later HTTP `404` mapping.
- `RunPipeline` must reuse the same backend factory, secret loading, transform
registry, warning generation, destination planning, publish execution,
notification behavior, and failure aggregation as `Run`.
- Do not add a public CLI `run --pipeline` flag in this stage.
Tests:
- Run only the requested pipeline from a multi-pipeline config.
- Return the chosen not-found error for an unknown pipeline ID.
- Preserve existing `app.Run` all-pipelines behavior.
- Run `go test ./internal/app ./internal/cli`.
### Stage 3: In-Memory Pipeline Run Coordinator
Add a narrow concurrency coordinator around the single-pipeline app seam.
Required behavior:
- Keep the coordinator in `internal/app` unless a later HTTP implementation
introduces a transport package.
- Allow different pipeline IDs to run concurrently.
- Reject a second in-flight run for the same pipeline ID with a typed or
predicate-detectable duplicate-run error.
- Track active runs in memory only.
- Clear active state after success, failure, or context cancellation.
- Do not queue duplicate runs and do not persist run state to disk or a
database.
Async-ready constraints:
- Model run IDs and status records so a later HTTP API can expose asynchronous
status.
- A future HTTP request context should guard admission; the actual run should
be tied to a server/coordinator lifetime context.
Tests:
- Concurrent same-pipeline requests produce exactly one accepted run and one
duplicate-run error.
- Concurrent different-pipeline requests both start.
- Active state is cleared after success and after failure.
- Unknown pipeline IDs do not leave active state behind.
- Run `go test ./internal/app`.
### Stage 4: Future HTTP Boundary Contract
Record the target HTTP boundary in this roadmap only; do not implement routes
yet.
Required target contract:
- Future trigger endpoint should accept only a pipeline ID.
- Future trigger behavior should be asynchronous: accept the run, return a run
ID, and expose status through a later status endpoint.
- Future duplicate in-flight pipeline runs should map to `409 Conflict`.
- Future unknown pipeline IDs should map to `404 Not Found`.
- Future server should default to private binding, such as `127.0.0.1`, and
rely on a reverse proxy, private network, or external mTLS for transport
security.
- Do not add bearer-token auth, in-app TLS config, or public-network exposure in
v1 unless a later roadmap explicitly changes this decision.
Tests:
- Documentation-only stage; no tests required beyond any repository doc checks
that exist.
### Stage 5: Stabilization Sweep
Clean up only code made obsolete by stages 1-3.
Required behavior:
- Remove duplicate result projection, warning collection, and
pipeline-selection helpers only when clearly replaced.
- Keep test helpers local unless they are broadly reusable.
- Update `docs/internal/app.md` only for implemented internal app contracts.
- Do not update `docs/cli.md`, `docs/config.md`, `docs/operations.md`, or
README for HTTP behavior because no HTTP API exists yet.
Tests:
- Run `go test ./internal/app ./internal/cli`.
- Run `go test ./...` before considering the roadmap complete.
## Acceptance Criteria
The roadmap is complete when:
- Existing `distributor run` behavior is unchanged.
- `internal/app` exposes a tested single-pipeline run path.
- Run results are available as structured data without scraping stdout.
- The coordinator prevents concurrent same-pipeline runs while allowing
different pipelines to run.
- Future HTTP status, duplicate, not-found, and private-bind decisions are
recorded under `docs/roadmap/`.
- The full test suite passes with `go test ./...`.
- trigger requests with known and unknown pipeline IDs;
- duplicate in-flight trigger requests;
- asynchronous acceptance and status lookup;
- private bind defaults;
- request-context admission behavior;
- coordinator-lifetime run execution.