170 lines
6.4 KiB
Markdown
170 lines
6.4 KiB
Markdown
# HTTP API Stabilization Roadmap
|
|
|
|
## Summary
|
|
|
|
Prepare `distributor` for a later HTTP API without implementing the HTTP
|
|
server in this roadmap. The goal is to make the existing run workflow callable
|
|
as a single-pipeline, structured, concurrency-safe application use case.
|
|
|
|
Chosen defaults:
|
|
|
|
- Scope: stabilization only; HTTP routes and `serve` command are deferred to a
|
|
later roadmap.
|
|
- Future HTTP trigger mode: asynchronous start with run status.
|
|
- Security boundary: private bind or reverse proxy/mTLS outside the app; no
|
|
app-level auth or in-app TLS in v1.
|
|
- Trigger input: pipeline ID only.
|
|
|
|
## Implementation Rules
|
|
|
|
- Implement stages in order; each stage should be one prompt or commit unless
|
|
trivially small.
|
|
- Before each stage, read `docs/policy/architecture.md`,
|
|
`docs/policy/development.md`, and `docs/policy/documentation.md`.
|
|
- Preserve existing CLI behavior, text output, JSON output, config semantics,
|
|
backend behavior, manifest schema, and destination state schema.
|
|
- Do not introduce external dependencies, a CLI framework, a generic workflow
|
|
engine, a plugin system, durable run storage, or broad adapter abstractions.
|
|
- Do not document the HTTP API as implemented outside `docs/roadmap/`.
|
|
|
|
## Stages
|
|
|
|
### Stage 1: Structured Run Core
|
|
|
|
Refactor `internal/app` so the run workflow first produces a structured run
|
|
report, then projects that report to text or JSON output.
|
|
|
|
Required behavior:
|
|
|
|
- Keep `app.Run` as the CLI-facing entrypoint.
|
|
- Move stdout writes out of the core planning/execution loop where practical.
|
|
- Preserve the existing `run --format text` and `run --format json` output
|
|
byte-for-byte except where tests already allow map ordering.
|
|
- Preserve partial-result behavior: destination-scoped failures produce a
|
|
structured report plus an aggregated error.
|
|
|
|
Suggested internal shape:
|
|
|
|
- Introduce an app-owned `RunReport` model containing dry-run state, pipeline
|
|
summaries, action records, summary counters, warnings, and output errors.
|
|
- Keep JSON tags compatible with the current `run` JSON output.
|
|
- Keep failure aggregation in `internal/app`; do not move it into `publish`,
|
|
`config`, or storage adapters.
|
|
|
|
Tests:
|
|
|
|
- Add or update `internal/app` tests proving structured reports include
|
|
warnings, actions, outputs, summary counters, and partial failures.
|
|
- Add CLI regression coverage for existing text and JSON output.
|
|
- Run `go test ./internal/app ./internal/cli`.
|
|
|
|
### Stage 2: Single-Pipeline Run Entry Point
|
|
|
|
Add an app-layer entrypoint for running exactly one configured pipeline by ID.
|
|
|
|
Required behavior:
|
|
|
|
- Add an exported internal app function such as
|
|
`RunPipeline(ctx, RunPipelineOptions) (RunReport, error)`.
|
|
- `RunPipelineOptions` must include config path, pipeline ID, dry-run, force,
|
|
and optional notifier.
|
|
- Invalid pipeline IDs must return a typed or predicate-detectable error
|
|
suitable for later HTTP `404` mapping.
|
|
- `RunPipeline` must reuse the same backend factory, secret loading, transform
|
|
registry, warning generation, destination planning, publish execution,
|
|
notification behavior, and failure aggregation as `Run`.
|
|
- Do not add a public CLI `run --pipeline` flag in this stage.
|
|
|
|
Tests:
|
|
|
|
- Run only the requested pipeline from a multi-pipeline config.
|
|
- Return the chosen not-found error for an unknown pipeline ID.
|
|
- Preserve existing `app.Run` all-pipelines behavior.
|
|
- Run `go test ./internal/app ./internal/cli`.
|
|
|
|
### Stage 3: In-Memory Pipeline Run Coordinator
|
|
|
|
Add a narrow concurrency coordinator around the single-pipeline app seam.
|
|
|
|
Required behavior:
|
|
|
|
- Keep the coordinator in `internal/app` unless a later HTTP implementation
|
|
introduces a transport package.
|
|
- Allow different pipeline IDs to run concurrently.
|
|
- Reject a second in-flight run for the same pipeline ID with a typed or
|
|
predicate-detectable duplicate-run error.
|
|
- Track active runs in memory only.
|
|
- Clear active state after success, failure, or context cancellation.
|
|
- Do not queue duplicate runs and do not persist run state to disk or a
|
|
database.
|
|
|
|
Async-ready constraints:
|
|
|
|
- Model run IDs and status records so a later HTTP API can expose asynchronous
|
|
status.
|
|
- A future HTTP request context should guard admission; the actual run should
|
|
be tied to a server/coordinator lifetime context.
|
|
|
|
Tests:
|
|
|
|
- Concurrent same-pipeline requests produce exactly one accepted run and one
|
|
duplicate-run error.
|
|
- Concurrent different-pipeline requests both start.
|
|
- Active state is cleared after success and after failure.
|
|
- Unknown pipeline IDs do not leave active state behind.
|
|
- Run `go test ./internal/app`.
|
|
|
|
### Stage 4: Future HTTP Boundary Contract
|
|
|
|
Record the target HTTP boundary in this roadmap only; do not implement routes
|
|
yet.
|
|
|
|
Required target contract:
|
|
|
|
- Future trigger endpoint should accept only a pipeline ID.
|
|
- Future trigger behavior should be asynchronous: accept the run, return a run
|
|
ID, and expose status through a later status endpoint.
|
|
- Future duplicate in-flight pipeline runs should map to `409 Conflict`.
|
|
- Future unknown pipeline IDs should map to `404 Not Found`.
|
|
- Future server should default to private binding, such as `127.0.0.1`, and
|
|
rely on a reverse proxy, private network, or external mTLS for transport
|
|
security.
|
|
- Do not add bearer-token auth, in-app TLS config, or public-network exposure in
|
|
v1 unless a later roadmap explicitly changes this decision.
|
|
|
|
Tests:
|
|
|
|
- Documentation-only stage; no tests required beyond any repository doc checks
|
|
that exist.
|
|
|
|
### Stage 5: Stabilization Sweep
|
|
|
|
Clean up only code made obsolete by stages 1-3.
|
|
|
|
Required behavior:
|
|
|
|
- Remove duplicate result projection, warning collection, and
|
|
pipeline-selection helpers only when clearly replaced.
|
|
- Keep test helpers local unless they are broadly reusable.
|
|
- Update `docs/internal/app.md` only for implemented internal app contracts.
|
|
- Do not update `docs/cli.md`, `docs/config.md`, `docs/operations.md`, or
|
|
README for HTTP behavior because no HTTP API exists yet.
|
|
|
|
Tests:
|
|
|
|
- Run `go test ./internal/app ./internal/cli`.
|
|
- Run `go test ./...` before considering the roadmap complete.
|
|
|
|
## Acceptance Criteria
|
|
|
|
The roadmap is complete when:
|
|
|
|
- Existing `distributor run` behavior is unchanged.
|
|
- `internal/app` exposes a tested single-pipeline run path.
|
|
- Run results are available as structured data without scraping stdout.
|
|
- The coordinator prevents concurrent same-pipeline runs while allowing
|
|
different pipelines to run.
|
|
- Future HTTP status, duplicate, not-found, and private-bind decisions are
|
|
recorded under `docs/roadmap/`.
|
|
- The full test suite passes with `go test ./...`.
|