350 lines
12 KiB
Markdown
350 lines
12 KiB
Markdown
# Implementation Roadmap
|
|
|
|
This roadmap records current implementation status and deferred work for
|
|
`distributor`. Implemented behavior is documented in the user, operator, and
|
|
internal documentation listed below.
|
|
|
|
Canonical current-behavior docs:
|
|
|
|
- `README.md`
|
|
- `docs/cli.md`
|
|
- `docs/config.md`
|
|
- `docs/operations.md`
|
|
- `docs/troubleshooting.md`
|
|
- `docs/internal/`
|
|
- `docs/integrations/markdown.md`
|
|
- `docs/policy/`
|
|
- `examples/`
|
|
|
|
Future, planned, or aspirational behavior belongs under `docs/roadmap/` until
|
|
it is implemented.
|
|
|
|
## Current State
|
|
|
|
`distributor` is ready for routine use against producer pipelines using the
|
|
implemented local, SSH/SFTP, and S3-compatible backends.
|
|
|
|
## Active Roadmap
|
|
|
|
The active roadmap implements the focused feature roadmaps in an order intended
|
|
to minimize later rewrites. The sequence establishes shared CLI output before
|
|
new CLI commands, establishes producer bundle APIs before manifest creation,
|
|
and implements destination path mapping before link generation so URL
|
|
construction does not need to be reworked later.
|
|
|
|
Each stage is sized for one implementation prompt. Future behavior stays under
|
|
`docs/roadmap/` until the stage is implemented and current-behavior docs are
|
|
updated.
|
|
|
|
### Stage 1: CLI Output Policy
|
|
|
|
Goal: implement the shared CLI output contract in
|
|
`docs/roadmap/cli_output_policy.md`.
|
|
|
|
Implementation scope:
|
|
|
|
- add per-command `--format text|json` to current output-producing commands:
|
|
`version`, `validate`, `inspect`, and `run`;
|
|
- keep `text` as the default and preserve existing text output unless the
|
|
policy explicitly requires a change;
|
|
- implement the shared JSON envelope, warnings array, fatal setup error
|
|
behavior, and `run` partial-result JSON with `ok: false`;
|
|
- keep help and usage output text-only.
|
|
|
|
Documentation updates after implementation:
|
|
|
|
- update `docs/cli.md` with the shared `--format text|json` policy;
|
|
- update `docs/operations.md` where JSON output materially improves automation
|
|
workflows;
|
|
- update `docs/troubleshooting.md` only for implemented JSON-mode recovery
|
|
behavior.
|
|
|
|
Tests:
|
|
|
|
- CLI parsing accepts `--format text` and `--format json`;
|
|
- invalid `--format` values are usage errors;
|
|
- each JSON-capable command emits exactly one valid JSON document on success;
|
|
- JSON-mode warnings appear in `warnings` and are not duplicated to stderr;
|
|
- fatal setup errors write no JSON stdout and return non-zero;
|
|
- `run --format json` emits partial-result JSON and exits non-zero when
|
|
destination failures occur after planning begins.
|
|
|
|
Completion criteria: all current output-producing commands support the shared
|
|
policy, text output remains backward compatible, and `go test ./...` passes.
|
|
|
|
### Stage 2: Public Bundle Package Core
|
|
|
|
Goal: implement the core and manifest-building parts of
|
|
`docs/roadmap/public_bundle_package.md`.
|
|
|
|
Implementation scope:
|
|
|
|
- introduce `pkg/bundle` with the public source manifest model, schema version,
|
|
digest logic, parsing, validation, explicit file-list building, scan-based
|
|
building, and zero-`Created` defaulting to current UTC;
|
|
- update internal validation to consume the shared implementation without
|
|
behavior drift;
|
|
- keep destination state, publish planning, storage backends, transforms,
|
|
notifications, and config internal.
|
|
|
|
Documentation updates after implementation:
|
|
|
|
- add package documentation for `pkg/bundle`;
|
|
- update `README.md` and `docs/operations.md` only for implemented Go producer
|
|
support;
|
|
- update `docs/internal/bundle.md` if internal ownership changes.
|
|
|
|
Tests:
|
|
|
|
- public package tests cover explicit lists, scan mode, ordering, digest
|
|
calculation, parsing, validation, unsafe paths, symlinks, and zero `Created`;
|
|
- internal bundle tests continue to pass against the shared implementation;
|
|
- public examples compile where practical.
|
|
|
|
Completion criteria: public and internal validation use one manifest contract,
|
|
and tests prove identical semantics.
|
|
|
|
### Stage 3: Public Bundle Local Writer
|
|
|
|
Goal: implement the local bundle writer portion of
|
|
`docs/roadmap/public_bundle_package.md`.
|
|
|
|
Implementation scope:
|
|
|
|
- add producer-side local bundle writing with staging and atomic promotion where
|
|
practical;
|
|
- keep the writer filesystem-local and producer-focused;
|
|
- do not expose distributor storage backends or publication behavior through
|
|
the public package.
|
|
|
|
Documentation updates after implementation:
|
|
|
|
- document the writer API in `pkg/bundle`;
|
|
- update producer workflow docs only for implemented behavior.
|
|
|
|
Tests:
|
|
|
|
- writer creates a complete valid local bundle;
|
|
- writer output validates through distributor's normal validation path;
|
|
- overwrite and failure behavior avoid leaving a completed bundle path without
|
|
a valid manifest where practical.
|
|
|
|
Completion criteria: producers can create complete valid local bundles through
|
|
the public API without hand-writing manifest files.
|
|
|
|
### Stage 4: Manifest Create CLI
|
|
|
|
Goal: implement `docs/roadmap/manifest_create.md`.
|
|
|
|
Implementation scope:
|
|
|
|
- add `distributor manifest create` over `pkg/bundle`;
|
|
- support explicit-first `--file` selection, recursive scan fallback,
|
|
`--overwrite`, optional `--created`, and shared `--format text|json`;
|
|
- use temp-and-rename replacement for `manifest.json` where practical;
|
|
- do not duplicate bundle contract logic in CLI or app code.
|
|
|
|
Documentation updates after implementation:
|
|
|
|
- update `docs/cli.md` with command syntax, flags, output, and examples;
|
|
- update `docs/operations.md` with a producer workflow;
|
|
- cross-reference `pkg/bundle` for Go producers.
|
|
|
|
Tests:
|
|
|
|
- command creates valid manifests for explicit and scanned files;
|
|
- ordering, dotfile inclusion, metadata exclusion, symlink rejection, and
|
|
overwrite behavior match the roadmap;
|
|
- text and JSON output follow the shared CLI output policy.
|
|
|
|
Completion criteria: the command writes valid `manifest.json`, supports
|
|
text/JSON output, and generated manifests validate through distributor.
|
|
|
|
### Stage 5: Remote Validate and Inspect
|
|
|
|
Goal: implement `docs/roadmap/remote_validate_inspect.md`.
|
|
|
|
Implementation scope:
|
|
|
|
- extend `validate` and `inspect` with mutually exclusive local-path and
|
|
`--config --pipeline` modes;
|
|
- reuse `run` source backend construction and secrets resolution;
|
|
- support configured local, SSH, and S3 sources;
|
|
- preserve `--pipeline` as required in config mode;
|
|
- keep the commands source-only and do not open destinations.
|
|
|
|
Documentation updates after implementation:
|
|
|
|
- update `docs/cli.md` with local and config-driven syntax;
|
|
- update `docs/operations.md` with remote validation examples;
|
|
- update `docs/troubleshooting.md` for common configured-source failures.
|
|
|
|
Tests:
|
|
|
|
- existing local behavior remains unchanged;
|
|
- config-mode validation and inspection work for local and fake/app-level
|
|
source backends;
|
|
- SSH and S3 behavior is covered by focused unit tests and existing opt-in
|
|
integration patterns;
|
|
- JSON output follows the shared CLI output policy.
|
|
|
|
Completion criteria: local behavior is stable, configured local/SSH/S3 sources
|
|
can be validated and inspected, and no destination backend is opened.
|
|
|
|
### Stage 6: HTML Index Mode
|
|
|
|
Goal: implement `docs/roadmap/html_index_mode.md`.
|
|
|
|
Implementation scope:
|
|
|
|
- add `markdown_to_html.mode: index` with fixed `index.html` output;
|
|
- keep `sidecar` as the default and existing explicit mode;
|
|
- implement deterministic input selection through explicit input or exactly one
|
|
manifest-listed Markdown file;
|
|
- reject enabled Markdown-to-HTML transform config when `publish.html` is
|
|
false;
|
|
- record generated state metadata with `transform: markdown_to_html`.
|
|
|
|
Documentation updates after implementation:
|
|
|
|
- update `docs/config.md` with Markdown-to-HTML modes and input selection;
|
|
- update `docs/integrations/markdown.md`;
|
|
- update `docs/operations.md` with an implemented static-site example where
|
|
useful.
|
|
|
|
Tests:
|
|
|
|
- sidecar behavior remains unchanged;
|
|
- index mode handles explicit input, single Markdown fallback, ambiguous input,
|
|
unsafe input, state metadata, dry-run, source-only publication, and output
|
|
collisions.
|
|
|
|
Completion criteria: source publication, sidecar mode, index mode, state
|
|
metadata, dry-run, and collision behavior are all covered.
|
|
|
|
### Stage 7: Latest Path Destinations
|
|
|
|
Goal: implement `docs/roadmap/latest_paths.md`.
|
|
|
|
Implementation scope:
|
|
|
|
- add destination-level `path_mapping.mode` with default `preserve_relative`
|
|
and new `fixed`;
|
|
- make destination bundle path mapping destination-local;
|
|
- for fixed destinations, select only the newest discovered bundle per
|
|
destination before planning writes;
|
|
- add fixed-path dry-run warnings and summary counts;
|
|
- allow `--force` for fixed backend roots while keeping deletion bounded to the
|
|
configured backend root.
|
|
|
|
Documentation updates after implementation:
|
|
|
|
- update `docs/config.md` with `path_mapping`;
|
|
- update `docs/operations.md` with archive-plus-latest fan-out examples;
|
|
- update `docs/cli.md` if dry-run output gains fixed-path indicators.
|
|
|
|
Tests:
|
|
|
|
- omitted and explicit `preserve_relative` match existing behavior;
|
|
- fixed destinations publish at local, SSH, and S3 backend roots;
|
|
- newest-only selection is deterministic;
|
|
- older discovered bundles are not planned or written to fixed destinations;
|
|
- dry-run reports candidate count, selected bundle, and destructive replacement
|
|
warnings;
|
|
- force remains bounded at the backend root.
|
|
|
|
Completion criteria: fixed destination roots work across implemented backends,
|
|
archive-style destinations remain unchanged, and destructive behavior is clear
|
|
in dry-run output.
|
|
|
|
### Stage 8: Link Generation Support
|
|
|
|
Goal: implement `docs/roadmap/link_generation.md`.
|
|
|
|
Implementation scope:
|
|
|
|
- add destination-level `links.base_url` and `links.primary`;
|
|
- generate per-output and primary URLs using URL semantics, not filesystem or
|
|
storage joins;
|
|
- handle `index.html` as a directory-style URL;
|
|
- store optional URL metadata under destination state schema version 1 while
|
|
pre-release;
|
|
- expose links through JSON command output only where supported by the CLI
|
|
output policy.
|
|
|
|
Documentation updates after implementation:
|
|
|
|
- update `docs/config.md` with destination `links` fields;
|
|
- update `docs/operations.md` with static-site URL examples;
|
|
- update `docs/internal/state.md` for URL metadata;
|
|
- update `docs/cli.md` only for implemented CLI link output.
|
|
|
|
Tests:
|
|
|
|
- config validation accepts valid links and rejects invalid schemes, query
|
|
strings, and fragments;
|
|
- nested bundle paths and fixed latest paths generate correct URLs;
|
|
- `index.html` omits the filename;
|
|
- primary selection is deterministic;
|
|
- destinations without `links` produce no URL metadata.
|
|
|
|
Completion criteria: URL generation respects archive and fixed path mapping,
|
|
state records optional URL metadata when configured, and unconfigured
|
|
destinations remain unchanged.
|
|
|
|
### Stage 9: Roadmap Closeout
|
|
|
|
Goal: remove completed roadmap drift after Stages 1-8 are implemented.
|
|
|
|
Implementation scope:
|
|
|
|
- remove or rewrite completed roadmap files whose behavior is fully documented
|
|
in current docs;
|
|
- ensure current docs describe implemented behavior;
|
|
- keep `docs/roadmap/` focused only on remaining future work.
|
|
|
|
Documentation updates after implementation:
|
|
|
|
- update `README.md`, `docs/cli.md`, `docs/config.md`,
|
|
`docs/operations.md`, `docs/troubleshooting.md`, `docs/internal/`,
|
|
`docs/integrations/markdown.md`, and examples only where implemented
|
|
behavior requires it.
|
|
|
|
Tests:
|
|
|
|
- run documentation consistency searches for completed-feature language that
|
|
still appears only as future work;
|
|
- run focused tests for any examples or docs backed by tests;
|
|
- run `go test ./...` if behavior docs and examples changed with code.
|
|
|
|
Completion criteria: completed features are documented as current behavior, and
|
|
`docs/roadmap/` contains only future or deferred work.
|
|
|
|
## Deferred Work
|
|
|
|
These items are not implemented and should stay out of current-behavior docs
|
|
until a roadmap entry is selected and implemented:
|
|
|
|
- external notification adapters;
|
|
- warning-only digest mismatch handling;
|
|
- additional auth mechanisms beyond the implemented SSH and S3 credential
|
|
paths;
|
|
- compatibility parsing for legacy SSH URI config;
|
|
- broad recursive destination deletion outside managed bundle paths;
|
|
- concurrent fan-out publishing;
|
|
- streaming, resumable, or multipart S3 uploads;
|
|
- cloud-provider-specific IAM integration docs;
|
|
- repository-managed packaging, release, and deployment automation.
|
|
|
|
## Roadmap Maintenance
|
|
|
|
When adding future roadmap work:
|
|
|
|
- describe user-visible behavior and safety boundaries;
|
|
- define which existing docs must change after implementation;
|
|
- keep examples secret-free and runnable or clearly environment-gated;
|
|
- avoid workflow labels in production code, tests, config fields, and user
|
|
documentation;
|
|
- run focused tests for the changed behavior and `go test ./...` for
|
|
cross-package changes.
|