Close completed catalog roadmap
All checks were successful
ci/woodpecker/tag/release Pipeline was successful

This commit is contained in:
2026-06-19 17:15:24 +00:00
parent 5e47d89355
commit 8366af6fb6
6 changed files with 11 additions and 689 deletions

View File

@@ -270,7 +270,9 @@ destinations:
Destination ids must be unique within a pipeline. Destination ids must be unique within a pipeline.
Legacy destination policy keys `state`, `reconciliation`, `takeover`, and `transfer` are not accepted config fields. YAML files containing those keys fail during config loading. Pre-workflow destination policy keys for state mode, conflict handling,
ownership adoption, and per-comparison copy decisions are not accepted config
fields. YAML files containing those keys fail during config loading.
## Destination Workflow ## Destination Workflow

View File

@@ -178,7 +178,7 @@ Application logic must interact with storage through internal backend interfaces
Adapters should be thin. Backend adapters should implement storage operations and translate backend-specific errors, but should not make bundle comparison, transform, routing, or replacement decisions. Adapters should be thin. Backend adapters should implement storage operations and translate backend-specific errors, but should not make bundle comparison, transform, routing, or replacement decisions.
Remote file-transfer support should prefer native protocol implementations over shelling out, unless a later design document records a reason to differ. Remote file copy support should prefer native protocol implementations over shelling out, unless a later design document records a reason to differ.
## Dependency Policy ## Dependency Policy
@@ -272,7 +272,7 @@ Errors should be actionable and preserve context. Wrap errors with operation, pi
Errors and logs must not expose secrets. Errors and logs must not expose secrets.
Use structured logging where practical. Logs should describe discovery, validation, planned actions, skipped transfers, conflicts, replacements, external calls, retries, and failure causes, but should not include large report contents by default. Use structured logging where practical. Logs should describe discovery, validation, planned actions, skipped copies, conflicts, replacements, external calls, retries, and failure causes, but should not include large report contents by default.
Skip and no-op decisions should be logged at an appropriate level so operators can distinguish successful publication from intentional no-op behavior. Skip and no-op decisions should be logged at an appropriate level so operators can distinguish successful publication from intentional no-op behavior.

View File

@@ -1,283 +0,0 @@
# Catalog State And Destination Workflow Roadmap
This roadmap records the intended first-class state model and destination
workflow configuration for replacement and additive publication workflows.
Current `single_owner` and `shared_root` state are oriented around comparing a
destination owner to one latest source manifest. That works well for replacement
workflows, where a producer maintains a curated source of truth and
`distributor` makes a destination match it. It is a poor fit for additive
workflows, where each run contributes new or updated output paths while
preserving unrelated managed outputs from previous runs and other pipelines.
The target model is a current-state catalog: `.distributor.json` records the
currently managed output paths at a destination root, and each output record
stores its current owner, compact source identity, content digest, and update
metadata. It is not intended to be an audit log.
Both replacement and additive workflows should use the same catalog state shape.
The workflow choice is runtime policy from destination config, not persisted
state.
## Locked Decisions
- Add one destination state mode named `catalog` for both replacement and
additive workflows.
- Catalog state writes `.distributor.json` schema version `4`.
- Add a destination-level `workflow` setting with accepted values
`replacement` and `additive`.
- Default `workflow` to `additive` because it is the least destructive workflow.
- `workflow` replaces the normal user-facing need to combine `state.mode`,
`reconciliation.mode`, `takeover.mode`, and transfer conflict settings for the
two primary workflows.
- `workflow: replacement` treats the current planned outputs as the authoritative
managed output set for the destination scope and removes previously managed
outputs in that scope when they are no longer planned.
- `workflow: additive` writes or overwrites the planned output paths and retains
unrelated managed outputs.
- If a planned path already exists as a managed output, the new publication may
overwrite it and becomes that path's current owner.
- If a planned path exists in storage but is not recorded in valid catalog state,
it remains unmanaged content and must not be adopted implicitly.
- If an output path changes owner, preserve the output record's existing
`created_at` and update only `updated_at`.
- Catalog state records current ownership only. It does not retain historical
owners, historical sources, or old versions of overwritten output records.
- `pipeline_id` and `destination_id` are stored separately on each output
record. They are not concatenated into one owner string.
- Catalog state does not include top-level `owners`; owners are derivable from
the output records.
- Catalog state does not include top-level `sources`; compact source identity is
stored directly on each output record.
- Catalog state does not record whether the last run used `replacement` or
`additive`. Workflow is execution policy, and persisting it would create
drift risk if config changes later.
- Legacy destination policy fields should be rejected outright in the new
workflow config model. Do not retain aliases for `state`, `reconciliation`,
`takeover`, or `transfer`.
## Destination Workflow Semantics
The user-facing destination config should express intent directly:
```yaml
destinations:
- id: weather-latest
backend: local
path: /srv/reports/weather/latest
workflow: additive
```
```yaml
destinations:
- id: weather-archive
backend: local
path: /srv/reports/weather/archive
workflow: replacement
```
`workflow` is orthogonal to backend config, path mapping, publish source/HTML
selection, transforms, links, and retention.
### Replacement Workflow
A replacement destination is for producers that maintain a curated source of
truth and expect `distributor` to make the destination's managed scope match the
current publication.
For a destination configured with `workflow: replacement`:
- planned outputs are written to their resolved destination paths;
- planned outputs may overwrite existing managed outputs at the same path;
- each written output record is replaced in-place with the current
publication's owner, source identity, hash, size, and timestamps;
- managed outputs in the destination scope that are not present in the current
plan are deleted and removed from catalog state;
- unmanaged destination content remains unmanaged and blocks planned path
collisions unless an explicit force workflow later chooses otherwise;
- the catalog state shape remains the same as additive workflow state.
### Additive Workflow
An additive destination is for producers that routinely contribute outputs to a
shared destination root.
For a destination configured with `workflow: additive`:
- planned outputs are written to their resolved destination paths;
- planned outputs may overwrite existing managed outputs at the same path;
- each overwritten output record is replaced in-place with the current
publication's owner, source identity, hash, size, and timestamps;
- managed outputs not present in the current plan are retained;
- unrelated managed outputs from other pipelines or destinations are retained;
- unmanaged destination content remains unmanaged and blocks planned path
collisions unless an explicit force workflow later chooses otherwise;
- pruning can select retained managed outputs by `updated_at` and, when useful,
by `pipeline_id` and/or `destination_id`.
Replacement workflow treats the current publication as the desired managed
output set for a destination scope. Additive workflow treats the current
publication as a patch to the catalog of currently managed outputs.
## Destination State Schema Version 4
Catalog state should use this top-level shape:
```json
{
"schema_version": 4,
"distributor_version": "dev",
"created_at": "2026-06-19T12:00:00Z",
"updated_at": "2026-06-19T12:05:00Z",
"state": {
"mode": "catalog"
},
"outputs": []
}
```
Top-level fields:
- `schema_version`: required. Value `4` for catalog state.
- `distributor_version`: optional diagnostic version string.
- `created_at`: required RFC3339 timestamp for when the catalog state file was
first created.
- `updated_at`: required RFC3339 timestamp for the latest catalog state update.
- `state.mode`: required. Value `catalog`.
- `outputs`: required array of currently managed output records.
Catalog state should not include top-level `pipeline_id`, `destination_id`,
`published_at`, `workflow`, `owners`, `sources`, or a full source manifest.
Those concepts belong on output records or in configuration.
## Output Record Schema
Each output record should be self-contained enough to support current
ownership, pruning, repair, bitrot checks, and basic provenance without a
separate owner or source catalog.
```json
{
"path": "tomorrow/index.html",
"pipeline_id": "weatherreporter.daily",
"destination_id": "latest-html",
"source": {
"id": "weatherreporter.tomorrow",
"digest": "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
"created": "2026-06-19T12:00:00Z"
},
"kind": "generated",
"source_path": "report.md",
"transform": "markdown_to_html",
"sha256": "sha256:abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789",
"size": 12345,
"created_at": "2026-06-19T12:00:00Z",
"updated_at": "2026-06-19T12:05:00Z",
"url": "https://reports.example.com/weather/tomorrow/"
}
```
Required output fields:
- `path`: destination-relative managed output path.
- `pipeline_id`: pipeline that most recently wrote this output path.
- `destination_id`: destination that most recently wrote this output path.
- `source`: compact identity of the source bundle that produced this output.
- `source.id`: source manifest id.
- `source.digest`: source manifest digest.
- `source.created`: source manifest creation timestamp, RFC3339.
- `kind`: `source` or `generated`.
- `sha256`: digest of the output bytes.
- `size`: output byte size.
- `created_at`: RFC3339 timestamp for when this output path first became
managed in catalog state.
- `updated_at`: RFC3339 timestamp for when this output path was most recently
written or updated.
Optional output fields:
- `source_path`: present only for generated outputs; source manifest path used
to produce the generated output.
- `transform`: present only for generated outputs; transform name used to
produce the generated output.
- `url`: present only when destination link configuration produces a public URL
for this output.
For copied source outputs, `source_path` and `transform` should be omitted. The
output `path` is already the copied source artifact path.
## Useful Properties
The catalog shape supports both primary workflows without unnecessary
normalization:
- current owner of each path is explicit;
- current source identity for each path is explicit;
- bitrot checks can compare storage bytes to `sha256`;
- pruning can use `updated_at`, optionally scoped by `pipeline_id` and
`destination_id`;
- repair can remove missing managed output records without consulting an owner
or source table;
- overwriting an output path updates one output record in place;
- no orphaned top-level owner/source records need to be maintained.
The schema intentionally avoids storing a full source manifest for every output.
The source manifest remains the producer-to-distributor validation contract, but
catalog state only needs compact source identity for currently managed outputs.
## Relationship To Existing State And Config Modes
Existing state modes remain current behavior until catalog mode is implemented:
- `single_owner` schema version `2` supports one owner for a destination bundle
path.
- `shared_root` schema version `3` supports multiple owners in one destination
root but still keeps owner records with latest source manifests.
- `catalog` schema version `4` should support replacement and additive
current-state ownership without top-level owner or source catalogs.
The intended user-facing config should move toward `workflow: replacement` and
`workflow: additive` instead of requiring ordinary users to combine
`state.mode`, `reconciliation.mode`, `takeover.mode`, and transfer conflict
settings.
Because the project is still alpha pre-release, catalog implementation should be
a clean break:
- remove legacy write paths for current `single_owner` and `shared_root` state;
- do not implement migration from schema versions `1`, `2`, or `3`;
- do not preserve backwards compatibility for legacy destination state;
- when a configured destination writes successfully, write schema version `4`
catalog state;
- if an existing `.distributor.json` has schema version lower than `4`, treat it
as superseded legacy state for planning purposes and overwrite it according to
the configured workflow, without attempting conversion.
For the first catalog run against superseded legacy state:
- `workflow: replacement` may clear the bounded destination root before writing
the planned outputs and schema version `4` catalog state.
- `workflow: additive` may overwrite only the planned output paths, then write
schema version `4` catalog state containing those planned outputs.
- unplanned files left behind by an additive run against superseded legacy state
are not recorded in catalog state and are treated as unmanaged content by
later catalog runs.
The target direction is that both primary workflows use catalog state.
## Prune And Reconcile-State Scope
Catalog maintenance commands should initially keep the existing ownership
selector model:
- `prune --pipeline <id> --destination <id>` operates only on outputs currently
owned by that pipeline/destination.
- `reconcile-state --pipeline <id> --destination <id>` repairs only outputs
currently owned by that pipeline/destination.
- `reconcile-state --all-owners` repairs all output records in the catalog.
- `prune` remains scoped to the selected pipeline/destination owner only in the
initial catalog implementation.
Do not add catalog-specific selectors in the initial implementation, such as
`--source-id`, `--path-prefix`, or `--kind`. Those may be useful later, but they
are not required to make additive workflow first-class.

View File

@@ -93,10 +93,10 @@ internal docs.
- Current upload status, queue, and idempotency state are memory-only. - Current upload status, queue, and idempotency state are memory-only.
- Producers submit complete tar or gzip-compressed tar source bundles today. - Producers submit complete tar or gzip-compressed tar source bundles today.
- Producers do not choose destination ids, destination paths, transforms, links, - Producers do not choose destination ids, destination paths, transforms, links,
publish policy, transfer policy, storage backends, reconciliation mode, state publish policy, destination workflow, storage backends, or retention policy
mode, or retention policy through upload requests. through upload requests.
- Source manifests remain free of routing, destination, transform, credential, - Source manifests remain free of routing, destination, transform, credential,
reconciliation, state, and retention data. workflow, state, and retention data.
- Public access policy, TLS termination, and rate limiting belong outside - Public access policy, TLS termination, and rate limiting belong outside
`distributor` unless a future implementation changes that boundary. `distributor` unless a future implementation changes that boundary.
- `distributor` is not a broad storage synchronization tool unless a future - `distributor` is not a broad storage synchronization tool unless a future

View File

@@ -1,399 +0,0 @@
# Catalog Follow-Up Implementation Roadmap
This is the active staged plan for closing the remaining catalog-state
implementation gaps found after the schema version `4` catalog workflow landed.
It supersedes the earlier catalog implementation plan.
The current code already supports destination `workflow: additive` and
`workflow: replacement`, writes catalog state schema version `4`, rejects legacy
destination policy YAML fields, and routes publish execution through catalog
planning for normal runs. The remaining work is to make the implemented behavior
release-ready: restore a meaningful catalog `--force` path, synchronize current
docs and examples, and remove legacy single-owner/shared-root implementation
debt.
## Current Baseline
- Destination config accepts `workflow: additive` and `workflow: replacement`.
- Omitted `workflow` defaults to `additive`.
- Destination YAML fields `state`, `reconciliation`, `takeover`, and `transfer`
are rejected by strict YAML decoding.
- Successful publish writes `.distributor.json` schema version `4` with
`state.mode: catalog`.
- Schema versions `1`, `2`, and `3` are treated as superseded legacy state for
publish planning.
- `run --force` remains in the CLI and docs, but catalog publish planning does
not currently select `force_replace`.
- Current docs and examples still describe legacy config/state behavior in
several places.
- Legacy single-owner and shared-root state types, tests, and execution helpers
remain in the codebase even though normal publish planning no longer uses
them.
## Implementation Principles
- Preserve the catalog state shape defined in `docs/roadmap/catalog.md`.
- Keep `workflow` as runtime config policy; do not persist it in
`.distributor.json`.
- Keep unmanaged content protected by default.
- Keep `--force` explicit, per-run, dry-runnable, and bounded to the configured
destination bundle path.
- Do not reintroduce legacy config aliases or compatibility migration.
- Keep backend adapters unaware of catalog, workflow, and force policy.
- Update current-behavior docs in the same stage that makes the described
behavior true.
## Active Implementation Stages
## Stage 1: Catalog Force Planning And Execution
Goal: make `run --force` meaningful for catalog workflows while keeping normal
catalog safety conservative.
Implementation scope:
- Update `internal/publish` planning so `req.Force` can select
`force_replace` for exceptional destructive catalog cases.
- Force should apply to these cases:
- no valid `.distributor.json` and destination bundle path has unmanaged
content;
- planned path collision with storage content not recorded in valid catalog
state;
- invalid destination state JSON or invalid destination state fields;
- unsupported future destination state schema, if the operator explicitly
chooses force replacement after dry-run review.
- Force should not be needed for ordinary valid catalog-managed upserts or
replacements. Additive and replacement workflow behavior remains normal managed
behavior.
- `force_replace` must delete only the bounded destination bundle path through
the storage abstraction, then write planned outputs and schema version `4`
catalog state.
- For fixed-path destinations, the bounded destination bundle path is the
configured backend root. Dry-run output must make that clear.
- Preserve default behavior without `--force`:
- unmanaged planned path collisions fail as `fail_unmanaged`;
- invalid or future state fails as `fail_conflict`;
- no-state non-empty destinations fail as unmanaged.
- Ensure `force_replace` state creation uses the same catalog output projection
as normal publish planning.
- Keep `Force` out of config. There must be no persistent force default.
- Remove or update skipped force tests that were disabled during catalog
implementation.
Current-behavior documentation updates:
- Update only the force-related sections of:
- `docs/cli.md`;
- `docs/operations.md`;
- `docs/troubleshooting.md`;
- `docs/internal/publish.md`;
- backend integration docs where they describe forced deletion boundaries.
- Document force as an exceptional catalog recovery/replacement workflow, not as
a normal way to handle valid managed catalog state.
Tests:
- `go test ./internal/publish`
- `go test ./internal/app ./internal/cli`
- `go test ./internal/adapters/local ./internal/storage ./internal/storage/fake`
- Planning tests:
- no-state non-empty destination returns `fail_unmanaged` without force and
`force_replace` with force;
- unmanaged planned path collision returns `fail_unmanaged` without force and
`force_replace` with force;
- invalid state returns `fail_conflict` without force and `force_replace` with
force;
- future schema returns `fail_conflict` without force and `force_replace` with
force;
- valid catalog additive and replacement actions do not become
`force_replace`.
- Execution tests:
- `force_replace` clears only the destination bundle path;
- sibling paths outside the bundle path survive for local, fake S3, and fake
SSH-style backends;
- fixed-path force clears the configured backend root and is reported as such;
- resulting state is schema version `4` catalog state.
- CLI/app tests:
- dry-run `--force` reports `force_replace` without writing;
- normal `--force` executes and increments only the `force_replace` counter;
- JSON output includes stable `force_replace` action and summary fields.
Completion criteria:
- `run --force` has observable, tested catalog behavior.
- Force is still unnecessary for normal managed catalog replacement/upsert.
- Unmanaged/invalid/future-state replacement remains impossible without explicit
`--force`.
## Stage 2: Current Documentation And Example Synchronization
Goal: make all current-behavior docs and copyable examples match catalog schema
version `4` and the `workflow` config model.
Implementation scope:
- Rewrite `docs/config.md` so destination behavior is described through:
- `workflow: additive`;
- `workflow: replacement`;
- `publish`;
- `transform`;
- `path_mapping`;
- `links`;
- `retention`.
- Remove current-behavior documentation for destination config fields:
- `state`;
- `reconciliation`;
- `takeover`;
- `transfer`.
- Rewrite `docs/integrations/destination-state.md` around schema version `4`
catalog state:
- top-level catalog fields;
- output record fields;
- source identity fields;
- generated-output-only `source_path` and `transform`;
- optional per-output `url`;
- superseded legacy schema handling for publish planning.
- Update `docs/operations.md`:
- additive workflow semantics;
- replacement workflow semantics;
- catalog `force_replace`;
- prune and reconcile-state catalog behavior;
- dry-run action labels.
- Update `docs/troubleshooting.md`:
- replace legacy conflict guidance with workflow/catalog guidance;
- describe rejected legacy config fields;
- describe force-only exceptional cases.
- Update `docs/cli.md`:
- run summary counters now use `publish_new`, `upsert_additive`,
`replace_catalog`, `skip_same`, `force_replace`, `fail_unmanaged`, and
`fail_conflict`;
- remove legacy action labels from current command reference.
- Update relevant internal docs:
- `docs/internal/app.md`;
- `docs/internal/config.md`;
- `docs/internal/publish.md`;
- `docs/internal/state.md`.
- Update policy docs where current architectural text still describes
single-owner/shared-root behavior as current:
- `docs/policy/architecture.md`;
- `docs/policy/development.md`.
- Update examples:
- remove or rewrite `examples/shared-root.yml`;
- remove or rewrite `examples/merge-reconciliation.yml`;
- ensure examples use `workflow` where workflow intent matters;
- keep examples valid, secret-free, and copyable.
- Do not document any future catalog selectors, state migration, or compatibility
aliases outside `docs/roadmap/`.
Tests and checks:
- `go test ./internal/config ./internal/cli`
- `go test ./...`
- Run the config/example checks already used by the test suite.
- Manual smoke checks:
- `go run ./cmd/distributor run --config examples/local-publish.yml --dry-run`
- `go run ./cmd/distributor run --config examples/archive-and-latest.yml --dry-run`
- smoke any rewritten replacement/additive examples.
- Consistency searches:
- `rg -n "state:|reconciliation:|takeover:|transfer:" examples docs --glob '!docs/roadmap/**'`
- `rg -n "single_owner|shared_root|schema version \`2\`|schema version \`3\`" docs --glob '!docs/roadmap/**'`
- `rg -n "replace_older|replace_newer|replace_conflict|replace_takeover|takeover_mode" docs README.md examples --glob '!docs/roadmap/**'`
- `rg -n "workflow: additive|workflow: replacement|schema_version.*4" docs examples`
Completion criteria:
- Current docs describe implemented catalog behavior only.
- Copyable examples load successfully.
- No current doc tells users to configure rejected legacy fields.
- Destination state documentation is schema version `4` first.
## Stage 3: Catalog Idempotent Skip
Goal: make `skip_same` a real catalog no-op optimization instead of a stale
legacy action label.
Implementation scope:
- Implement `skip_same` when every planned output is already catalog-managed
with matching:
- pipeline id;
- destination id;
- source id;
- source digest;
- source created timestamp;
- output path;
- output kind;
- output digest;
- output size;
- generated output `source_path`;
- generated output `transform`;
- output URL metadata.
- Do not read destination bytes for this optimization. Trust valid catalog
metadata; bitrot detection remains a separate future concern.
- For `workflow: additive`, allow `skip_same` when all planned outputs match and
no planned output needs to be written. Unrelated catalog outputs are ignored
for the skip decision and remain retained.
- For `workflow: replacement`, allow `skip_same` only when all planned outputs
match and there are no current-owner catalog outputs that replacement workflow
would delete.
- Never return `skip_same` for superseded legacy state, invalid state, unmanaged
content, future state, or force replacement.
- `skip_same` must not write outputs, rewrite state, delete files, or notify.
- Count `skip_same` in text and JSON summaries.
- Keep behavior deterministic and identical across local, SSH, and S3
destinations.
Current-behavior documentation updates:
- Update `docs/cli.md`, `docs/operations.md`, and `docs/internal/publish.md` to
describe catalog `skip_same`.
- Document that `skip_same` is metadata-based and does not validate destination
bytes.
Tests:
- `go test ./internal/publish ./internal/app ./internal/cli`
- Repeated additive publish with identical catalog metadata returns
`skip_same`.
- Repeated replacement publish with identical catalog metadata and no omitted
current-owner outputs returns `skip_same`.
- Changed source digest, output digest, URL, transform mode, owner, output size,
or generated source path prevents skip.
- Replacement workflow does not skip when it would delete omitted current-owner
outputs.
- `skip_same` does not write outputs or state and does not notify.
- `go test ./...`
Completion criteria:
- Catalog action vocabulary has no stale action label.
- Repeated publish behavior is intentional, documented, and tested.
## Stage 4: Legacy Publish And State Code Removal
Goal: remove dead or near-dead single-owner/shared-root write and planning code
after catalog force behavior, docs, and skip semantics are settled.
Implementation scope:
- Remove legacy publish actions that are no longer planned or documented:
- `replace_older`;
- `replace_newer`;
- `replace_conflict`;
- `replace_takeover`;
- `skip_destination_newer`.
- Keep only catalog-era actions:
- `publish_new`;
- `upsert_additive`;
- `replace_catalog`;
- `skip_same`;
- `force_replace`;
- `fail_unmanaged`;
- `fail_conflict`.
- Remove legacy fields from `publish.Plan`:
- `StateMode`;
- `Reconciliation`;
- `TakeoverMode`;
- `ExistingState`;
- `ExistingSharedRoot`;
- shared-root owner/output carry fields retained only for old execution.
- Remove single-owner and shared-root execution branches from
`internal/publish/execute.go`.
- Remove unused single-owner/shared-root helper functions from `internal/publish`
once no tests or callers use them.
- Remove legacy destination config internals from `internal/config`:
- `StatePolicy`;
- `ReconciliationPolicy`;
- `TakeoverPolicy`;
- `TransferPolicy`;
- legacy constants and defaults that are no longer referenced.
- In `internal/state`, keep only what is needed for:
- parsing schema version `4` catalog state;
- identifying schema versions `1`, `2`, and `3` as superseded legacy by
schema number;
- validating and writing catalog state;
- pruning and reconciling catalog outputs.
- Delete or rewrite tests that assert legacy state parsing, validation,
comparison, shared-root ownership, or single-owner output behavior.
- Preserve test fixtures only where they are used to create superseded legacy
state for first catalog-run behavior. Prefer small local helpers over keeping
broad legacy state builders.
- Remove stale skipped tests that only represent old behavior. If a skipped test
still represents current expected behavior, unskip and update it.
Current-behavior documentation updates:
- No new user docs should be needed if Stage 2 is complete.
- Update internal docs only if removal changes internal package contracts beyond
what Stage 2 already documented.
Tests:
- `go test ./internal/state`
- `go test ./internal/publish`
- `go test ./internal/app ./internal/cli`
- `go test ./internal/config`
- `go test ./...`
- Consistency searches:
- `rg -n "ActionReplaceOlder|ActionReplaceNewer|ActionReplaceConflict|ActionReplaceTakeover|ActionSkipDestinationNewer" internal`
- `rg -n "StateModeSingleOwner|StateModeSharedRoot|SharedRootState|DistributorState|ParseSharedRoot|ReconciliationPolicy|TakeoverPolicy|TransferPolicy" internal --glob '!**/*_test.go'`
- `rg -n "Retained while the executor is migrated to catalog state" internal`
Completion criteria:
- Production publish execution has one catalog code path plus explicit
`force_replace`.
- Legacy config policy types are gone from production config structs/defaults.
- State package no longer exposes full v2/v3 implementation machinery unless it
is required by tests that generate superseded legacy fixtures.
- Full test suite passes without skipped tests that mask catalog cleanup work.
## Stage 5: Roadmap Closeout
Goal: leave `docs/roadmap/` in a clean state after the follow-up work is
implemented.
Implementation scope:
- Remove or rewrite completed roadmap material:
- this `implementation.md`;
- `docs/roadmap/catalog.md`, if catalog behavior is fully documented in
current docs;
- any future roadmap entries that still describe completed catalog cleanup as
pending work.
- Keep only genuinely future work in `docs/roadmap/future.md` or another active
roadmap file.
- Ensure future work remains out of current behavior docs.
Tests and checks:
- `go test ./...`
- `git status --short`
- `rg -n "future catalog|planned catalog|single_owner|shared_root|takeover|reconciliation|transfer" docs README.md examples`
- `rg -n "workflow: additive|workflow: replacement|schema_version.*4" docs README.md examples`
Completion criteria:
- Current docs, examples, and code describe the same implemented behavior.
- Roadmap docs contain only future work.
- Full test suite passes.
## Refactors To Avoid
- Do not reintroduce `state`, `reconciliation`, `takeover`, or `transfer` YAML
fields as deprecated aliases.
- Do not migrate legacy destination state into catalog state.
- Do not add top-level `owners` or `sources` to catalog state.
- Do not persist `workflow` in `.distributor.json`.
- Do not add new prune/reconcile selectors as part of this cleanup.
- Do not move catalog or force policy into storage adapters.
- Do not redesign the run JSON envelope while cleaning action labels.
## Open Questions
No open questions remain. This roadmap locks the follow-up decisions:
catalog `--force` will be implemented for exceptional destructive replacement,
current docs and examples will be synchronized to catalog schema version `4`,
catalog `skip_same` will be implemented as a metadata-only no-op optimization,
and legacy single-owner/shared-root implementation debt will be removed after
the catalog action vocabulary is complete.

View File

@@ -24,7 +24,9 @@ Reference: [Configuration](config.md#config-file-loading).
Symptom: `parse config ... field not found`. Symptom: `parse config ... field not found`.
Likely cause: the YAML contains a key that is not part of the implemented config schema. Legacy destination policy keys `state`, `reconciliation`, `takeover`, and `transfer` are rejected. Likely cause: the YAML contains a key that is not part of the implemented
config schema. Pre-workflow destination policy keys for state mode, conflict
handling, ownership adoption, or per-comparison copy decisions are rejected.
Diagnostic: Diagnostic: