diff --git a/docs/config.md b/docs/config.md index 68a5791..f96f94d 100644 --- a/docs/config.md +++ b/docs/config.md @@ -270,7 +270,9 @@ destinations: Destination ids must be unique within a pipeline. -Legacy destination policy keys `state`, `reconciliation`, `takeover`, and `transfer` are not accepted config fields. YAML files containing those keys fail during config loading. +Pre-workflow destination policy keys for state mode, conflict handling, +ownership adoption, and per-comparison copy decisions are not accepted config +fields. YAML files containing those keys fail during config loading. ## Destination Workflow diff --git a/docs/policy/architecture.md b/docs/policy/architecture.md index b274a96..46390b8 100644 --- a/docs/policy/architecture.md +++ b/docs/policy/architecture.md @@ -178,7 +178,7 @@ Application logic must interact with storage through internal backend interfaces Adapters should be thin. Backend adapters should implement storage operations and translate backend-specific errors, but should not make bundle comparison, transform, routing, or replacement decisions. -Remote file-transfer support should prefer native protocol implementations over shelling out, unless a later design document records a reason to differ. +Remote file copy support should prefer native protocol implementations over shelling out, unless a later design document records a reason to differ. ## Dependency Policy @@ -272,7 +272,7 @@ Errors should be actionable and preserve context. Wrap errors with operation, pi Errors and logs must not expose secrets. -Use structured logging where practical. Logs should describe discovery, validation, planned actions, skipped transfers, conflicts, replacements, external calls, retries, and failure causes, but should not include large report contents by default. +Use structured logging where practical. Logs should describe discovery, validation, planned actions, skipped copies, conflicts, replacements, external calls, retries, and failure causes, but should not include large report contents by default. Skip and no-op decisions should be logged at an appropriate level so operators can distinguish successful publication from intentional no-op behavior. diff --git a/docs/roadmap/catalog.md b/docs/roadmap/catalog.md deleted file mode 100644 index e95d524..0000000 --- a/docs/roadmap/catalog.md +++ /dev/null @@ -1,283 +0,0 @@ -# Catalog State And Destination Workflow Roadmap - -This roadmap records the intended first-class state model and destination -workflow configuration for replacement and additive publication workflows. - -Current `single_owner` and `shared_root` state are oriented around comparing a -destination owner to one latest source manifest. That works well for replacement -workflows, where a producer maintains a curated source of truth and -`distributor` makes a destination match it. It is a poor fit for additive -workflows, where each run contributes new or updated output paths while -preserving unrelated managed outputs from previous runs and other pipelines. - -The target model is a current-state catalog: `.distributor.json` records the -currently managed output paths at a destination root, and each output record -stores its current owner, compact source identity, content digest, and update -metadata. It is not intended to be an audit log. - -Both replacement and additive workflows should use the same catalog state shape. -The workflow choice is runtime policy from destination config, not persisted -state. - -## Locked Decisions - -- Add one destination state mode named `catalog` for both replacement and - additive workflows. -- Catalog state writes `.distributor.json` schema version `4`. -- Add a destination-level `workflow` setting with accepted values - `replacement` and `additive`. -- Default `workflow` to `additive` because it is the least destructive workflow. -- `workflow` replaces the normal user-facing need to combine `state.mode`, - `reconciliation.mode`, `takeover.mode`, and transfer conflict settings for the - two primary workflows. -- `workflow: replacement` treats the current planned outputs as the authoritative - managed output set for the destination scope and removes previously managed - outputs in that scope when they are no longer planned. -- `workflow: additive` writes or overwrites the planned output paths and retains - unrelated managed outputs. -- If a planned path already exists as a managed output, the new publication may - overwrite it and becomes that path's current owner. -- If a planned path exists in storage but is not recorded in valid catalog state, - it remains unmanaged content and must not be adopted implicitly. -- If an output path changes owner, preserve the output record's existing - `created_at` and update only `updated_at`. -- Catalog state records current ownership only. It does not retain historical - owners, historical sources, or old versions of overwritten output records. -- `pipeline_id` and `destination_id` are stored separately on each output - record. They are not concatenated into one owner string. -- Catalog state does not include top-level `owners`; owners are derivable from - the output records. -- Catalog state does not include top-level `sources`; compact source identity is - stored directly on each output record. -- Catalog state does not record whether the last run used `replacement` or - `additive`. Workflow is execution policy, and persisting it would create - drift risk if config changes later. -- Legacy destination policy fields should be rejected outright in the new - workflow config model. Do not retain aliases for `state`, `reconciliation`, - `takeover`, or `transfer`. - -## Destination Workflow Semantics - -The user-facing destination config should express intent directly: - -```yaml -destinations: - - id: weather-latest - backend: local - path: /srv/reports/weather/latest - workflow: additive -``` - -```yaml -destinations: - - id: weather-archive - backend: local - path: /srv/reports/weather/archive - workflow: replacement -``` - -`workflow` is orthogonal to backend config, path mapping, publish source/HTML -selection, transforms, links, and retention. - -### Replacement Workflow - -A replacement destination is for producers that maintain a curated source of -truth and expect `distributor` to make the destination's managed scope match the -current publication. - -For a destination configured with `workflow: replacement`: - -- planned outputs are written to their resolved destination paths; -- planned outputs may overwrite existing managed outputs at the same path; -- each written output record is replaced in-place with the current - publication's owner, source identity, hash, size, and timestamps; -- managed outputs in the destination scope that are not present in the current - plan are deleted and removed from catalog state; -- unmanaged destination content remains unmanaged and blocks planned path - collisions unless an explicit force workflow later chooses otherwise; -- the catalog state shape remains the same as additive workflow state. - -### Additive Workflow - -An additive destination is for producers that routinely contribute outputs to a -shared destination root. - -For a destination configured with `workflow: additive`: - -- planned outputs are written to their resolved destination paths; -- planned outputs may overwrite existing managed outputs at the same path; -- each overwritten output record is replaced in-place with the current - publication's owner, source identity, hash, size, and timestamps; -- managed outputs not present in the current plan are retained; -- unrelated managed outputs from other pipelines or destinations are retained; -- unmanaged destination content remains unmanaged and blocks planned path - collisions unless an explicit force workflow later chooses otherwise; -- pruning can select retained managed outputs by `updated_at` and, when useful, - by `pipeline_id` and/or `destination_id`. - -Replacement workflow treats the current publication as the desired managed -output set for a destination scope. Additive workflow treats the current -publication as a patch to the catalog of currently managed outputs. - -## Destination State Schema Version 4 - -Catalog state should use this top-level shape: - -```json -{ - "schema_version": 4, - "distributor_version": "dev", - "created_at": "2026-06-19T12:00:00Z", - "updated_at": "2026-06-19T12:05:00Z", - "state": { - "mode": "catalog" - }, - "outputs": [] -} -``` - -Top-level fields: - -- `schema_version`: required. Value `4` for catalog state. -- `distributor_version`: optional diagnostic version string. -- `created_at`: required RFC3339 timestamp for when the catalog state file was - first created. -- `updated_at`: required RFC3339 timestamp for the latest catalog state update. -- `state.mode`: required. Value `catalog`. -- `outputs`: required array of currently managed output records. - -Catalog state should not include top-level `pipeline_id`, `destination_id`, -`published_at`, `workflow`, `owners`, `sources`, or a full source manifest. -Those concepts belong on output records or in configuration. - -## Output Record Schema - -Each output record should be self-contained enough to support current -ownership, pruning, repair, bitrot checks, and basic provenance without a -separate owner or source catalog. - -```json -{ - "path": "tomorrow/index.html", - "pipeline_id": "weatherreporter.daily", - "destination_id": "latest-html", - "source": { - "id": "weatherreporter.tomorrow", - "digest": "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef", - "created": "2026-06-19T12:00:00Z" - }, - "kind": "generated", - "source_path": "report.md", - "transform": "markdown_to_html", - "sha256": "sha256:abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789", - "size": 12345, - "created_at": "2026-06-19T12:00:00Z", - "updated_at": "2026-06-19T12:05:00Z", - "url": "https://reports.example.com/weather/tomorrow/" -} -``` - -Required output fields: - -- `path`: destination-relative managed output path. -- `pipeline_id`: pipeline that most recently wrote this output path. -- `destination_id`: destination that most recently wrote this output path. -- `source`: compact identity of the source bundle that produced this output. -- `source.id`: source manifest id. -- `source.digest`: source manifest digest. -- `source.created`: source manifest creation timestamp, RFC3339. -- `kind`: `source` or `generated`. -- `sha256`: digest of the output bytes. -- `size`: output byte size. -- `created_at`: RFC3339 timestamp for when this output path first became - managed in catalog state. -- `updated_at`: RFC3339 timestamp for when this output path was most recently - written or updated. - -Optional output fields: - -- `source_path`: present only for generated outputs; source manifest path used - to produce the generated output. -- `transform`: present only for generated outputs; transform name used to - produce the generated output. -- `url`: present only when destination link configuration produces a public URL - for this output. - -For copied source outputs, `source_path` and `transform` should be omitted. The -output `path` is already the copied source artifact path. - -## Useful Properties - -The catalog shape supports both primary workflows without unnecessary -normalization: - -- current owner of each path is explicit; -- current source identity for each path is explicit; -- bitrot checks can compare storage bytes to `sha256`; -- pruning can use `updated_at`, optionally scoped by `pipeline_id` and - `destination_id`; -- repair can remove missing managed output records without consulting an owner - or source table; -- overwriting an output path updates one output record in place; -- no orphaned top-level owner/source records need to be maintained. - -The schema intentionally avoids storing a full source manifest for every output. -The source manifest remains the producer-to-distributor validation contract, but -catalog state only needs compact source identity for currently managed outputs. - -## Relationship To Existing State And Config Modes - -Existing state modes remain current behavior until catalog mode is implemented: - -- `single_owner` schema version `2` supports one owner for a destination bundle - path. -- `shared_root` schema version `3` supports multiple owners in one destination - root but still keeps owner records with latest source manifests. -- `catalog` schema version `4` should support replacement and additive - current-state ownership without top-level owner or source catalogs. - -The intended user-facing config should move toward `workflow: replacement` and -`workflow: additive` instead of requiring ordinary users to combine -`state.mode`, `reconciliation.mode`, `takeover.mode`, and transfer conflict -settings. - -Because the project is still alpha pre-release, catalog implementation should be -a clean break: - -- remove legacy write paths for current `single_owner` and `shared_root` state; -- do not implement migration from schema versions `1`, `2`, or `3`; -- do not preserve backwards compatibility for legacy destination state; -- when a configured destination writes successfully, write schema version `4` - catalog state; -- if an existing `.distributor.json` has schema version lower than `4`, treat it - as superseded legacy state for planning purposes and overwrite it according to - the configured workflow, without attempting conversion. - -For the first catalog run against superseded legacy state: - -- `workflow: replacement` may clear the bounded destination root before writing - the planned outputs and schema version `4` catalog state. -- `workflow: additive` may overwrite only the planned output paths, then write - schema version `4` catalog state containing those planned outputs. -- unplanned files left behind by an additive run against superseded legacy state - are not recorded in catalog state and are treated as unmanaged content by - later catalog runs. - -The target direction is that both primary workflows use catalog state. - -## Prune And Reconcile-State Scope - -Catalog maintenance commands should initially keep the existing ownership -selector model: - -- `prune --pipeline --destination ` operates only on outputs currently - owned by that pipeline/destination. -- `reconcile-state --pipeline --destination ` repairs only outputs - currently owned by that pipeline/destination. -- `reconcile-state --all-owners` repairs all output records in the catalog. -- `prune` remains scoped to the selected pipeline/destination owner only in the - initial catalog implementation. - -Do not add catalog-specific selectors in the initial implementation, such as -`--source-id`, `--path-prefix`, or `--kind`. Those may be useful later, but they -are not required to make additive workflow first-class. diff --git a/docs/roadmap/future.md b/docs/roadmap/future.md index 2fa4044..819ff44 100644 --- a/docs/roadmap/future.md +++ b/docs/roadmap/future.md @@ -93,10 +93,10 @@ internal docs. - Current upload status, queue, and idempotency state are memory-only. - Producers submit complete tar or gzip-compressed tar source bundles today. - Producers do not choose destination ids, destination paths, transforms, links, - publish policy, transfer policy, storage backends, reconciliation mode, state - mode, or retention policy through upload requests. + publish policy, destination workflow, storage backends, or retention policy + through upload requests. - Source manifests remain free of routing, destination, transform, credential, - reconciliation, state, and retention data. + workflow, state, and retention data. - Public access policy, TLS termination, and rate limiting belong outside `distributor` unless a future implementation changes that boundary. - `distributor` is not a broad storage synchronization tool unless a future diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md deleted file mode 100644 index cc663eb..0000000 --- a/docs/roadmap/implementation.md +++ /dev/null @@ -1,399 +0,0 @@ -# Catalog Follow-Up Implementation Roadmap - -This is the active staged plan for closing the remaining catalog-state -implementation gaps found after the schema version `4` catalog workflow landed. -It supersedes the earlier catalog implementation plan. - -The current code already supports destination `workflow: additive` and -`workflow: replacement`, writes catalog state schema version `4`, rejects legacy -destination policy YAML fields, and routes publish execution through catalog -planning for normal runs. The remaining work is to make the implemented behavior -release-ready: restore a meaningful catalog `--force` path, synchronize current -docs and examples, and remove legacy single-owner/shared-root implementation -debt. - -## Current Baseline - -- Destination config accepts `workflow: additive` and `workflow: replacement`. -- Omitted `workflow` defaults to `additive`. -- Destination YAML fields `state`, `reconciliation`, `takeover`, and `transfer` - are rejected by strict YAML decoding. -- Successful publish writes `.distributor.json` schema version `4` with - `state.mode: catalog`. -- Schema versions `1`, `2`, and `3` are treated as superseded legacy state for - publish planning. -- `run --force` remains in the CLI and docs, but catalog publish planning does - not currently select `force_replace`. -- Current docs and examples still describe legacy config/state behavior in - several places. -- Legacy single-owner and shared-root state types, tests, and execution helpers - remain in the codebase even though normal publish planning no longer uses - them. - -## Implementation Principles - -- Preserve the catalog state shape defined in `docs/roadmap/catalog.md`. -- Keep `workflow` as runtime config policy; do not persist it in - `.distributor.json`. -- Keep unmanaged content protected by default. -- Keep `--force` explicit, per-run, dry-runnable, and bounded to the configured - destination bundle path. -- Do not reintroduce legacy config aliases or compatibility migration. -- Keep backend adapters unaware of catalog, workflow, and force policy. -- Update current-behavior docs in the same stage that makes the described - behavior true. - -## Active Implementation Stages - -## Stage 1: Catalog Force Planning And Execution - -Goal: make `run --force` meaningful for catalog workflows while keeping normal -catalog safety conservative. - -Implementation scope: - -- Update `internal/publish` planning so `req.Force` can select - `force_replace` for exceptional destructive catalog cases. -- Force should apply to these cases: - - no valid `.distributor.json` and destination bundle path has unmanaged - content; - - planned path collision with storage content not recorded in valid catalog - state; - - invalid destination state JSON or invalid destination state fields; - - unsupported future destination state schema, if the operator explicitly - chooses force replacement after dry-run review. -- Force should not be needed for ordinary valid catalog-managed upserts or - replacements. Additive and replacement workflow behavior remains normal managed - behavior. -- `force_replace` must delete only the bounded destination bundle path through - the storage abstraction, then write planned outputs and schema version `4` - catalog state. -- For fixed-path destinations, the bounded destination bundle path is the - configured backend root. Dry-run output must make that clear. -- Preserve default behavior without `--force`: - - unmanaged planned path collisions fail as `fail_unmanaged`; - - invalid or future state fails as `fail_conflict`; - - no-state non-empty destinations fail as unmanaged. -- Ensure `force_replace` state creation uses the same catalog output projection - as normal publish planning. -- Keep `Force` out of config. There must be no persistent force default. -- Remove or update skipped force tests that were disabled during catalog - implementation. - -Current-behavior documentation updates: - -- Update only the force-related sections of: - - `docs/cli.md`; - - `docs/operations.md`; - - `docs/troubleshooting.md`; - - `docs/internal/publish.md`; - - backend integration docs where they describe forced deletion boundaries. -- Document force as an exceptional catalog recovery/replacement workflow, not as - a normal way to handle valid managed catalog state. - -Tests: - -- `go test ./internal/publish` -- `go test ./internal/app ./internal/cli` -- `go test ./internal/adapters/local ./internal/storage ./internal/storage/fake` -- Planning tests: - - no-state non-empty destination returns `fail_unmanaged` without force and - `force_replace` with force; - - unmanaged planned path collision returns `fail_unmanaged` without force and - `force_replace` with force; - - invalid state returns `fail_conflict` without force and `force_replace` with - force; - - future schema returns `fail_conflict` without force and `force_replace` with - force; - - valid catalog additive and replacement actions do not become - `force_replace`. -- Execution tests: - - `force_replace` clears only the destination bundle path; - - sibling paths outside the bundle path survive for local, fake S3, and fake - SSH-style backends; - - fixed-path force clears the configured backend root and is reported as such; - - resulting state is schema version `4` catalog state. -- CLI/app tests: - - dry-run `--force` reports `force_replace` without writing; - - normal `--force` executes and increments only the `force_replace` counter; - - JSON output includes stable `force_replace` action and summary fields. - -Completion criteria: - -- `run --force` has observable, tested catalog behavior. -- Force is still unnecessary for normal managed catalog replacement/upsert. -- Unmanaged/invalid/future-state replacement remains impossible without explicit - `--force`. - -## Stage 2: Current Documentation And Example Synchronization - -Goal: make all current-behavior docs and copyable examples match catalog schema -version `4` and the `workflow` config model. - -Implementation scope: - -- Rewrite `docs/config.md` so destination behavior is described through: - - `workflow: additive`; - - `workflow: replacement`; - - `publish`; - - `transform`; - - `path_mapping`; - - `links`; - - `retention`. -- Remove current-behavior documentation for destination config fields: - - `state`; - - `reconciliation`; - - `takeover`; - - `transfer`. -- Rewrite `docs/integrations/destination-state.md` around schema version `4` - catalog state: - - top-level catalog fields; - - output record fields; - - source identity fields; - - generated-output-only `source_path` and `transform`; - - optional per-output `url`; - - superseded legacy schema handling for publish planning. -- Update `docs/operations.md`: - - additive workflow semantics; - - replacement workflow semantics; - - catalog `force_replace`; - - prune and reconcile-state catalog behavior; - - dry-run action labels. -- Update `docs/troubleshooting.md`: - - replace legacy conflict guidance with workflow/catalog guidance; - - describe rejected legacy config fields; - - describe force-only exceptional cases. -- Update `docs/cli.md`: - - run summary counters now use `publish_new`, `upsert_additive`, - `replace_catalog`, `skip_same`, `force_replace`, `fail_unmanaged`, and - `fail_conflict`; - - remove legacy action labels from current command reference. -- Update relevant internal docs: - - `docs/internal/app.md`; - - `docs/internal/config.md`; - - `docs/internal/publish.md`; - - `docs/internal/state.md`. -- Update policy docs where current architectural text still describes - single-owner/shared-root behavior as current: - - `docs/policy/architecture.md`; - - `docs/policy/development.md`. -- Update examples: - - remove or rewrite `examples/shared-root.yml`; - - remove or rewrite `examples/merge-reconciliation.yml`; - - ensure examples use `workflow` where workflow intent matters; - - keep examples valid, secret-free, and copyable. -- Do not document any future catalog selectors, state migration, or compatibility - aliases outside `docs/roadmap/`. - -Tests and checks: - -- `go test ./internal/config ./internal/cli` -- `go test ./...` -- Run the config/example checks already used by the test suite. -- Manual smoke checks: - - `go run ./cmd/distributor run --config examples/local-publish.yml --dry-run` - - `go run ./cmd/distributor run --config examples/archive-and-latest.yml --dry-run` - - smoke any rewritten replacement/additive examples. -- Consistency searches: - - `rg -n "state:|reconciliation:|takeover:|transfer:" examples docs --glob '!docs/roadmap/**'` - - `rg -n "single_owner|shared_root|schema version \`2\`|schema version \`3\`" docs --glob '!docs/roadmap/**'` - - `rg -n "replace_older|replace_newer|replace_conflict|replace_takeover|takeover_mode" docs README.md examples --glob '!docs/roadmap/**'` - - `rg -n "workflow: additive|workflow: replacement|schema_version.*4" docs examples` - -Completion criteria: - -- Current docs describe implemented catalog behavior only. -- Copyable examples load successfully. -- No current doc tells users to configure rejected legacy fields. -- Destination state documentation is schema version `4` first. - -## Stage 3: Catalog Idempotent Skip - -Goal: make `skip_same` a real catalog no-op optimization instead of a stale -legacy action label. - -Implementation scope: - -- Implement `skip_same` when every planned output is already catalog-managed - with matching: - - pipeline id; - - destination id; - - source id; - - source digest; - - source created timestamp; - - output path; - - output kind; - - output digest; - - output size; - - generated output `source_path`; - - generated output `transform`; - - output URL metadata. -- Do not read destination bytes for this optimization. Trust valid catalog - metadata; bitrot detection remains a separate future concern. -- For `workflow: additive`, allow `skip_same` when all planned outputs match and - no planned output needs to be written. Unrelated catalog outputs are ignored - for the skip decision and remain retained. -- For `workflow: replacement`, allow `skip_same` only when all planned outputs - match and there are no current-owner catalog outputs that replacement workflow - would delete. -- Never return `skip_same` for superseded legacy state, invalid state, unmanaged - content, future state, or force replacement. -- `skip_same` must not write outputs, rewrite state, delete files, or notify. -- Count `skip_same` in text and JSON summaries. -- Keep behavior deterministic and identical across local, SSH, and S3 - destinations. - -Current-behavior documentation updates: - -- Update `docs/cli.md`, `docs/operations.md`, and `docs/internal/publish.md` to - describe catalog `skip_same`. -- Document that `skip_same` is metadata-based and does not validate destination - bytes. - -Tests: - -- `go test ./internal/publish ./internal/app ./internal/cli` -- Repeated additive publish with identical catalog metadata returns - `skip_same`. -- Repeated replacement publish with identical catalog metadata and no omitted - current-owner outputs returns `skip_same`. -- Changed source digest, output digest, URL, transform mode, owner, output size, - or generated source path prevents skip. -- Replacement workflow does not skip when it would delete omitted current-owner - outputs. -- `skip_same` does not write outputs or state and does not notify. -- `go test ./...` - -Completion criteria: - -- Catalog action vocabulary has no stale action label. -- Repeated publish behavior is intentional, documented, and tested. - -## Stage 4: Legacy Publish And State Code Removal - -Goal: remove dead or near-dead single-owner/shared-root write and planning code -after catalog force behavior, docs, and skip semantics are settled. - -Implementation scope: - -- Remove legacy publish actions that are no longer planned or documented: - - `replace_older`; - - `replace_newer`; - - `replace_conflict`; - - `replace_takeover`; - - `skip_destination_newer`. -- Keep only catalog-era actions: - - `publish_new`; - - `upsert_additive`; - - `replace_catalog`; - - `skip_same`; - - `force_replace`; - - `fail_unmanaged`; - - `fail_conflict`. -- Remove legacy fields from `publish.Plan`: - - `StateMode`; - - `Reconciliation`; - - `TakeoverMode`; - - `ExistingState`; - - `ExistingSharedRoot`; - - shared-root owner/output carry fields retained only for old execution. -- Remove single-owner and shared-root execution branches from - `internal/publish/execute.go`. -- Remove unused single-owner/shared-root helper functions from `internal/publish` - once no tests or callers use them. -- Remove legacy destination config internals from `internal/config`: - - `StatePolicy`; - - `ReconciliationPolicy`; - - `TakeoverPolicy`; - - `TransferPolicy`; - - legacy constants and defaults that are no longer referenced. -- In `internal/state`, keep only what is needed for: - - parsing schema version `4` catalog state; - - identifying schema versions `1`, `2`, and `3` as superseded legacy by - schema number; - - validating and writing catalog state; - - pruning and reconciling catalog outputs. -- Delete or rewrite tests that assert legacy state parsing, validation, - comparison, shared-root ownership, or single-owner output behavior. -- Preserve test fixtures only where they are used to create superseded legacy - state for first catalog-run behavior. Prefer small local helpers over keeping - broad legacy state builders. -- Remove stale skipped tests that only represent old behavior. If a skipped test - still represents current expected behavior, unskip and update it. - -Current-behavior documentation updates: - -- No new user docs should be needed if Stage 2 is complete. -- Update internal docs only if removal changes internal package contracts beyond - what Stage 2 already documented. - -Tests: - -- `go test ./internal/state` -- `go test ./internal/publish` -- `go test ./internal/app ./internal/cli` -- `go test ./internal/config` -- `go test ./...` -- Consistency searches: - - `rg -n "ActionReplaceOlder|ActionReplaceNewer|ActionReplaceConflict|ActionReplaceTakeover|ActionSkipDestinationNewer" internal` - - `rg -n "StateModeSingleOwner|StateModeSharedRoot|SharedRootState|DistributorState|ParseSharedRoot|ReconciliationPolicy|TakeoverPolicy|TransferPolicy" internal --glob '!**/*_test.go'` - - `rg -n "Retained while the executor is migrated to catalog state" internal` - -Completion criteria: - -- Production publish execution has one catalog code path plus explicit - `force_replace`. -- Legacy config policy types are gone from production config structs/defaults. -- State package no longer exposes full v2/v3 implementation machinery unless it - is required by tests that generate superseded legacy fixtures. -- Full test suite passes without skipped tests that mask catalog cleanup work. - -## Stage 5: Roadmap Closeout - -Goal: leave `docs/roadmap/` in a clean state after the follow-up work is -implemented. - -Implementation scope: - -- Remove or rewrite completed roadmap material: - - this `implementation.md`; - - `docs/roadmap/catalog.md`, if catalog behavior is fully documented in - current docs; - - any future roadmap entries that still describe completed catalog cleanup as - pending work. -- Keep only genuinely future work in `docs/roadmap/future.md` or another active - roadmap file. -- Ensure future work remains out of current behavior docs. - -Tests and checks: - -- `go test ./...` -- `git status --short` -- `rg -n "future catalog|planned catalog|single_owner|shared_root|takeover|reconciliation|transfer" docs README.md examples` -- `rg -n "workflow: additive|workflow: replacement|schema_version.*4" docs README.md examples` - -Completion criteria: - -- Current docs, examples, and code describe the same implemented behavior. -- Roadmap docs contain only future work. -- Full test suite passes. - -## Refactors To Avoid - -- Do not reintroduce `state`, `reconciliation`, `takeover`, or `transfer` YAML - fields as deprecated aliases. -- Do not migrate legacy destination state into catalog state. -- Do not add top-level `owners` or `sources` to catalog state. -- Do not persist `workflow` in `.distributor.json`. -- Do not add new prune/reconcile selectors as part of this cleanup. -- Do not move catalog or force policy into storage adapters. -- Do not redesign the run JSON envelope while cleaning action labels. - -## Open Questions - -No open questions remain. This roadmap locks the follow-up decisions: -catalog `--force` will be implemented for exceptional destructive replacement, -current docs and examples will be synchronized to catalog schema version `4`, -catalog `skip_same` will be implemented as a metadata-only no-op optimization, -and legacy single-owner/shared-root implementation debt will be removed after -the catalog action vocabulary is complete. diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index ed30998..53742e0 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -24,7 +24,9 @@ Reference: [Configuration](config.md#config-file-loading). Symptom: `parse config ... field not found`. -Likely cause: the YAML contains a key that is not part of the implemented config schema. Legacy destination policy keys `state`, `reconciliation`, `takeover`, and `transfer` are rejected. +Likely cause: the YAML contains a key that is not part of the implemented +config schema. Pre-workflow destination policy keys for state mode, conflict +handling, ownership adoption, or per-comparison copy decisions are rejected. Diagnostic: