Synchronize catalog documentation and examples

This commit is contained in:
2026-06-19 16:50:40 +00:00
parent 9a2eaf8e5e
commit 484fda2514
16 changed files with 296 additions and 871 deletions

View File

@@ -96,18 +96,15 @@ The source manifest should remain minimal. Routing, destination selection, publi
Each destination bundle path is managed by `.distributor.json`. This file is both the destination sentinel and the destination state record.
Publish execution writes single-owner or shared-root destination state according to destination `state.mode`. In shared-root state, one `.distributor.json` records multiple pipeline/destination owners and every managed output carries its owner identity.
Publish execution writes catalog destination state. One `.distributor.json` records all managed outputs under the destination bundle path, and each output carries its owning pipeline id and destination id.
Single-owner state records:
Catalog state records:
- `distributor` state schema version;
- pipeline id;
- destination id;
- publication timestamp;
- state creation and update timestamps;
- single-owner state mode;
- reconciliation mode;
- the normalized source manifest used for publication;
- catalog state mode;
- owner identity for each managed output;
- compact source identity for each managed output;
- metadata for copied source outputs;
- metadata for generated outputs, such as HTML files;
- optional URL metadata for published outputs;
@@ -117,40 +114,23 @@ A representative destination state file is:
```json
{
"schema_version": 2,
"schema_version": 4,
"distributor_version": "0.1.0",
"pipeline_id": "weather-daily",
"destination_id": "static-html",
"published_at": "2026-05-30T11:12:00Z",
"created_at": "2026-05-30T11:12:00Z",
"updated_at": "2026-05-30T11:12:00Z",
"state": {
"mode": "single_owner"
},
"reconciliation": {
"mode": "replace"
},
"source": {
"manifest": {
"schema_version": 1,
"id": "weather.daily.brentwood.2026-05-30",
"digest": "sha256:...",
"created": "2026-05-30T11:10:00Z",
"files": [
{
"path": "report.md",
"sha256": "sha256:...",
"size": 12345
}
]
}
},
"links": {
"primary_url": "https://reports.example.com/weather-daily/"
"mode": "catalog"
},
"outputs": [
{
"path": "index.html",
"pipeline_id": "weather-daily",
"destination_id": "static-html",
"source": {
"id": "weather.daily.brentwood.2026-05-30",
"digest": "sha256:...",
"created": "2026-05-30T11:10:00Z"
},
"kind": "generated",
"source_path": "report.md",
"transform": "markdown_to_html",
@@ -167,13 +147,11 @@ A representative destination state file is:
Destination comparison rules are based on `.distributor.json`:
- No `.distributor.json`: publish normally only if the destination bundle path is empty.
- Existing state embeds the same normalized source manifest: skip as already published.
- Existing state has the same source id and an older source `created`: replace, subject to destructive-operation safety rules.
- Existing state has the same source id and a newer source `created`: skip by default, or replace when `transfer.on_destination_newer: replace` is configured.
- Existing state has the same source id and same `created` but different digest: fail by default, or replace as valid managed conflict state when `transfer.on_conflict: replace` is configured.
- Existing valid managed state has a different source id, pipeline id, destination id, or shared-root output owner: replace when destination `takeover.mode` permits that ownership transfer, or replace as valid managed conflict state when `transfer.on_conflict: replace` is configured; otherwise fail as a conflict.
For older destination state, destination `reconciliation.mode` controls output updates. `replace` rewrites the managed output set to match the new plan. `merge` retains omitted managed outputs, overwrites only existing managed paths, and rejects planned paths that collide with unmanaged storage content.
- Existing catalog state with additive workflow: write planned outputs and retain unrelated managed outputs.
- Existing catalog state with replacement workflow: write planned outputs and remove omitted outputs for the current pipeline and destination owner.
- Planned paths that collide with unmanaged storage content fail by default.
- Invalid or unsupported destination state fails by default.
- Explicit forced replacement may clear the bounded destination bundle path after dry-run review.
## Publication and Transform Policy
@@ -264,8 +242,7 @@ Pipeline configuration should express:
- per-destination transform policy;
- per-destination public link policy;
- validation behavior;
- per-destination takeover behavior;
- destination conflict/replacement behavior.
- per-destination workflow and retention behavior.
## Modules and Registries

View File

@@ -18,7 +18,7 @@ Use it with `docs/policy/architecture.md` and `docs/policy/documentation.md`.
- `internal/adapters/ssh`: SSH/SFTP backend.
- `internal/adapters/s3`: S3-compatible object storage backend.
- `internal/storage/fake`: in-memory backend for tests.
- `internal/publish`: destination inspection, output planning, takeover planning, reconciliation, execution, managed cleanup, and explicit forced replacement.
- `internal/publish`: destination inspection, output planning, catalog workflow planning, execution, managed cleanup, and explicit forced replacement.
- `internal/transform`: transform interface and registry.
- `internal/transform/markdown`: Markdown-to-HTML transform.
- `internal/notify`: notification interface and current no-op notifier.