Synchronize catalog documentation and examples
This commit is contained in:
@@ -96,18 +96,15 @@ The source manifest should remain minimal. Routing, destination selection, publi
|
||||
|
||||
Each destination bundle path is managed by `.distributor.json`. This file is both the destination sentinel and the destination state record.
|
||||
|
||||
Publish execution writes single-owner or shared-root destination state according to destination `state.mode`. In shared-root state, one `.distributor.json` records multiple pipeline/destination owners and every managed output carries its owner identity.
|
||||
Publish execution writes catalog destination state. One `.distributor.json` records all managed outputs under the destination bundle path, and each output carries its owning pipeline id and destination id.
|
||||
|
||||
Single-owner state records:
|
||||
Catalog state records:
|
||||
|
||||
- `distributor` state schema version;
|
||||
- pipeline id;
|
||||
- destination id;
|
||||
- publication timestamp;
|
||||
- state creation and update timestamps;
|
||||
- single-owner state mode;
|
||||
- reconciliation mode;
|
||||
- the normalized source manifest used for publication;
|
||||
- catalog state mode;
|
||||
- owner identity for each managed output;
|
||||
- compact source identity for each managed output;
|
||||
- metadata for copied source outputs;
|
||||
- metadata for generated outputs, such as HTML files;
|
||||
- optional URL metadata for published outputs;
|
||||
@@ -117,40 +114,23 @@ A representative destination state file is:
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 2,
|
||||
"schema_version": 4,
|
||||
"distributor_version": "0.1.0",
|
||||
"pipeline_id": "weather-daily",
|
||||
"destination_id": "static-html",
|
||||
"published_at": "2026-05-30T11:12:00Z",
|
||||
"created_at": "2026-05-30T11:12:00Z",
|
||||
"updated_at": "2026-05-30T11:12:00Z",
|
||||
"state": {
|
||||
"mode": "single_owner"
|
||||
},
|
||||
"reconciliation": {
|
||||
"mode": "replace"
|
||||
},
|
||||
"source": {
|
||||
"manifest": {
|
||||
"schema_version": 1,
|
||||
"id": "weather.daily.brentwood.2026-05-30",
|
||||
"digest": "sha256:...",
|
||||
"created": "2026-05-30T11:10:00Z",
|
||||
"files": [
|
||||
{
|
||||
"path": "report.md",
|
||||
"sha256": "sha256:...",
|
||||
"size": 12345
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"links": {
|
||||
"primary_url": "https://reports.example.com/weather-daily/"
|
||||
"mode": "catalog"
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"path": "index.html",
|
||||
"pipeline_id": "weather-daily",
|
||||
"destination_id": "static-html",
|
||||
"source": {
|
||||
"id": "weather.daily.brentwood.2026-05-30",
|
||||
"digest": "sha256:...",
|
||||
"created": "2026-05-30T11:10:00Z"
|
||||
},
|
||||
"kind": "generated",
|
||||
"source_path": "report.md",
|
||||
"transform": "markdown_to_html",
|
||||
@@ -167,13 +147,11 @@ A representative destination state file is:
|
||||
Destination comparison rules are based on `.distributor.json`:
|
||||
|
||||
- No `.distributor.json`: publish normally only if the destination bundle path is empty.
|
||||
- Existing state embeds the same normalized source manifest: skip as already published.
|
||||
- Existing state has the same source id and an older source `created`: replace, subject to destructive-operation safety rules.
|
||||
- Existing state has the same source id and a newer source `created`: skip by default, or replace when `transfer.on_destination_newer: replace` is configured.
|
||||
- Existing state has the same source id and same `created` but different digest: fail by default, or replace as valid managed conflict state when `transfer.on_conflict: replace` is configured.
|
||||
- Existing valid managed state has a different source id, pipeline id, destination id, or shared-root output owner: replace when destination `takeover.mode` permits that ownership transfer, or replace as valid managed conflict state when `transfer.on_conflict: replace` is configured; otherwise fail as a conflict.
|
||||
|
||||
For older destination state, destination `reconciliation.mode` controls output updates. `replace` rewrites the managed output set to match the new plan. `merge` retains omitted managed outputs, overwrites only existing managed paths, and rejects planned paths that collide with unmanaged storage content.
|
||||
- Existing catalog state with additive workflow: write planned outputs and retain unrelated managed outputs.
|
||||
- Existing catalog state with replacement workflow: write planned outputs and remove omitted outputs for the current pipeline and destination owner.
|
||||
- Planned paths that collide with unmanaged storage content fail by default.
|
||||
- Invalid or unsupported destination state fails by default.
|
||||
- Explicit forced replacement may clear the bounded destination bundle path after dry-run review.
|
||||
|
||||
## Publication and Transform Policy
|
||||
|
||||
@@ -264,8 +242,7 @@ Pipeline configuration should express:
|
||||
- per-destination transform policy;
|
||||
- per-destination public link policy;
|
||||
- validation behavior;
|
||||
- per-destination takeover behavior;
|
||||
- destination conflict/replacement behavior.
|
||||
- per-destination workflow and retention behavior.
|
||||
|
||||
## Modules and Registries
|
||||
|
||||
|
||||
@@ -18,7 +18,7 @@ Use it with `docs/policy/architecture.md` and `docs/policy/documentation.md`.
|
||||
- `internal/adapters/ssh`: SSH/SFTP backend.
|
||||
- `internal/adapters/s3`: S3-compatible object storage backend.
|
||||
- `internal/storage/fake`: in-memory backend for tests.
|
||||
- `internal/publish`: destination inspection, output planning, takeover planning, reconciliation, execution, managed cleanup, and explicit forced replacement.
|
||||
- `internal/publish`: destination inspection, output planning, catalog workflow planning, execution, managed cleanup, and explicit forced replacement.
|
||||
- `internal/transform`: transform interface and registry.
|
||||
- `internal/transform/markdown`: Markdown-to-HTML transform.
|
||||
- `internal/notify`: notification interface and current no-op notifier.
|
||||
|
||||
Reference in New Issue
Block a user