Add destination link metadata

This commit is contained in:
2026-06-01 21:42:00 +00:00
parent a8564035d3
commit 980ae15249
24 changed files with 737 additions and 16 deletions

View File

@@ -6,9 +6,9 @@
## Inputs and outputs
`Run` accepts a context, optional config path, dry-run flag, force flag, stdout writer, and optional notifier. It loads YAML config, discovers source bundles for each configured pipeline, plans each destination independently, optionally executes publish plans, writes summary output when stdout is supplied, and returns an aggregated error if any destination fails.
`Run` accepts a context, optional config path, dry-run flag, force flag, stdout writer, output format, and optional notifier. It loads YAML config, discovers source bundles for each configured pipeline, plans each destination independently, optionally executes publish plans, writes text or JSON output when stdout is supplied, and returns an aggregated error if any destination fails.
`Validate` and `Inspect` accept a local path. `Validate` discovers and validates bundles. `Inspect` writes bundle metadata and manifest file entries to stdout when provided.
`Validate` and `Inspect` accept either a local path or one configured pipeline source. `Validate` discovers and validates bundles. `Inspect` writes bundle metadata and manifest file entries to stdout when provided.
## Run flow
@@ -17,11 +17,12 @@ The runner:
1. loads config from the supplied path or `config.DefaultConfigPath`;
2. opens the configured source backend;
3. discovers validated bundles from the source root;
4. opens each destination backend independently;
5. builds a publish plan for each bundle and destination;
6. prints plan lines and records summary counters;
7. executes publish or replacement plans unless dry-run is enabled;
8. invokes the notifier after successful publish or replacement actions.
4. selects source bundles for each destination according to destination path mapping;
5. opens each destination backend independently;
6. builds publish plans for the selected bundle and destination combinations;
7. prints plan lines or JSON action records and records summary counters;
8. executes publish or replacement plans unless dry-run is enabled;
9. invokes the notifier after successful publish or replacement actions.
Destination failures are collected while later destinations continue to run. Source open and source discovery failures stop the run because there are no valid bundles to fan out.
@@ -47,7 +48,7 @@ Stdout write errors are returned immediately because the caller's requested outp
`internal/app` coordinates packages but does not own manifest validation rules, destination state comparison, storage path rules, output planning, transform rendering, or backend-specific filesystem behavior.
`Validate` and `Inspect` are local path commands. Remote execution wiring currently belongs to `Run`.
Configured-source `Validate` and `Inspect` share source backend construction with `Run` and do not open destinations.
## Tests

View File

@@ -26,6 +26,7 @@ Defaults are applied in `ApplyDefaults`:
- destination publish policy defaults to source output only;
- Markdown-to-HTML mode defaults to `sidecar` when a transform block is present and mode is omitted;
- destination path mapping defaults to `preserve_relative`;
- destination link primary policy defaults to `auto` when a `links` block is present;
- `transfer.on_destination_same` defaults to `skip`;
- `transfer.on_destination_older` defaults to `replace`;
- `transfer.on_destination_newer` defaults to `skip`;
@@ -33,7 +34,7 @@ Defaults are applied in `ApplyDefaults`:
## Validation responsibilities
Validation requires at least one pipeline, slug-like unique pipeline ids, one source per pipeline, at least one destination, slug-like unique destination ids within each pipeline, backend-specific required fields, valid validation policy, valid publish and transform combinations, valid destination path mapping mode, and valid transfer actions.
Validation requires at least one pipeline, slug-like unique pipeline ids, one source per pipeline, at least one destination, slug-like unique destination ids within each pipeline, backend-specific required fields, valid validation policy, valid publish and transform combinations, valid destination path mapping mode, valid destination link config, and valid transfer actions.
Transfer validation accepts `replace` for `on_destination_newer` and `on_conflict`, but publish planning honors those destructive actions only when the current run explicitly requests force.
@@ -41,6 +42,8 @@ Transfer validation accepts `replace` for `on_destination_newer` and `on_conflic
Destination path mapping accepts `preserve_relative` and `fixed`. The app layer applies the mapping when it selects destination bundle paths; config owns only YAML shape, defaulting, and validation.
Destination links are optional. When a `links` block is present, `base_url` is required, must use `http` or `https`, and must not include a query string or fragment. `primary` accepts `auto`, `html`, and `source`.
## Executable support boundary
Config validation accepts `local`, `ssh`, and `s3` backend shapes. Runtime execution opens all three through `internal/app`.

View File

@@ -6,9 +6,9 @@
## Inputs and outputs
Inputs are a source bundle, source backend, destination backend, pipeline id, destination id, publish policy, transform policy, transformer resolver, transfer policy, path mapping mode, destination bundle path, existing destination state, and whether explicit force was requested for the current run.
Inputs are a source bundle, source backend, destination backend, pipeline id, destination id, publish policy, transform policy, optional link policy, transformer resolver, transfer policy, path mapping mode, destination bundle path, existing destination state, and whether explicit force was requested for the current run.
Output is a plan with an action, reason, and selected source or generated outputs. Execution writes selected source files, generated files, and `.distributor.json` for publish or replacement actions.
Output is a plan with an action, reason, optional primary URL, and selected source or generated outputs. Execution writes selected source files, generated files, and `.distributor.json` for publish or replacement actions.
## Actions
@@ -28,6 +28,8 @@ The package uses `internal/state` for destination comparison, `internal/storage`
The app layer computes the destination bundle path before planning. `preserve_relative` destinations pass the source-root-relative bundle path. `fixed` destinations pass an empty destination bundle path, which means the destination backend root, and pass only the newest selected source bundle for that destination.
When link config is present, publish planning builds per-output URLs from `links.base_url`, the destination bundle path, and each output path. `index.html` outputs use directory-style URLs. The primary URL is selected from planned outputs according to the destination primary policy.
## Safety
Normal replacement deletes only outputs recorded in existing destination state plus `.distributor.json`. Forced replacement deletes the bounded destination bundle path before writing outputs and state. Failed writes trigger cleanup of outputs written during the failed attempt where practical.
@@ -40,6 +42,7 @@ Before changing publish behavior, inspect tests under `internal/publish` and run
- Publish planning is deterministic for the same source, destination state, policies, and transform outputs.
- Destination bundle paths are caller-supplied and are interpreted relative to the destination backend root.
- URL generation uses URL path semantics and does not infer public URLs from backend configuration.
- Normal replacement deletes only managed paths recorded in existing state plus `.distributor.json`.
- Forced replacement is explicit per run and deletes only within the destination bundle path.
- Publish execution writes destination state after selected outputs are written.

View File

@@ -19,13 +19,15 @@ Input is JSON destination state plus the current source manifest, pipeline id, d
- `source.manifest`
- `outputs`
`distributor_version` is optional diagnostic metadata. `published_at` parses as RFC3339 and distributor-written state serializes it as RFC3339 UTC.
`distributor_version` is optional diagnostic metadata. `links` is optional URL metadata. `published_at` parses as RFC3339 and distributor-written state serializes it as RFC3339 UTC.
The embedded `source.manifest` is validated with the same source manifest rules used by `internal/bundle`.
## Outputs
Each output records `path`, `kind`, `source_path`, `sha256`, and `size`. Supported output kinds are `source` and `generated`. Generated outputs require `transform`.
Each output records `path`, `kind`, `source_path`, `sha256`, and `size`. Supported output kinds are `source` and `generated`. Generated outputs require `transform`. Outputs may record `url` when the destination has link generation configured.
The optional top-level `links.primary_url` records the selected primary URL for the published destination bundle. It is omitted when link generation is not configured or when the destination primary policy has no matching output.
## Comparison
@@ -33,7 +35,7 @@ Comparison outcomes cover absent destination state, unmanaged destination conten
## Failure behavior
Invalid JSON, invalid state schema, invalid embedded source manifests, unsafe output paths, unsupported output kinds, missing generated-output transform names, and mismatched pipeline or destination ids produce comparison outcomes that publish planning can turn into fail actions. Supported identity and source-manifest conflicts can become forced replacement only when publish planning receives explicit force and compatible transfer policy.
Invalid JSON, invalid state schema, invalid embedded source manifests, unsafe output paths, invalid stored URLs, unsupported output kinds, missing generated-output transform names, and mismatched pipeline or destination ids produce comparison outcomes that publish planning can turn into fail actions. Supported identity and source-manifest conflicts can become forced replacement only when publish planning receives explicit force and compatible transfer policy.
## Boundaries
@@ -48,5 +50,6 @@ Before changing destination state behavior, inspect tests under `internal/state`
- `.distributor.json` is the destination sentinel and state record.
- Embedded source manifests use the same validation rules as source bundles.
- Generated outputs always record a transform id.
- Stored URLs are optional and must be absolute HTTP or HTTPS URLs when present.
- Comparison returns outcomes and reasons; it does not mutate storage.
- `distributor_version` is diagnostic metadata, not a comparison key.