Add Markdown index HTML output mode
This commit is contained in:
@@ -54,7 +54,7 @@ pipelines:
|
||||
|
||||
## HTML Publication
|
||||
|
||||
To publish generated HTML from Markdown files:
|
||||
To publish generated sidecar HTML from Markdown files:
|
||||
|
||||
```yaml
|
||||
publish:
|
||||
@@ -68,6 +68,21 @@ transform:
|
||||
|
||||
Sidecar generation writes `report.html` for `report.md`. It does not mutate the source bundle.
|
||||
|
||||
To publish a single Markdown file as `index.html`:
|
||||
|
||||
```yaml
|
||||
publish:
|
||||
source: false
|
||||
html: true
|
||||
transform:
|
||||
markdown_to_html:
|
||||
enabled: true
|
||||
mode: index
|
||||
input: report.md
|
||||
```
|
||||
|
||||
When `mode: index` omits `input`, the source manifest must list exactly one Markdown file.
|
||||
|
||||
## Reference
|
||||
|
||||
Top level:
|
||||
@@ -164,7 +179,15 @@ Publish policy:
|
||||
- `publish.source`: publish source artifacts.
|
||||
- `publish.html`: publish generated HTML artifacts from Markdown source files.
|
||||
|
||||
At least one output type must be enabled. When `publish.html` is true, `transform.markdown_to_html.enabled` must be `true` and `transform.markdown_to_html.mode` must be `sidecar`.
|
||||
At least one output type must be enabled. When `publish.html` is true, `transform.markdown_to_html.enabled` must be `true`.
|
||||
|
||||
Markdown-to-HTML transform:
|
||||
|
||||
- `transform.markdown_to_html.enabled`: enables Markdown-to-HTML generation for destinations with `publish.html: true`.
|
||||
- `transform.markdown_to_html.mode`: optional; defaults to `sidecar`. Accepted values are `sidecar` and `index`.
|
||||
- `transform.markdown_to_html.input`: optional source manifest path for `index` mode. It must identify a listed Markdown file.
|
||||
|
||||
`sidecar` mode renders each manifest-listed `.md` file to a same-directory `.html` output. `index` mode renders one selected Markdown file to `index.html` at the destination bundle path. Enabled Markdown-to-HTML config is rejected when `publish.html` is false, and `input` is valid only with `mode: index`.
|
||||
|
||||
Transfer policy:
|
||||
|
||||
@@ -184,6 +207,7 @@ Defaults are applied after YAML decoding and before validation:
|
||||
- SSH `host_key_policy: accept-new`
|
||||
- S3 `region: us-east-1`
|
||||
- S3 `force_path_style: true`
|
||||
- `transform.markdown_to_html.mode: sidecar` when a Markdown-to-HTML transform block is present and mode is omitted
|
||||
- `publish.source: true`
|
||||
- `publish.html: false`
|
||||
- `transfer.on_destination_same: skip`
|
||||
@@ -216,6 +240,7 @@ Maintained examples live under [examples](../examples/):
|
||||
- `local-to-local.yml`: minimal local config.
|
||||
- `local-publish.yml`: runnable local source publication.
|
||||
- `local-html.yml`: runnable local HTML publication.
|
||||
- `local-index.yml`: runnable local `index.html` publication.
|
||||
- `fan-out.yml`: runnable local fan-out publication to source and HTML destinations.
|
||||
- `ssh-destination.yml`: environment-gated local-to-SSH publication example.
|
||||
- `s3-destination.yml`: environment-gated local-to-S3 publication example.
|
||||
|
||||
@@ -12,7 +12,12 @@ Rendering uses `github.com/yuin/goldmark`. The exact dependency version is pinne
|
||||
|
||||
`internal/transform/markdown.New` constructs the renderer with `goldmark.New()` and no project-specific extensions or renderer options.
|
||||
|
||||
For each source bundle file ending in `.md`, the transform reads the Markdown source and generates an HTML sidecar in the same logical directory. The output path replaces the `.md` suffix with `.html`, so `report.md` produces `report.html`. Non-Markdown source files produce no Markdown outputs.
|
||||
The transform supports two output modes:
|
||||
|
||||
- `sidecar`: reads each source bundle file ending in `.md` and generates an HTML sidecar in the same logical directory. The output path replaces the `.md` suffix with `.html`, so `report.md` produces `report.html`. Non-Markdown source files produce no Markdown outputs.
|
||||
- `index`: renders one selected Markdown source to `index.html` at the destination bundle path.
|
||||
|
||||
In `index` mode, `transform.markdown_to_html.input` can name the source manifest path to render. If `input` is omitted, the manifest must list exactly one Markdown file. The selected input must be a safe relative source path, must be listed in the source manifest, and must end in `.md`.
|
||||
|
||||
Raw HTML embedded in Markdown is not passed through by the current renderer behavior. Tests allow Goldmark's disabled-or-escaped raw HTML output forms and reject literal script tags in generated HTML.
|
||||
|
||||
@@ -42,7 +47,7 @@ Generated outputs record:
|
||||
|
||||
Markdown rendering does not mutate source bundles, publish files, write `.distributor.json`, select outputs, or choose transfer actions. Publish planning decides whether generated HTML is selected for a destination.
|
||||
|
||||
Only sidecar output mode is supported for current behavior.
|
||||
Publish planning chooses the configured mode and input for each destination. Markdown rendering does not inspect destinations, publish files, write `.distributor.json`, or choose transfer actions.
|
||||
|
||||
## Tests
|
||||
|
||||
@@ -52,4 +57,4 @@ Before changing Markdown renderer behavior, inspect and run:
|
||||
go test ./internal/transform/markdown
|
||||
```
|
||||
|
||||
The tests cover sidecar naming, ignored non-Markdown files, raw HTML handling, deterministic output, digest metadata, and size metadata.
|
||||
The tests cover sidecar naming, index input selection, ignored non-Markdown files, raw HTML handling, deterministic output, digest metadata, and size metadata.
|
||||
|
||||
@@ -24,6 +24,7 @@ Defaults are applied in `ApplyDefaults`:
|
||||
- SSH backend `port` defaults to `22`;
|
||||
- SSH backend `host_key_policy` defaults to `accept-new`;
|
||||
- destination publish policy defaults to source output only;
|
||||
- Markdown-to-HTML mode defaults to `sidecar` when a transform block is present and mode is omitted;
|
||||
- `transfer.on_destination_same` defaults to `skip`;
|
||||
- `transfer.on_destination_older` defaults to `replace`;
|
||||
- `transfer.on_destination_newer` defaults to `skip`;
|
||||
@@ -35,7 +36,7 @@ Validation requires at least one pipeline, slug-like unique pipeline ids, one so
|
||||
|
||||
Transfer validation accepts `replace` for `on_destination_newer` and `on_conflict`, but publish planning honors those destructive actions only when the current run explicitly requests force.
|
||||
|
||||
`ValidatePublishTransformPolicy` is shared with publish planning so destination policy combinations are checked consistently. Publishing HTML requires an enabled Markdown-to-HTML transform in `sidecar` mode. A publish policy must select source output, HTML output, or both.
|
||||
`ValidatePublishTransformPolicy` is shared with publish planning so destination policy combinations are checked consistently. Publishing HTML requires an enabled Markdown-to-HTML transform in `sidecar` or `index` mode. Enabled Markdown-to-HTML config is rejected when `publish.html` is false. `input` is accepted only for enabled `index` mode. A publish policy must select source output, HTML output, or both.
|
||||
|
||||
## Executable support boundary
|
||||
|
||||
|
||||
@@ -22,7 +22,7 @@ Execution fails if a write, delete, state serialization, or context check fails.
|
||||
|
||||
## Boundaries
|
||||
|
||||
The current implementation publishes source files and Markdown-to-HTML sidecar outputs. Backend behavior is supplied through `internal/storage`; app runtime currently supplies local, SSH, and S3 backends.
|
||||
The current implementation publishes source files and Markdown-to-HTML outputs. Markdown sidecar mode writes same-directory `.html` outputs, and Markdown index mode writes `index.html`. Backend behavior is supplied through `internal/storage`; app runtime currently supplies local, SSH, and S3 backends.
|
||||
|
||||
The package uses `internal/state` for destination comparison, `internal/storage` for IO, and the shared `internal/config` publish/transform policy helper for request validation. It resolves transforms through a narrow resolver supplied by the caller; concrete transform registration is owned by the app layer. It does not parse CLI flags or load config files.
|
||||
|
||||
|
||||
@@ -2,11 +2,11 @@
|
||||
|
||||
## Purpose
|
||||
|
||||
`internal/transform` defines generated publication artifacts. `internal/transform/markdown` implements Markdown-to-HTML sidecar generation.
|
||||
`internal/transform` defines generated publication artifacts. `internal/transform/markdown` implements Markdown-to-HTML generation.
|
||||
|
||||
## Inputs and outputs
|
||||
|
||||
Inputs are a validated source bundle and source backend. Outputs include destination path, source path, transform id, generated bytes, SHA-256, and size.
|
||||
Inputs are a validated source bundle, source backend, and transform options supplied by publish planning. Outputs include destination path, source path, transform id, generated bytes, SHA-256, and size.
|
||||
|
||||
## Registry
|
||||
|
||||
@@ -14,7 +14,7 @@ Inputs are a validated source bundle and source backend. Outputs include destina
|
||||
|
||||
## Markdown behavior
|
||||
|
||||
Markdown files ending in `.md` generate `.html` files in the same logical directory. Non-Markdown files do not generate outputs. Raw HTML embedded in Markdown is not passed through by the renderer.
|
||||
Markdown sidecar mode renders files ending in `.md` to `.html` files in the same logical directory. Markdown index mode renders one selected manifest-listed Markdown file to `index.html`. Non-Markdown files do not generate sidecar outputs. Raw HTML embedded in Markdown is not passed through by the renderer.
|
||||
|
||||
Generated HTML is deterministic for the same source content and transform configuration.
|
||||
|
||||
@@ -22,7 +22,7 @@ See `docs/integrations/markdown.md` for the Goldmark integration contract.
|
||||
|
||||
## Failure behavior
|
||||
|
||||
Transform resolution fails when a requested transform id is not registered. Markdown rendering fails when the source file cannot be read or rendered. Publish planning fails when HTML output is requested and the selected transform produces no outputs for a bundle.
|
||||
Transform resolution fails when a requested transform id is not registered. Markdown rendering fails when the source file cannot be read or rendered. Index input selection fails when the configured input is unsafe, not listed, not Markdown, or when no configured input can be inferred from exactly one manifest-listed Markdown file. Publish planning fails when HTML output is requested and the selected transform produces no outputs for a bundle.
|
||||
|
||||
## Boundaries
|
||||
|
||||
@@ -42,5 +42,6 @@ Before changing transform behavior, inspect tests under:
|
||||
- Source bundle files are never mutated by transforms.
|
||||
- Generated outputs record destination path, source path, transform id, SHA-256, and size.
|
||||
- Markdown sidecar naming changes only the `.md` extension to `.html`.
|
||||
- Markdown index mode always writes `index.html`.
|
||||
- Non-Markdown source files do not generate Markdown outputs.
|
||||
- Transform registration stays outside publish planning.
|
||||
|
||||
@@ -26,6 +26,12 @@ Run the local HTML publication:
|
||||
go run ./cmd/distributor run --config examples/local-html.yml
|
||||
```
|
||||
|
||||
Run the local `index.html` publication:
|
||||
|
||||
```sh
|
||||
go run ./cmd/distributor run --config examples/local-index.yml
|
||||
```
|
||||
|
||||
Preview local fan-out publication:
|
||||
|
||||
```sh
|
||||
@@ -115,6 +121,37 @@ go run ./cmd/distributor validate <bundle-path>
|
||||
|
||||
Use repeated `--file` flags to preserve a specific file order. If no `--file` flags are provided, the command scans the bundle directory recursively using the same filtering rules as `pkg/bundle.BuildManifest`.
|
||||
|
||||
## Static HTML Publication
|
||||
|
||||
Markdown-to-HTML publication can write sidecar files or a fixed `index.html`.
|
||||
|
||||
Use sidecar mode when each Markdown source should keep a matching HTML filename:
|
||||
|
||||
```yaml
|
||||
publish:
|
||||
source: false
|
||||
html: true
|
||||
transform:
|
||||
markdown_to_html:
|
||||
enabled: true
|
||||
mode: sidecar
|
||||
```
|
||||
|
||||
Use index mode for static-site destinations that should serve a bundle through `index.html`:
|
||||
|
||||
```yaml
|
||||
publish:
|
||||
source: false
|
||||
html: true
|
||||
transform:
|
||||
markdown_to_html:
|
||||
enabled: true
|
||||
mode: index
|
||||
input: report.md
|
||||
```
|
||||
|
||||
If `input` is omitted in index mode, the source manifest must list exactly one Markdown file. Generated HTML is recorded in `.distributor.json` with `kind: generated`, `source_path`, `transform: markdown_to_html`, digest, and size metadata.
|
||||
|
||||
## Source Validation and Inspection
|
||||
|
||||
`validate` and `inspect` can operate on a local path or on one configured pipeline source. Configured source mode requires both `--config` and `--pipeline`; it loads the normal config, resolves `secrets.directory`, opens only the selected source backend, and does not open any destinations.
|
||||
|
||||
@@ -19,7 +19,7 @@ Use it with `docs/policy/architecture.md` and `docs/policy/documentation.md`.
|
||||
- `internal/storage/fake`: in-memory backend for tests.
|
||||
- `internal/publish`: destination inspection, output planning, reconciliation, execution, managed cleanup, and explicit forced replacement.
|
||||
- `internal/transform`: transform interface and registry.
|
||||
- `internal/transform/markdown`: Markdown-to-HTML sidecar transform.
|
||||
- `internal/transform/markdown`: Markdown-to-HTML transform.
|
||||
- `internal/notify`: notification interface and current no-op notifier.
|
||||
- `internal/testutil`: shared test fixtures. Production code must not import this package.
|
||||
- `docs`: current user, operator, policy, internal, and roadmap documentation.
|
||||
@@ -128,8 +128,9 @@ When adding or changing commands or flags:
|
||||
3. Add or update CLI tests in `internal/cli`.
|
||||
4. Update `docs/cli.md` if syntax, flags, output expectations, or workflows change.
|
||||
|
||||
`validate` and `inspect` are local path commands. `run` loads configured
|
||||
pipelines and executes local, SSH, and S3 backends.
|
||||
`validate` and `inspect` support a local path shortcut and configured
|
||||
source-only diagnostics. `run` loads configured pipelines and executes local,
|
||||
SSH, and S3 backends.
|
||||
|
||||
## Storage Backends
|
||||
|
||||
|
||||
Reference in New Issue
Block a user