From a51228c80354a164a3f80ab3d08f02330d6fcb03 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sun, 24 May 2026 19:15:42 -0500 Subject: [PATCH] Add a documentation roadmap for the upcoming render stage feature --- docs/roadmap/render.md | 206 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 206 insertions(+) create mode 100644 docs/roadmap/render.md diff --git a/docs/roadmap/render.md b/docs/roadmap/render.md new file mode 100644 index 0000000..ce0ad00 --- /dev/null +++ b/docs/roadmap/render.md @@ -0,0 +1,206 @@ +# Roadmap: Render Stage + +Status: Planned + +This roadmap defines a post-1.0 feature addition: a new `render` stage that uses `seriatim render` to produce human-readable Markdown versions of the final transcript artifacts. + +Planned behavior belongs only in this roadmap until implementation lands. Current-behavior docs, examples, and command references must be updated only after the code is implemented and tested. + +## Goal + +Add a first-class stage between `trim` and `analyze`: + +1. `prepare` +2. `transcribe` +3. `merge` +4. `polish` +5. `normalize` +6. `trim` +7. `render` +8. `analyze` +9. `publish` +10. `notify` + +The stage renders Markdown versions of: + +- `narratio.transcript.final` +- `narratio.transcript.final_trimmed` + +The stage produces new built-in artifacts: + +| Source ID | Canonical path | Output kind | +| --- | --- | --- | +| `narratio.transcript.final_markdown` | `transcripts/final.md` | `transcript_final_markdown` | +| `narratio.transcript.final_trimmed_markdown` | `transcripts/final.trimmed.md` | `transcript_final_trimmed_markdown` | + +Default publish outputs should include: + +- `narratio.transcript.final_trimmed` +- `narratio.transcript.final_markdown` +- `narratio.transcript.final_trimmed_markdown` + +## Public Contract + +Add `pipeline.render` with strict YAML decoding. + +Fields: + +| Field | Type | Default | Validation | +| --- | --- | --- | --- | +| `enabled` | bool | `true` | optional | +| `format` | string | `markdown` | only `markdown` is supported | +| `title` | string | empty | optional | +| `include_timestamps` | bool | `true` | optional | +| `include_segment_ids` | bool | `false` | optional | +| `include_metadata` | bool | `false` | optional | + +Title behavior: + +- if `pipeline.render.title` is non-empty, pass it as `--title`; +- otherwise, if `session.title` is non-empty, pass `session.title` as `--title`; +- otherwise, omit `--title` and let Seriatim use its default. + +The initial implementation supports only Markdown. Future formats require explicit config validation and artifact naming decisions. + +## Implementation Stages + +### Stage 1: Artifact, Config, and Adapter Contracts + +- Extend the transcript artifact model with the two Markdown built-ins. +- Register Markdown artifacts as text content in the artifact registry so they work in artifact resolution, publish outputs, locks, status, artifacts list, and Scriptorium inputs. +- Add `RenderConfig` under `PipelineConfig` and apply defaults in the config loader/defaulting path. +- Validate `pipeline.render.format` as `markdown` and keep unknown fields rejected by strict YAML decoding. +- Extend the Seriatim adapter interface with `Render(ctx, RenderRequest)`. +- Add subprocess support for `seriatim render` with: + - `--input-file` + - `--output-file` + - `--format markdown` + - optional `--title` + - explicit boolean behavior for timestamps, segment IDs, and metadata. +- Validate render output as non-empty text, not JSON. +- Write Seriatim render stdout/stderr logs and generated invocation config consistently with existing Seriatim stage calls. + +### Stage 2: Render Stage Runtime + +- Add `renderStage` to the stage package. +- Insert `renderStage{}` into `stage.All()` after `trimStage{}` and before `analyzeStage{}`. +- Make `run-stage render ` work through the existing stage selection path. +- If render is disabled, mark the stage succeeded with metadata and no outputs. +- If enabled, resolve inputs manifest-first using existing artifact resolution: + - final transcript from `narratio.transcript.final`; + - final trimmed transcript from `narratio.transcript.final_trimmed`. +- Render run-local Markdown outputs first, then materialize canonical outputs: + - `transcripts/final.md` + - `transcripts/final.trimmed.md` +- Record manifest outputs with the new output kinds and source IDs. +- Record metadata for input paths/provenance, canonical/run-local output paths, format, resolved title, boolean render settings, adapter duration/exit code/binary, and adapter metadata. +- On missing required JSON inputs, fail clearly with guidance to run `normalize` or `trim` as appropriate. + +### Stage 3: Publish, Analyze, Docs, and Examples + +- Update default publish outputs to include both Markdown artifacts in addition to final trimmed JSON. +- Ensure publish output destination derivation works for Markdown built-ins through the shared artifact policy path. +- Ensure Scriptorium input validation accepts Markdown built-ins as ordinary built-in sources. +- Update analyze missing-input guidance so required Markdown built-in inputs point operators to `run-stage render`. +- Update user/operator/internal docs only after implementation: + - `docs/cli.md` + - `docs/config.md` + - `docs/operations.md` + - `docs/internal/README.md` + - `docs/internal/artifacts.md` + - new `docs/internal/stage-render.md` + - `docs/integrations/seriatim.md` +- Update examples only where useful; defaults should work without an explicit `pipeline.render` block. + +## Seriatim Adapter Contract + +Add a narrow render request/result beside the existing merge, normalize, and trim contracts. + +Request fields: + +- binary +- input transcript path +- output Markdown path +- format +- title +- include timestamps +- include segment IDs +- include metadata +- stdout log path +- stderr log path +- generated config path +- timeout + +Result fields: + +- output path +- stdout log path +- stderr log path +- generated config path +- exit code +- duration +- invoked binary +- format +- title +- metadata + +The adapter owns subprocess command construction and validation of the non-empty output file. Stage logic should express intent in Narratio terms and should not construct subprocess arguments directly. + +## Testing Guidance + +Focused tests: + +- `internal/artifactmodel` and `internal/artifacts` + - new Markdown source IDs, canonical paths, output kinds, producer stage, text validation, catalog ordering; + - resolver fallback from canonical Markdown paths; + - publish destination derivation for Markdown built-ins. +- `internal/config` + - render defaults; + - strict decode rejects unknown render fields; + - invalid format fails validation; + - default publish outputs include final trimmed JSON plus both Markdown artifacts. +- `internal/adapters/seriatim` + - render command args; + - title omission vs explicit title; + - boolean flag behavior; + - generated config; + - stdout/stderr logs; + - non-empty output validation; + - failure wrapping. +- `internal/stage` + - render resolves final and final-trimmed inputs from manifest outputs before canonical fallback; + - render writes run-local outputs and materializes canonical Markdown outputs; + - render records manifest outputs, logs, generated configs, and metadata; + - disabled render succeeds without outputs; + - missing final/final-trimmed inputs fail clearly. +- `internal/app` + - full plan order includes `render`; + - `run-stage render ` works; + - force rerunning render marks analyze, publish, and notify stale; + - status and artifacts list include Markdown built-ins; + - publish defaults include Markdown outputs. + +Validation commands: + +- `go test ./internal/artifactmodel ./internal/artifacts -v` +- `go test ./internal/config -v` +- `go test ./internal/adapters/seriatim -v` +- `go test ./internal/stage -run Render -v` +- `go test ./internal/app -run 'Plan|RunStage|Publish|Artifacts|Status' -v` +- `go test ./...` + +## Non-Goals + +- Do not change existing JSON transcript source IDs, canonical paths, or output kinds. +- Do not make Markdown output paths configurable in the first implementation. +- Do not add additional render formats before the format naming and artifact naming contract is defined. +- Do not move Seriatim subprocess details into stage logic. +- Do not document the render stage as implemented outside this roadmap until implementation lands. + +## Assumptions + +- `render` is enabled by default. +- Markdown canonical paths are fixed built-in artifact paths. +- `format: markdown` is the only supported initial format. +- Both Markdown outputs are included in default publish outputs. +- Existing publish layout and current-state commit behavior remain unchanged.