Cleanup following the render stage implementation and remove the completed roadmap
All checks were successful
ci/woodpecker/tag/release Pipeline was successful
All checks were successful
ci/woodpecker/tag/release Pipeline was successful
This commit is contained in:
@@ -1,206 +0,0 @@
|
||||
# Roadmap: Render Stage
|
||||
|
||||
Status: Completed
|
||||
|
||||
This roadmap defines a post-1.0 feature addition: a new `render` stage that uses `seriatim render` to produce human-readable Markdown versions of the final transcript artifacts.
|
||||
|
||||
Planned behavior belongs only in this roadmap until implementation lands. Current-behavior docs, examples, and command references must be updated only after the code is implemented and tested.
|
||||
|
||||
## Goal
|
||||
|
||||
Add a first-class stage between `trim` and `analyze`:
|
||||
|
||||
1. `prepare`
|
||||
2. `transcribe`
|
||||
3. `merge`
|
||||
4. `polish`
|
||||
5. `normalize`
|
||||
6. `trim`
|
||||
7. `render`
|
||||
8. `analyze`
|
||||
9. `publish`
|
||||
10. `notify`
|
||||
|
||||
The stage renders Markdown versions of:
|
||||
|
||||
- `narratio.transcript.final`
|
||||
- `narratio.transcript.final_trimmed`
|
||||
|
||||
The stage produces new built-in artifacts:
|
||||
|
||||
| Source ID | Canonical path | Output kind |
|
||||
| --- | --- | --- |
|
||||
| `narratio.transcript.final_markdown` | `transcripts/final.md` | `transcript_final_markdown` |
|
||||
| `narratio.transcript.final_trimmed_markdown` | `transcripts/final.trimmed.md` | `transcript_final_trimmed_markdown` |
|
||||
|
||||
Default publish outputs should include:
|
||||
|
||||
- `narratio.transcript.final_trimmed`
|
||||
- `narratio.transcript.final_markdown`
|
||||
- `narratio.transcript.final_trimmed_markdown`
|
||||
|
||||
## Public Contract
|
||||
|
||||
Add `pipeline.render` with strict YAML decoding.
|
||||
|
||||
Fields:
|
||||
|
||||
| Field | Type | Default | Validation |
|
||||
| --- | --- | --- | --- |
|
||||
| `enabled` | bool | `true` | optional |
|
||||
| `format` | string | `markdown` | only `markdown` is supported |
|
||||
| `title` | string | empty | optional |
|
||||
| `include_timestamps` | bool | `true` | optional |
|
||||
| `include_segment_ids` | bool | `false` | optional |
|
||||
| `include_metadata` | bool | `false` | optional |
|
||||
|
||||
Title behavior:
|
||||
|
||||
- if `pipeline.render.title` is non-empty, pass it as `--title`;
|
||||
- otherwise, if `session.title` is non-empty, pass `session.title` as `--title`;
|
||||
- otherwise, omit `--title` and let Seriatim use its default.
|
||||
|
||||
The initial implementation supports only Markdown. Future formats require explicit config validation and artifact naming decisions.
|
||||
|
||||
## Implementation Stages
|
||||
|
||||
### Stage 1: Artifact, Config, and Adapter Contracts
|
||||
|
||||
- Extend the transcript artifact model with the two Markdown built-ins.
|
||||
- Register Markdown artifacts as text content in the artifact registry so they work in artifact resolution, publish outputs, locks, status, artifacts list, and Scriptorium inputs.
|
||||
- Add `RenderConfig` under `PipelineConfig` and apply defaults in the config loader/defaulting path.
|
||||
- Validate `pipeline.render.format` as `markdown` and keep unknown fields rejected by strict YAML decoding.
|
||||
- Extend the Seriatim adapter interface with `Render(ctx, RenderRequest)`.
|
||||
- Add subprocess support for `seriatim render` with:
|
||||
- `--input-file`
|
||||
- `--output-file`
|
||||
- `--format markdown`
|
||||
- optional `--title`
|
||||
- explicit boolean behavior for timestamps, segment IDs, and metadata.
|
||||
- Validate render output as non-empty text, not JSON.
|
||||
- Write Seriatim render stdout/stderr logs and generated invocation config consistently with existing Seriatim stage calls.
|
||||
|
||||
### Stage 2: Render Stage Runtime
|
||||
|
||||
- Add `renderStage` to the stage package.
|
||||
- Insert `renderStage{}` into `stage.All()` after `trimStage{}` and before `analyzeStage{}`.
|
||||
- Make `run-stage render <session_id>` work through the existing stage selection path.
|
||||
- If render is disabled, mark the stage succeeded with metadata and no outputs.
|
||||
- If enabled, resolve inputs manifest-first using existing artifact resolution:
|
||||
- final transcript from `narratio.transcript.final`;
|
||||
- final trimmed transcript from `narratio.transcript.final_trimmed`.
|
||||
- Render run-local Markdown outputs first, then materialize canonical outputs:
|
||||
- `transcripts/final.md`
|
||||
- `transcripts/final.trimmed.md`
|
||||
- Record manifest outputs with the new output kinds and source IDs.
|
||||
- Record metadata for input paths/provenance, canonical/run-local output paths, format, resolved title, boolean render settings, adapter duration/exit code/binary, and adapter metadata.
|
||||
- On missing required JSON inputs, fail clearly with guidance to run `normalize` or `trim` as appropriate.
|
||||
|
||||
### Stage 3: Publish, Analyze, Docs, and Examples
|
||||
|
||||
- Update default publish outputs to include both Markdown artifacts in addition to final trimmed JSON.
|
||||
- Ensure publish output destination derivation works for Markdown built-ins through the shared artifact policy path.
|
||||
- Ensure Scriptorium input validation accepts Markdown built-ins as ordinary built-in sources.
|
||||
- Update analyze missing-input guidance so required Markdown built-in inputs point operators to `run-stage render`.
|
||||
- Update user/operator/internal docs only after implementation:
|
||||
- `docs/cli.md`
|
||||
- `docs/config.md`
|
||||
- `docs/operations.md`
|
||||
- `docs/internal/README.md`
|
||||
- `docs/internal/artifacts.md`
|
||||
- new `docs/internal/stage-render.md`
|
||||
- `docs/integrations/seriatim.md`
|
||||
- Update examples only where useful; defaults should work without an explicit `pipeline.render` block.
|
||||
|
||||
## Seriatim Adapter Contract
|
||||
|
||||
Add a narrow render request/result beside the existing merge, normalize, and trim contracts.
|
||||
|
||||
Request fields:
|
||||
|
||||
- binary
|
||||
- input transcript path
|
||||
- output Markdown path
|
||||
- format
|
||||
- title
|
||||
- include timestamps
|
||||
- include segment IDs
|
||||
- include metadata
|
||||
- stdout log path
|
||||
- stderr log path
|
||||
- generated config path
|
||||
- timeout
|
||||
|
||||
Result fields:
|
||||
|
||||
- output path
|
||||
- stdout log path
|
||||
- stderr log path
|
||||
- generated config path
|
||||
- exit code
|
||||
- duration
|
||||
- invoked binary
|
||||
- format
|
||||
- title
|
||||
- metadata
|
||||
|
||||
The adapter owns subprocess command construction and validation of the non-empty output file. Stage logic should express intent in Narratio terms and should not construct subprocess arguments directly.
|
||||
|
||||
## Testing Guidance
|
||||
|
||||
Focused tests:
|
||||
|
||||
- `internal/artifactmodel` and `internal/artifacts`
|
||||
- new Markdown source IDs, canonical paths, output kinds, producer stage, text validation, catalog ordering;
|
||||
- resolver fallback from canonical Markdown paths;
|
||||
- publish destination derivation for Markdown built-ins.
|
||||
- `internal/config`
|
||||
- render defaults;
|
||||
- strict decode rejects unknown render fields;
|
||||
- invalid format fails validation;
|
||||
- default publish outputs include final trimmed JSON plus both Markdown artifacts.
|
||||
- `internal/adapters/seriatim`
|
||||
- render command args;
|
||||
- title omission vs explicit title;
|
||||
- boolean flag behavior;
|
||||
- generated config;
|
||||
- stdout/stderr logs;
|
||||
- non-empty output validation;
|
||||
- failure wrapping.
|
||||
- `internal/stage`
|
||||
- render resolves final and final-trimmed inputs from manifest outputs before canonical fallback;
|
||||
- render writes run-local outputs and materializes canonical Markdown outputs;
|
||||
- render records manifest outputs, logs, generated configs, and metadata;
|
||||
- disabled render succeeds without outputs;
|
||||
- missing final/final-trimmed inputs fail clearly.
|
||||
- `internal/app`
|
||||
- full plan order includes `render`;
|
||||
- `run-stage render <session_id>` works;
|
||||
- force rerunning render marks analyze, publish, and notify stale;
|
||||
- status and artifacts list include Markdown built-ins;
|
||||
- publish defaults include Markdown outputs.
|
||||
|
||||
Validation commands:
|
||||
|
||||
- `go test ./internal/artifactmodel ./internal/artifacts -v`
|
||||
- `go test ./internal/config -v`
|
||||
- `go test ./internal/adapters/seriatim -v`
|
||||
- `go test ./internal/stage -run Render -v`
|
||||
- `go test ./internal/app -run 'Plan|RunStage|Publish|Artifacts|Status' -v`
|
||||
- `go test ./...`
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Do not change existing JSON transcript source IDs, canonical paths, or output kinds.
|
||||
- Do not make Markdown output paths configurable in the first implementation.
|
||||
- Do not add additional render formats before the format naming and artifact naming contract is defined.
|
||||
- Do not move Seriatim subprocess details into stage logic.
|
||||
- Do not document the render stage as implemented outside this roadmap until implementation lands.
|
||||
|
||||
## Assumptions
|
||||
|
||||
- `render` is enabled by default.
|
||||
- Markdown canonical paths are fixed built-in artifact paths.
|
||||
- `format: markdown` is the only supported initial format.
|
||||
- Both Markdown outputs are included in default publish outputs.
|
||||
- Existing publish layout and current-state commit behavior remain unchanged.
|
||||
Reference in New Issue
Block a user