Initial documentation cleanup pass

This commit is contained in:
2026-05-23 07:11:17 -05:00
parent 71395bb076
commit ab59bab044
8 changed files with 0 additions and 1142 deletions

View File

@@ -1,231 +0,0 @@
# Roadmap: Campaign Registry
Status: Implemented
## Problem
Narratio currently treats campaign configuration as one selected
`campaign.yml` file:
- command flags use `--campaign <path>`;
- default discovery searches fixed system file locations;
- `campaign.yml` uses `campaign:` as the identity field.
That model works for a single campaign, but it is awkward for installations
that manage multiple campaigns. Operators need to pass file paths or maintain a
single global campaign config, while the newer session-oriented CLI already
uses concise positional session IDs and remote session lookup.
The campaign selection model should become ID-based and pipeline-owned.
Pipeline config should describe where campaigns live, commands should select a
campaign by ID, and each campaign directory should contain its stable campaign
materials.
## Target Model
`pipeline.yml` owns the campaign registry:
campaigns:
root: /usr/local/share/narratio/campaigns
default_campaign_id: dilfs
Campaign files live at the conventional path:
{campaigns.root}/{campaign_id}/campaign.yml
The first implementation should use only the conventional path. Recursive
discovery of every `campaign.yml` under `campaigns.root` is deferred to a
future stage.
Each campaign file uses `campaign_id` as the canonical identity field:
campaign_id: dilfs
session_template_file: ./session.template.yml
inputs:
speakers_file: ./speakers.yml
autocorrect_file: ./autocorrect.yml
glossary_file: ./glossary.yml
Campaign-relative files continue to resolve relative to the selected
`campaign.yml`, including stable input files and `session_template_file`.
The public CLI changes from path-based campaign selection to ID-based campaign
selection:
- `--campaign <id>` selects a campaign ID.
- `--campaign-file <path>` explicitly loads one campaign file for
development, tests, and unusual local workflows.
- `--campaign` and `--campaign-file` are mutually exclusive.
If neither `--campaign` nor `--campaign-file` is passed, Narratio uses
`pipeline.campaigns.default_campaign_id`. If no campaign can be selected,
commands fail clearly before session loading or stage execution.
Resolved campaign ID remains the campaign segment used for:
- workspace paths;
- spool paths;
- S3 session prefixes;
- remote `session.yml` lookup;
- archive locks and promoted output keys;
- session/campaign mismatch validation;
- status, plan, restore, and helper output.
## Compatibility Policy
This is a breaking public/config contract change.
After the cutover:
- `--campaign` no longer accepts a filesystem path;
- default fixed campaign file discovery is removed;
- `campaign:` is no longer accepted in `campaign.yml`;
- `campaign_id:` is required.
Keep `--campaign-file` as the only explicit file override. Do not retain hidden
aliases for the old `--campaign <path>` behavior.
## Implementation Stages
### Stage 1: Add Campaign Registry Selection
Status: Implemented
Add the registry model and switch command loading to resolve campaigns through
pipeline config.
Implementation requirements:
- Add `pipeline.campaigns.root`.
- Add `pipeline.campaigns.default_campaign_id`.
- Add `campaign_id` to campaign config and make it the canonical identity.
- Resolve pipeline config first, then campaign selection.
- Use this selection order:
1. explicit `--campaign-file <path>`;
2. explicit `--campaign <id>`;
3. `pipeline.campaigns.default_campaign_id`;
4. fail clearly.
- For ID selection, load `{campaigns.root}/{campaign_id}/campaign.yml`.
- Validate that the loaded `campaign_id` matches the selected ID.
- Reject `--campaign` with `--campaign-file`.
- Preserve strict YAML decoding.
- Preserve campaign-relative stable input and session template resolution.
- Keep storage details behind the existing storage adapter and object-store
helper.
- Keep remote session lookup and archive key construction based on the
resolved campaign ID.
Acceptance criteria:
- Commands can run with only a pipeline config and the pipeline default
campaign ID.
- Commands can select another campaign with `--campaign <id>`.
- Commands can load a specific file with `--campaign-file <path>`.
- Existing session loading, remote session fallback, prepare materialization,
restore, archive, locks, clean, analyze, and publish behavior continue to use
the same resolved campaign identity.
- No generic config registry framework is introduced.
### Stage 2: Remove Old Single-File Campaign Behavior
Status: Implemented
Remove the old public campaign file model after registry selection is in
place.
Implementation requirements:
- Remove fixed default campaign config discovery from command loading.
- Remove `DefaultCampaignConfigSearchPaths` and related path-only resolution if
no current tests or helpers still need them.
- Remove support for `campaign:` from `campaign.yml`.
- Update validation errors to refer to `campaign_id`.
- Update examples to use campaign directories and `campaign_id`.
- Update current-behavior docs to document:
- `pipeline.campaigns.root`;
- `pipeline.campaigns.default_campaign_id`;
- `campaign_id`;
- `--campaign <id>`;
- `--campaign-file <path>`.
- Update troubleshooting examples that currently pass `--campaign <path>`.
Acceptance criteria:
- `campaign.yml` files with `campaign:` fail strict decoding.
- `--campaign /path/to/campaign.yml` is treated as a campaign ID and fails
unless that ID exists under `campaigns.root`.
- `--campaign-file /path/to/campaign.yml` is the supported file override.
- User-facing docs no longer describe fixed campaign config discovery.
## Test Guidance
Focused tests:
- `go test ./internal/config -v`
- `go test ./internal/app -v`
- `go test ./internal/stage -run Prepare -v`
Full validation:
- `go test ./...`
Config tests to add or update:
- strict decode accepts `pipeline.campaigns.root`;
- strict decode accepts `pipeline.campaigns.default_campaign_id`;
- strict decode accepts `campaign_id`;
- selected campaign ID mismatch fails;
- missing campaign root fails when ID selection is needed;
- missing default campaign ID fails when no explicit campaign selector is
passed;
- old `campaign:` fails after Stage 2.
App tests to add or update:
- `--campaign <id>` resolves `{campaigns.root}/{id}/campaign.yml`;
- omitted `--campaign` uses `pipeline.campaigns.default_campaign_id`;
- `--campaign-file` loads an explicit campaign file;
- `--campaign` plus `--campaign-file` fails;
- remote session fallback uses the resolved campaign ID;
- `session init`, `run`, `run-stage`, `resume`, `analyze`, `publish`, `clean`,
and `session` subcommands all use the same campaign selection path;
- path-based `--campaign` examples and tests are removed after Stage 2.
## Documentation Guidance
Update current-behavior docs only after implementation lands:
- `docs/config.md`
- `docs/cli.md`
- `docs/operations.md`
- `docs/troubleshooting.md`
- relevant files under `docs/internal/`
- `examples/`
Planned campaign registry behavior belongs only in this roadmap until the code,
tests, examples, and current-behavior docs are updated.
## Architecture Guardrails
- Keep Narratio explicit and stage-driven.
- Do not introduce a generic configuration registry or workflow framework.
- Keep YAML decoding strict.
- Keep defaults centralized and testable.
- Keep campaign-relative path resolution centralized.
- Use centralized S3 and workspace path helpers.
- Keep storage details behind `storage.ObjectStore`.
- Keep secret-backed object-store construction in `internal/app`.
- Preserve manifest-driven resume and restore behavior.
- Do not store raw secrets in campaign configs, manifests, logs, generated
configs, or archive metadata.
## Assumptions
- The canonical pipeline schema is grouped under `campaigns`.
- The canonical campaign identity field is `campaign_id`.
- `--campaign` means campaign ID.
- `--campaign-file` is retained as an explicit override.
- Recursive discovery is planned but not part of the first implementation.
- Existing production configs can be migrated from `campaign:` to
`campaign_id:` and from `--campaign <path>` to `--campaign <id>` or
`--campaign-file <path>`.

View File

@@ -1,159 +0,0 @@
# Roadmap: Legacy Config Cleanup
Status: Implemented
## Problem
Narratio's current pipeline config schema still accepts fields that predate the current storage, artifact, and previous-session models:
- `pipeline.storage.bucket`
- `pipeline.storage.prefix`
- `pipeline.analyzer.*`
- `previous_session_artifact`
These names make the config reference harder to trust because they suggest supported behavior that operators should no longer use. The modern interface is:
- `pipeline.storage.s3.*` for remote storage.
- Scriptorium configured artifacts under `pipeline.scriptorium.artifacts`.
- Canonical artifact source IDs such as `narratio.artifact.<configured_artifact_key>`.
- Canonical previous-session artifact sources such as `narratio.previous_session.artifact.<configured_artifact_key>`.
Strict YAML decoding should reject removed legacy fields once this cleanup lands.
## Current State
`pipeline.storage.bucket` and `pipeline.storage.prefix` were inert compatibility fields and have been removed:
- They are no longer present on `config.StorageConfig`.
- Strict decoding rejects them.
- Runtime S3 behavior uses `pipeline.storage.s3.bucket` and `pipeline.storage.s3.root_prefix`.
- No current code reads the top-level storage bucket or prefix fields.
`pipeline.analyzer.*` was legacy code surface and has been removed:
- `config.PipelineConfig` no longer includes analyzer config.
- Strict decoding rejects `pipeline.analyzer`.
- `stage.Env` no longer exposes an analyzer runner, and `internal/adapters/analyzer` has been deleted.
- Modern analyze execution is Scriptorium-backed; the analyzer adapter is not used by current stage execution.
`previous_session_artifact` was a live legacy behavior and has been removed:
- Config validation rejects it as an unsupported Scriptorium input source.
- The analyze stage no longer has path-based previous-artifact resolution through `inputs.<name>.path`.
- Tests cover canonical previous-session sources and the rejection of the legacy source.
- The canonical replacement is `narratio.previous_session.artifact.<configured_artifact_key>`, resolved through the previous-session cache/catalog model.
## Target Model
The pipeline config schema should expose only current behavior:
- Remote storage is configured only through `pipeline.storage.s3.*`.
- Generated artifacts are configured only through `pipeline.scriptorium.artifacts`.
- Scriptorium artifact inputs use canonical source IDs.
- Previous-session artifact inputs use `narratio.previous_session.artifact.<configured_artifact_key>`.
- Unknown legacy fields fail strict YAML decoding.
No compatibility aliases should remain unless a future migration requirement explicitly reintroduces them.
## Cleanup Order
### Stage 1: Remove Inert Storage Compatibility Fields
Status: Implemented
Remove `pipeline.storage.bucket` and `pipeline.storage.prefix`.
Implementation requirements:
- Delete `StorageConfig.Bucket` and `StorageConfig.Prefix`.
- Keep `StorageConfig.Backend` and `StorageConfig.S3`.
- Confirm all runtime storage paths continue to use `storage.s3.bucket` and `storage.s3.root_prefix`.
- Update examples and docs to remove top-level storage `bucket` and `prefix`.
- Add or update strict-decode tests proving `pipeline.storage.bucket` and `pipeline.storage.prefix` are rejected.
Acceptance criteria:
- Existing S3 workflows still pass with `pipeline.storage.s3.bucket`.
- Pipeline configs containing top-level `storage.bucket` or `storage.prefix` fail to load.
- No docs or examples present those fields as available.
### Stage 2: Remove Legacy Analyzer Schema and Adapter Surface
Status: Implemented
Remove the unused analyzer configuration and adapter contract.
Implementation requirements:
- Delete `PipelineConfig.Analyzer`.
- Delete `AnalyzerConfig` and `ArtifactSettings`.
- Remove analyzer timeout validation.
- Remove `stage.Env.Analyzer`.
- Delete `internal/adapters/analyzer` if no remaining code imports it.
- Remove `pipeline.analyzer.*` from tests, examples, and docs.
- Add or update strict-decode tests proving `pipeline.analyzer` is rejected.
Acceptance criteria:
- Analyze behavior remains fully Scriptorium-backed.
- No runtime code imports `internal/adapters/analyzer`.
- Pipeline configs containing `pipeline.analyzer` fail to load.
- Contributor and internal adapter docs no longer list the analyzer adapter.
### Stage 3: Remove Path-Based Previous Session Artifact Source
Status: Implemented
Remove `previous_session_artifact` and require canonical previous-session artifact sources.
Implementation requirements:
- Remove `previous_session_artifact` from supported Scriptorium input sources.
- Remove analyze-stage special-case handling that resolves `inputs.<name>.path` for previous artifacts.
- Keep canonical handling for `narratio.previous_session.artifact.<configured_artifact_key>`.
- Rewrite tests that use `previous_session_artifact` to use canonical sources and prepared previous-cache fixtures.
- Add validation tests proving `previous_session_artifact` is rejected.
- Update docs to remove the legacy path-based source and document only canonical previous-session sources.
Acceptance criteria:
- `pipeline.scriptorium.artifacts.*.inputs.*.source: previous_session_artifact` fails validation.
- Canonical previous-session sources continue to work for required and optional inputs.
- Prepare/restore previous-cache behavior remains unchanged.
- No docs or examples mention `previous_session_artifact` as supported.
## Test Guidance
Run focused tests after each stage:
- `go test ./internal/config -v`
- `go test ./internal/stage -run Analyze -v`
- `go test ./internal/app -v`
- `go test ./...`
For Stage 1, focus on config load/strict-decode and S3 workflow regression tests.
For Stage 2, focus on compile-time removal, config strict-decode tests, and full app/stage tests to catch stale adapter references.
For Stage 3, focus on Scriptorium config validation, analyze-stage input resolution, previous-cache behavior, and restore/analyze workflows.
## Documentation Updates
Update current-behavior docs only after the corresponding code removal lands:
- `docs/config.md`
- `docs/cli.md`, only if command behavior text references removed fields.
- `docs/operations.md`, only if operator workflow text references removed fields.
- `docs/internal/stage-analyze.md`
- `docs/internal/adapters.md`
- `examples/pipeline.full.annotated.yml`
- `examples/pipeline.production.yml`
Do not preserve removed fields in examples as compatibility notes. The goal is to make strict config behavior and documentation line up.
## Assumptions
- This is a hard cleanup; no backward-compatible aliases are retained.
- Current production configs can be migrated to `storage.s3.*`, Scriptorium artifacts, and canonical previous-session sources before this lands.
- Removing the unused analyzer adapter does not block any active stage behavior.
- The cleanup should be implemented in the listed order so inert schema removal is separated from behavior removal.

View File

@@ -1,255 +0,0 @@
# Roadmap: Session-Oriented CLI Cleanup
Status: Implemented
## Problem
Narratio's public CLI has accumulated too many top-level commands. Several
commands are session-scoped operator helpers, but they currently appear as
independent top-level verbs:
- `plan`
- `status`
- `restore`
- `artifacts list`
- `locks`
- `session validate`
- `session init`
This makes the command surface harder to learn because the CLI does not clearly
separate primary workflow actions from session inspection, initialization,
restore, and helper operations.
## Target Model
Keep primary workflow commands at top level:
- `run`
- `run-stage`
- `resume`
- `analyze`
- `publish`
- `clean`
- `session`
Keep `clean` top-level because it can operate on one session or all local
sessions and is a workspace maintenance command, not only a session helper.
Move session-scoped helper commands under `narratio session` and use positional
session identifiers:
- `narratio session init <session_id> [--remote|--output <path>] [--flags]`
- `narratio session validate <session_id> [--flags]`
- `narratio session status <session_id> [--flags]`
- `narratio session plan <session_id> [--flags]`
- `narratio session restore <session_id> [--flags]`
- `narratio session artifacts <session_id> [--remote] [--flags]`
- `narratio session locks <session_id> [--flags]`
- `narratio session locks add <session_id> <source> [--reason <text>] [--force] [--flags]`
- `narratio session locks remove <session_id> <source> [--flags]`
Update top-level workflow commands to use positional session identifiers:
- `narratio run <session_id> [--flags]`
- `narratio resume <session_id> [--flags]`
- `narratio analyze <session_id> [--flags]`
- `narratio publish <session_id> [--flags]`
- `narratio run-stage <stage> <session_id> [--flags]`
The positional session ID replaces `--session-id` as the primary public
interface. Existing `--config`, `--campaign`, `--session`, and
`--previous-session-id` flags remain available where they are meaningful.
## Command Mapping
| Current command | Target command |
| --- | --- |
| `narratio run --session-id <id>` | `narratio run <id>` |
| `narratio resume --session-id <id>` | `narratio resume <id>` |
| `narratio analyze --session-id <id>` | `narratio analyze <id>` |
| `narratio publish --session-id <id>` | `narratio publish <id>` |
| `narratio run-stage [flags] <stage> --session-id <id>` | `narratio run-stage <stage> <id> [flags]` |
| `narratio plan --session-id <id>` | `narratio session plan <id>` |
| `narratio status --session-id <id>` | `narratio session status <id>` |
| `narratio restore --session-id <id>` | `narratio session restore <id>` |
| `narratio artifacts list --session-id <id>` | `narratio session artifacts <id>` |
| `narratio locks --session-id <id>` | `narratio session locks <id>` |
| `narratio locks add --session-id <id> <source>` | `narratio session locks add <id> <source>` |
| `narratio locks remove --session-id <id> <source>` | `narratio session locks remove <id> <source>` |
| `narratio session validate --session-id <id>` | `narratio session validate <id>` |
| `narratio session init --session-id <id>` | `narratio session init <id>` |
| `narratio clean --session-id <id>` | `narratio clean <id>` |
| `narratio clean --all` | unchanged |
`clean` remains top-level, but its session-scoped form should also move from
`--session-id` to positional `<session_id>` for consistency.
## Compatibility Policy
This is a hard public CLI cleanup after the migration step lands.
During Step 1, old forms may remain as compatibility aliases to keep the
implementation reviewable. During Step 2, remove the old forms from command
dispatch, tests, docs, and examples:
- remove top-level `plan`;
- remove top-level `status`;
- remove top-level `restore`;
- remove top-level `artifacts`;
- remove top-level `locks`;
- remove `--session-id` from the public command syntax for session-aware
commands.
Do not keep long-term deprecated aliases unless a later roadmap explicitly
chooses a compatibility window.
`status --manifest` does not fit the session-oriented command shape. Remove it
from the public CLI in this cleanup. If direct manifest inspection is needed
later, add a separate diagnostic command in a future roadmap rather than keeping
it as a special case in `session status`.
## Implementation Step 1: Add New Session-Oriented Interface
Status: Implemented
Add the target command forms while preserving current behavior internally.
Implementation requirements:
- Add positional session ID parsing helpers in `internal/app`.
- Keep the existing `loadCommandConfig` behavior and populate
`config.SessionLoadOptions.SessionID` from the positional ID.
- Add or update command wrappers:
- `Run(ctx, args, out)` parses `run <session_id>`.
- `Resume(ctx, args, out)` parses `resume <session_id>`.
- `Analyze(ctx, args, out)` parses `analyze <session_id>`.
- `Publish(ctx, args, out)` parses `publish <session_id>`.
- `RunStage(ctx, args, out)` parses `run-stage <stage> <session_id>`.
- `Clean(ctx, args, out)` parses `clean <session_id>` and keeps
`clean --all`.
- Extend `Session(ctx, args, out)` dispatch to support:
- `init <session_id>`
- `validate <session_id>`
- `status <session_id>`
- `plan <session_id>`
- `restore <session_id>`
- `artifacts <session_id>`
- `locks <session_id>`
- `locks add <session_id> <source>`
- `locks remove <session_id> <source>`
- Keep storage access through the existing app-level object-store helper.
- Keep AWS SDK details behind storage adapters.
- Keep the runner, stages, manifest behavior, archive behavior, restore
planning, lock semantics, and artifact catalog behavior unchanged.
Acceptance criteria:
- New forms execute the same code paths and produce equivalent results.
- Positional session ID mismatch with concrete local or remote `session.yml`
fails through existing session identity checks.
- Remote session fallback still uses the positional session ID as the lookup
value.
- Current command tests cover the new forms before old forms are removed.
## Implementation Step 2: Remove Old Public Forms
Status: Implemented
Remove compatibility aliases and make the session-oriented interface the only
documented and supported public CLI.
Implementation requirements:
- Remove top-level dispatch for:
- `plan`
- `status`
- `restore`
- `artifacts`
- `locks`
- Remove `--session-id` flags from public session-aware commands.
- Keep `--previous-session-id` as an expected previous-session identity flag.
- Keep explicit `--session <path>` for loading a local concrete session file,
but still require the positional session ID for commands that operate on a
session.
- Remove `status --manifest`.
- Update usage text and invalid-command errors.
- Update `docs/cli.md` and `docs/operations.md` to use only the new forms.
- Update any roadmap docs that mention old helper command names.
- Update tests to expect old top-level helper commands and `--session-id` forms
to fail.
Acceptance criteria:
- Top-level command list is exactly:
- `run`
- `run-stage`
- `resume`
- `analyze`
- `publish`
- `clean`
- `session`
- All session-oriented commands use `narratio session <subcommand>
<session_id> [--flags]`, except nested lock mutation forms, which use
`narratio session locks add|remove <session_id> <source> [--flags]`.
- `clean <session_id>` and `clean --all` remain top-level.
- Current-behavior docs and tests no longer advertise `--session-id`.
## Test Guidance
Focused tests:
- `go test ./internal/app -run TestExecute -v`
- `go test ./internal/app -run 'Session|Status|Restore|Clean|Locks|Artifacts|Plan|RunStage|Analyze|Publish' -v`
- `go test ./internal/config -v`
Full validation:
- `go test ./...`
Test cases to add or update:
- `run <session_id>` loads local and remote sessions through the existing
config path.
- `resume <session_id>`, `analyze <session_id>`, and `publish <session_id>`
preserve current behavior.
- `run-stage <stage> <session_id>` preserves current run-stage output and
force/artifact-selection behavior.
- `session plan <session_id>` replaces top-level `plan`.
- `session status <session_id>` replaces top-level session status.
- `session validate <session_id>` replaces `session validate --session-id`.
- `session init <session_id>` writes the same local or remote concrete
`session.yml`.
- `session restore <session_id>` preserves restore planning/execution.
- `session artifacts <session_id> --remote` preserves promoted-output
availability reporting.
- `session locks <session_id>`, `session locks add <session_id> <source>`, and
`session locks remove <session_id> <source>` preserve static/remote lock
semantics.
- `clean <session_id>` preserves session cleanup behavior, while `clean --all`
remains unchanged.
- Old top-level helper commands fail after Step 2.
- `--session-id` fails after Step 2.
- `status --manifest` fails after Step 2.
## Documentation Guidance
Update only after implementation lands:
- `docs/cli.md`
- `docs/operations.md`
- any internal docs that list command names or examples
Keep planned behavior only in this roadmap until the command refactor is
implemented.
## Architecture Guardrails
- Keep Narratio explicit and stage-driven.
- Do not introduce a generic workflow or command framework abstraction.
- Reuse existing app command helpers where practical.
- Keep config loading strict and centralized.
- Keep storage details behind `storage.ObjectStore`.
- Keep secret-backed object-store construction in `internal/app`.
- Preserve manifest-driven resume and restore behavior.
- Treat command renaming as a public CLI contract change, not a runtime stage
behavior change.

View File

@@ -1,287 +0,0 @@
# Roadmap: Publish Contract
Status: Planned
## Problem
Narratio currently uses several terms for one operator-facing concept:
- `archive` is the stage that uploads run state and commits remote current
state.
- `publish` is the convenience command that force-runs the archive stage.
- `promote`, `promoted`, and `promote_artifacts` describe configured top-level
remote output writes.
This mixed vocabulary makes the public contract harder to explain. Operators
should not need to distinguish "archive the run", "publish the run", and
"promote artifacts" when these are all part of the same publish action.
The public model should use:
- `publish` for the stage, command, config section, and action;
- `published` for an expected remote output that exists at its top-level
current destination;
- `publish rules` for the configured source-to-destination output rules;
- `locked` for sources whose top-level published destination must not be
overwritten;
- `run history` for immutable per-run records under `runs/<run_id>/`.
## Target Model
The public stage is `publish`.
The convenience command:
narratio publish <session_id>
is equivalent to:
narratio run-stage publish <session_id> --force
Pipeline configuration uses `publish`:
publish:
enabled: true
upload_run: true
outputs:
- source: narratio.transcript.final_trimmed
- source: narratio.artifact.session_recap
locks:
- source: narratio.artifact.session_recap
reason: Final recap was manually edited.
Publish output rules are source-based. Each rule writes one artifact source to
a top-level remote destination. If `dest` is omitted, Narratio derives the
destination from the artifact registry or configured artifact output path.
The mutable remote lock store remains:
{session_prefix}/locks.yml
Remote availability output uses `published`:
Published:
- narratio.transcript.final_trimmed remote=published
- narratio.artifact.session_recap locked remote=published
The remote key layout is otherwise unchanged:
- immutable run history stays under `{session_prefix}/runs/{run_id}/`;
- current state stays under `{session_prefix}/current/manifest.json`;
- the final commit marker stays `{session_prefix}/current/run_id.txt`;
- `current/run_id.txt` is still written last.
## Compatibility Policy
This is a hard cutover.
After implementation:
- `pipeline.archive` is rejected by strict YAML decoding.
- `pipeline.archive.promote_artifacts` is rejected.
- `pipeline.workspace.cleanup_after_archive` is rejected.
- `pipeline.spool.delete_audio_after_archive` is rejected.
- `narratio run-stage archive <session_id>` is an unknown stage.
- manifests that record an `archive` stage are not migrated.
- old archive/promotion metadata keys are not read as compatibility fallbacks.
Existing remote objects are not moved or renamed. Remote layout remains stable;
the rename changes configuration, stage names, status output, metadata, helper
names, tests, examples, and documentation.
## Implementation Stages
### Stage 1: Public Schema and Stage Cutover
Status: Planned
Switch the public config and stage contract to publish terminology.
Implementation requirements:
- Replace `pipeline.archive` with `pipeline.publish`.
- Replace `archive.promote_artifacts` with `publish.outputs`.
- Keep output rule fields:
- `source`
- `dest`
- `required`
- Replace `pipeline.archive.locks` with `pipeline.publish.locks`.
- Rename post-publish cleanup fields:
- `pipeline.workspace.cleanup_after_publish`
- `pipeline.spool.delete_audio_after_publish`
- Rename the registered stage from `archive` to `publish`.
- Update stage order so `publish` runs after `analyze` and before `notify`.
- Update top-level `narratio publish` to target stage `publish`.
- Keep `run-stage --artifacts <names> publish` support.
- Reject `run-stage --artifacts <names>` for stages other than `analyze` and
`publish`.
- Preserve the remote commit ordering and storage adapter boundaries.
Acceptance criteria:
- `narratio run-stage publish <session_id>` executes the publish stage.
- `narratio publish <session_id>` force-runs the publish stage.
- `narratio run-stage archive <session_id>` fails clearly as an unknown stage.
- Old archive config fields fail strict decoding.
- New publish config fields load, default, and validate.
### Stage 2: Runtime Terminology and Metadata Cutover
Status: Planned
Rename implementation concepts and runtime output to publish terminology.
Implementation requirements:
- Rename archive/promotion config and runtime types conceptually to
publish/output terms.
- Rename the remote key helper intent from promoted artifact to published
output while keeping generated keys unchanged.
- Change helper output:
- `Promoted:` becomes `Published:`
- `remote=promoted` becomes `remote=published`
- lock output uses `published` / `not-published`
- Rename publish-stage metadata, including:
- `promoted_paths` to `published_paths`
- `promoted_files_uploaded` to `published_files_uploaded`
- `skipped_optional_promotions` to `skipped_optional_outputs`
- `skipped_unselected_promotions` to `skipped_unselected_outputs`
- `locked_promotion_count` to `locked_output_count`
- `locked_promotions` to `locked_outputs`
- Update previous-cache and restore logic to use the `publish` stage and
`published_paths` metadata only.
- Keep run-local stage output materialization separate from remote publish
terminology. If local helper names are confusing, rename them to
materialization-oriented names rather than publish names.
Acceptance criteria:
- Status and artifact helper output use `Published:` and `remote=published`.
- Publish metadata contains only publish/output terminology.
- Previous-cache and restore behavior works with publish metadata and does not
depend on old archive metadata.
- Storage adapters still receive explicit keys and no AWS SDK details leak into
app or stage logic.
### Stage 3: Documentation, Examples, and Final Cleanup
Status: Planned
Update implemented-behavior docs and remove stale public terminology after the
runtime cutover lands.
Implementation requirements:
- Update current-behavior docs:
- `docs/config.md`
- `docs/cli.md`
- `docs/operations.md`
- `docs/troubleshooting.md`
- `docs/architecture.md`
- relevant files under `docs/internal/`
- Rename `docs/internal/stage-archive.md` to
`docs/internal/stage-publish.md`.
- Update internal documentation links and references.
- Update examples to use:
- `publish.outputs`
- `publish.locks`
- `cleanup_after_publish`
- `delete_audio_after_publish`
- Update tests and final searches so old terminology remains only in this
roadmap as historical context.
Acceptance criteria:
- Maintained examples load and validate.
- Current-behavior docs describe only implemented publish terminology.
- Internal docs describe run history, published outputs, locks, and current
commit ordering clearly.
- Old user-facing archive/promote wording is removed except where discussing
historical behavior in this roadmap.
## Test Guidance
Focused tests:
- `go test ./internal/config -v`
- `go test ./internal/app -v`
- `go test ./internal/stage -v`
- `go test ./internal/artifacts -v`
Full validation:
- `go test ./...`
Config tests to add or update:
- `publish.outputs` defaults and validates.
- `publish.outputs[].dest` derives from the artifact registry when omitted.
- `publish.locks` validates with the same source rules as publish outputs.
- old `archive` fails strict decode.
- old `promote_artifacts` fails strict decode.
- old cleanup fields fail strict decode.
App and stage tests to add or update:
- stage order uses `publish` before `notify`.
- `run-stage publish` succeeds.
- `run-stage archive` fails clearly.
- `narratio publish` force-runs the `publish` stage.
- `--artifacts` is accepted for `run-stage publish`.
- `--artifacts` error text names `analyze` and `publish`.
- status and artifact list output show `Published:` and `remote=published`.
- lock output says `published` or `not-published`.
- previous-cache and restore use `publish` stage metadata.
Final searches:
- Config/stage names:
- `pipeline.archive`
- `archive:`
- `promote_artifacts`
- `cleanup_after_archive`
- `delete_audio_after_archive`
- User-facing output:
- `Promoted:`
- `remote=promoted`
- `not-promoted`
- Runtime symbols and metadata:
- `ArchiveConfig`
- `ArchivePromotionRule`
- `S3PromotedArtifactKey`
- `promoted_paths`
- `promoted_files_uploaded`
- `locked_promotions`
Expected remaining matches should be limited to this roadmap and narrowly
justified historical references until the roadmap is fully retired.
## Architecture Guardrails
- Keep Narratio explicit and stage-driven.
- Do not introduce a generic workflow or DAG abstraction.
- Keep strict YAML decoding.
- Keep remote path construction centralized.
- Keep storage details behind `storage.ObjectStore`.
- Keep AWS SDK types inside storage adapters.
- Preserve manifest-driven resume and restore behavior.
- Preserve current-state commit ordering with `current/run_id.txt` written
last.
- Keep raw secrets out of configs, manifests, logs, generated configs, and
publish metadata.
- Keep planned behavior only in this roadmap until implementation lands.
## Assumptions
- This is a breaking public/config/stage contract change.
- No compatibility aliases are retained.
- No migration logic is needed for in-progress local manifests.
- No migration logic is needed for old remote manifests.
- Existing remote objects are not moved or renamed.
- `publish` means uploading run history, writing configured published outputs,
and committing current state.
- `run history` is the preferred term for immutable per-run records under
`runs/<run_id>/`.
- `archive` remains acceptable only as a generic English concept in historical
roadmap context, not as a public Narratio command, config field, stage name,
or metadata term after implementation.

View File

@@ -1,210 +0,0 @@
# Roadmap: Transcript Artifact Naming
Status: Implemented
## Problem
Narratio's built-in transcript artifact names and canonical paths currently mix
operator-facing artifact meaning with historical stage and tool terminology:
- `narratio.transcript.merged` maps to `transcripts/merged.json`.
- `narratio.transcript.polished` maps to `transcripts/processed.json`.
- `narratio.transcript.full` maps to `transcripts/normalized.json`.
- `narratio.transcript.trimmed` maps to `transcripts/trimmed.json`.
This makes the public artifact surface harder to reason about. Operators see
`full`, `normalized`, `processed`, `polished`, `merged`, and `trimmed` used in
different places for the same transcript lineage.
The transcript source IDs, canonical paths, and manifest output kinds should
use one vocabulary based on each transcript's role in the session artifact
model.
## Target Model
Built-in transcript artifacts should use these public source IDs, canonical
paths, and manifest output kinds:
| Source ID | Canonical path | Output kind | Meaning |
| --- | --- | --- | --- |
| `narratio.transcript.base` | `transcripts/base.json` | `transcript_base` | First unified transcript produced by merging per-speaker raw transcripts. |
| `narratio.transcript.polished` | `transcripts/polished.json` | `transcript_polished` | Audita-polished transcript. |
| `narratio.transcript.final` | `transcripts/final.json` | `transcript_final` | Full final transcript after normalization. |
| `narratio.transcript.final_trimmed` | `transcripts/final.trimmed.json` | `transcript_final_trimmed` | Trimmed version of the final transcript. |
Stage names remain process-oriented and unchanged:
- `merge`
- `polish`
- `normalize`
- `trim`
Downstream adapter contracts also remain process-oriented. The rename changes
Narratio's artifact model, canonical paths, config examples, archive promotion
sources, lock sources, status output, and documentation. It should not rename
the stages themselves or move external integration details into stage logic.
## Compatibility Policy
This is a hard cutover.
After implementation, these old source IDs should be rejected:
- `narratio.transcript.merged`
- `narratio.transcript.full`
- `narratio.transcript.trimmed`
These old canonical paths should not be compatibility fallbacks:
- `transcripts/merged.json`
- `transcripts/processed.json`
- `transcripts/normalized.json`
- `transcripts/trimmed.json`
Existing remote archives are not migrated automatically. Operators who want
new promoted keys for old sessions should republish those sessions after
updating configuration.
## Implementation Stages
### Stage 1: Centralize Transcript Artifact Naming
Status: Implemented
Consolidate transcript artifact source IDs, canonical paths, and output kinds
in the artifact/path layer before changing runtime behavior.
Implementation requirements:
- Add or consolidate constants/helpers for built-in transcript source IDs.
- Add or consolidate constants/helpers for canonical transcript paths.
- Add or consolidate constants/helpers for transcript manifest output kinds.
- Keep source ID, path, and output-kind mappings in one registry or one
obviously shared artifact model.
- Update artifact registry tests to prove the target mapping.
- Avoid changing stage output behavior in this stage unless the implementation
is simpler and still reviewable.
Acceptance criteria:
- There is one clear source of truth for built-in transcript artifact names,
paths, and output kinds.
- Tests prove the new target mapping in the artifact layer.
- No generic workflow abstraction is introduced.
### Stage 2: Rename Runtime Outputs and Defaults
Status: Implemented
Switch runtime behavior to the new transcript artifact model.
Implementation requirements:
- Update `merge` to write and record `transcripts/base.json` with
`transcript_base`.
- Update `polish` to write and record `transcripts/polished.json` with
`transcript_polished`.
- Update `normalize` to write and record `transcripts/final.json` with
`transcript_final`.
- Update `trim` to write and record `transcripts/final.trimmed.json` with
`transcript_final_trimmed`.
- Update normalize and trim defaults to:
- `pipeline.normalize.output_path: transcripts/final.json`
- `pipeline.trim.output_path: transcripts/final.trimmed.json`
- Update built-in artifact resolution, archive promotion destination
derivation, archive locks, status output, artifact catalog output,
previous-cache resolution, restore planning, and restore execution to use
the new registry values.
- Ensure old source IDs fail config validation.
Acceptance criteria:
- New runs produce the target canonical transcript files.
- Manifest outputs use the target output kinds.
- Archive promotion and lock validation accept new source IDs and reject old
source IDs.
- Status and artifact listing display new source IDs.
- Restore uses the new canonical paths and does not restore old transcript
paths as canonical outputs.
### Stage 3: Update Tests, Examples, and Current Documentation
Status: Implemented
Update all implemented-behavior references after the runtime cutover lands.
Implementation requirements:
- Update examples to use `narratio.transcript.final_trimmed` and
`transcripts/final.trimmed.json` where trimmed final transcript is intended.
- Update examples that refer to full final transcripts to use
`narratio.transcript.final` and `transcripts/final.json`.
- Update `docs/config.md`, `docs/internal/artifacts.md`, stage docs,
CLI examples, operations examples, archive examples, lock examples, and
status/artifact-list examples.
- Add strict validation tests proving old source IDs are rejected.
- Mark roadmap stages implemented only after code, tests, examples, and
current-behavior docs agree.
Acceptance criteria:
- Maintained examples load and validate.
- Current-behavior docs describe only implemented new names.
- Old names remain only in this roadmap as historical/planning context until
this roadmap is retired or archived.
## Test Guidance
Run focused tests while implementing:
- `go test ./internal/artifacts -v`
- `go test ./internal/config -v`
- `go test ./internal/stage -v`
- `go test ./internal/app -v`
Run full validation before finishing:
- `go test ./...`
Run final searches:
- Old source IDs:
- `narratio.transcript.merged`
- `narratio.transcript.full`
- `narratio.transcript.trimmed`
- Old paths:
- `transcripts/merged.json`
- `transcripts/processed.json`
- `transcripts/normalized.json`
- `transcripts/trimmed.json`
- Old output kinds:
- `transcript_merged`
- `transcript_processed`
- `transcript_normalized`
- `transcript_trimmed`
Expected remaining matches should be limited to this roadmap's
historical/planning references until the roadmap is fully completed.
## Architecture Guardrails
- Keep Narratio explicit and stage-driven; do not introduce a generic workflow
or DAG abstraction.
- Keep path and artifact naming in centralized helpers rather than scattered
string concatenation.
- Preserve manifest-driven resume behavior.
- Keep storage details behind storage adapters.
- Do not move Seriatim, Audita, or Scriptorium command details out of their
adapter boundaries.
- Keep current-behavior documentation in sync only after implementation lands;
planned behavior belongs in this roadmap until then.
## Assumptions
- The cutover is intentionally not backward-compatible.
- Existing remote archive objects are not renamed or migrated automatically.
- Stage names and downstream adapter request field names remain unchanged.
- The term `base` is preferred over `merged` for the first unified transcript.
- The term `final` is preferred over `full` or `normalized` for the full final
transcript.
- The trimmed final path is `transcripts/final.trimmed.json`.