# Roadmap: Publish Contract Status: Planned ## Problem Narratio currently uses several terms for one operator-facing concept: - `archive` is the stage that uploads run state and commits remote current state. - `publish` is the convenience command that force-runs the archive stage. - `promote`, `promoted`, and `promote_artifacts` describe configured top-level remote output writes. This mixed vocabulary makes the public contract harder to explain. Operators should not need to distinguish "archive the run", "publish the run", and "promote artifacts" when these are all part of the same publish action. The public model should use: - `publish` for the stage, command, config section, and action; - `published` for an expected remote output that exists at its top-level current destination; - `publish rules` for the configured source-to-destination output rules; - `locked` for sources whose top-level published destination must not be overwritten; - `run history` for immutable per-run records under `runs//`. ## Target Model The public stage is `publish`. The convenience command: narratio publish is equivalent to: narratio run-stage publish --force Pipeline configuration uses `publish`: publish: enabled: true upload_run: true outputs: - source: narratio.transcript.final_trimmed - source: narratio.artifact.session_recap locks: - source: narratio.artifact.session_recap reason: Final recap was manually edited. Publish output rules are source-based. Each rule writes one artifact source to a top-level remote destination. If `dest` is omitted, Narratio derives the destination from the artifact registry or configured artifact output path. The mutable remote lock store remains: {session_prefix}/locks.yml Remote availability output uses `published`: Published: - narratio.transcript.final_trimmed remote=published - narratio.artifact.session_recap locked remote=published The remote key layout is otherwise unchanged: - immutable run history stays under `{session_prefix}/runs/{run_id}/`; - current state stays under `{session_prefix}/current/manifest.json`; - the final commit marker stays `{session_prefix}/current/run_id.txt`; - `current/run_id.txt` is still written last. ## Compatibility Policy This is a hard cutover. After implementation: - `pipeline.archive` is rejected by strict YAML decoding. - `pipeline.archive.promote_artifacts` is rejected. - `pipeline.workspace.cleanup_after_archive` is rejected. - `pipeline.spool.delete_audio_after_archive` is rejected. - `narratio run-stage archive ` is an unknown stage. - manifests that record an `archive` stage are not migrated. - old archive/promotion metadata keys are not read as compatibility fallbacks. Existing remote objects are not moved or renamed. Remote layout remains stable; the rename changes configuration, stage names, status output, metadata, helper names, tests, examples, and documentation. ## Implementation Stages ### Stage 1: Public Schema and Stage Cutover Status: Planned Switch the public config and stage contract to publish terminology. Implementation requirements: - Replace `pipeline.archive` with `pipeline.publish`. - Replace `archive.promote_artifacts` with `publish.outputs`. - Keep output rule fields: - `source` - `dest` - `required` - Replace `pipeline.archive.locks` with `pipeline.publish.locks`. - Rename post-publish cleanup fields: - `pipeline.workspace.cleanup_after_publish` - `pipeline.spool.delete_audio_after_publish` - Rename the registered stage from `archive` to `publish`. - Update stage order so `publish` runs after `analyze` and before `notify`. - Update top-level `narratio publish` to target stage `publish`. - Keep `run-stage --artifacts publish` support. - Reject `run-stage --artifacts ` for stages other than `analyze` and `publish`. - Preserve the remote commit ordering and storage adapter boundaries. Acceptance criteria: - `narratio run-stage publish ` executes the publish stage. - `narratio publish ` force-runs the publish stage. - `narratio run-stage archive ` fails clearly as an unknown stage. - Old archive config fields fail strict decoding. - New publish config fields load, default, and validate. ### Stage 2: Runtime Terminology and Metadata Cutover Status: Planned Rename implementation concepts and runtime output to publish terminology. Implementation requirements: - Rename archive/promotion config and runtime types conceptually to publish/output terms. - Rename the remote key helper intent from promoted artifact to published output while keeping generated keys unchanged. - Change helper output: - `Promoted:` becomes `Published:` - `remote=promoted` becomes `remote=published` - lock output uses `published` / `not-published` - Rename publish-stage metadata, including: - `promoted_paths` to `published_paths` - `promoted_files_uploaded` to `published_files_uploaded` - `skipped_optional_promotions` to `skipped_optional_outputs` - `skipped_unselected_promotions` to `skipped_unselected_outputs` - `locked_promotion_count` to `locked_output_count` - `locked_promotions` to `locked_outputs` - Update previous-cache and restore logic to use the `publish` stage and `published_paths` metadata only. - Keep run-local stage output materialization separate from remote publish terminology. If local helper names are confusing, rename them to materialization-oriented names rather than publish names. Acceptance criteria: - Status and artifact helper output use `Published:` and `remote=published`. - Publish metadata contains only publish/output terminology. - Previous-cache and restore behavior works with publish metadata and does not depend on old archive metadata. - Storage adapters still receive explicit keys and no AWS SDK details leak into app or stage logic. ### Stage 3: Documentation, Examples, and Final Cleanup Status: Planned Update implemented-behavior docs and remove stale public terminology after the runtime cutover lands. Implementation requirements: - Update current-behavior docs: - `docs/config.md` - `docs/cli.md` - `docs/operations.md` - `docs/troubleshooting.md` - `docs/architecture.md` - relevant files under `docs/internal/` - Rename `docs/internal/stage-archive.md` to `docs/internal/stage-publish.md`. - Update internal documentation links and references. - Update examples to use: - `publish.outputs` - `publish.locks` - `cleanup_after_publish` - `delete_audio_after_publish` - Update tests and final searches so old terminology remains only in this roadmap as historical context. Acceptance criteria: - Maintained examples load and validate. - Current-behavior docs describe only implemented publish terminology. - Internal docs describe run history, published outputs, locks, and current commit ordering clearly. - Old user-facing archive/promote wording is removed except where discussing historical behavior in this roadmap. ## Test Guidance Focused tests: - `go test ./internal/config -v` - `go test ./internal/app -v` - `go test ./internal/stage -v` - `go test ./internal/artifacts -v` Full validation: - `go test ./...` Config tests to add or update: - `publish.outputs` defaults and validates. - `publish.outputs[].dest` derives from the artifact registry when omitted. - `publish.locks` validates with the same source rules as publish outputs. - old `archive` fails strict decode. - old `promote_artifacts` fails strict decode. - old cleanup fields fail strict decode. App and stage tests to add or update: - stage order uses `publish` before `notify`. - `run-stage publish` succeeds. - `run-stage archive` fails clearly. - `narratio publish` force-runs the `publish` stage. - `--artifacts` is accepted for `run-stage publish`. - `--artifacts` error text names `analyze` and `publish`. - status and artifact list output show `Published:` and `remote=published`. - lock output says `published` or `not-published`. - previous-cache and restore use `publish` stage metadata. Final searches: - Config/stage names: - `pipeline.archive` - `archive:` - `promote_artifacts` - `cleanup_after_archive` - `delete_audio_after_archive` - User-facing output: - `Promoted:` - `remote=promoted` - `not-promoted` - Runtime symbols and metadata: - `ArchiveConfig` - `ArchivePromotionRule` - `S3PromotedArtifactKey` - `promoted_paths` - `promoted_files_uploaded` - `locked_promotions` Expected remaining matches should be limited to this roadmap and narrowly justified historical references until the roadmap is fully retired. ## Architecture Guardrails - Keep Narratio explicit and stage-driven. - Do not introduce a generic workflow or DAG abstraction. - Keep strict YAML decoding. - Keep remote path construction centralized. - Keep storage details behind `storage.ObjectStore`. - Keep AWS SDK types inside storage adapters. - Preserve manifest-driven resume and restore behavior. - Preserve current-state commit ordering with `current/run_id.txt` written last. - Keep raw secrets out of configs, manifests, logs, generated configs, and publish metadata. - Keep planned behavior only in this roadmap until implementation lands. ## Assumptions - This is a breaking public/config/stage contract change. - No compatibility aliases are retained. - No migration logic is needed for in-progress local manifests. - No migration logic is needed for old remote manifests. - Existing remote objects are not moved or renamed. - `publish` means uploading run history, writing configured published outputs, and committing current state. - `run history` is the preferred term for immutable per-run records under `runs//`. - `archive` remains acceptable only as a generic English concept in historical roadmap context, not as a public Narratio command, config field, stage name, or metadata term after implementation.