eric/narratio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 15 Wiki Activity

Rewrite docs for the publish stage contract and current behavior

Browse Source
This commit is contained in:
Eric Rakestraw
2026-05-23 04:51:16 +00:00
parent 79737edf79
commit 71395bb076
22 changed files with 496 additions and 1174 deletions
Download Patch File Download Diff File Expand all files Collapse all files

285
docs/cli.md
View File

@@ -6,91 +6,64 @@
narratio run 2026-04-04
```
This command uses default system discovery for `pipeline.yml`, the pipeline default campaign ID, and local `session.yml`. If local session discovery misses and S3 storage is configured, the positional session ID loads remote `session.yml` from the canonical session prefix.
Default pipeline and session discovery checks system config locations only. Pass `--config`, `--campaign-file`, and `--session` to use files from the current working directory. Pass `--campaign <id>` to select a campaign from `pipeline.campaigns.root`.
Ordinary local and remote `session.yml` files must be concrete YAML. Templates belong to `narratio session init`, which renders a configured campaign template before writing the concrete file.
This runs the full pipeline for the given session ID using default config discovery and campaign selection.
## Command Overview
Top-level commands:
- `run <session_id>`: execute pipeline stages and persist manifest state.
- `run <session_id>`: execute the pipeline.
- `resume <session_id>`: continue from first non-succeeded stage.
- `run-stage <stage> <session_id>`: execute exactly one stage.
- `resume <session_id>`: continue from first non-succeeded stage unless forced.
- `analyze <session_id>`: force-rerun the analyze stage.
- `publish <session_id>`: force-rerun the archive stage.
- `analyze <session_id>`: force-rerun analyze stage.
- `publish <session_id>`: force-rerun publish stage.
- `clean <session_id>|--all`: remove local workspace/spool state.
- `session <subcommand>`: session-scoped helper commands.
Session subcommands:
- `session init <session_id>`: create local or remote `session.yml`.
- `session validate <session_id>`: run read-only preflight checks.
- `session status <session_id>`: inspect local/remote session state.
- `session plan <session_id>`: validate config, prepare workspace layout, and print stage run/skip decisions.
- `session restore <session_id>`: restore durable local state from committed remote archive state.
- `session artifacts <session_id>`: list effective artifact source IDs.
- `session locks <session_id>`: list archive promotion locks.
- `session locks add <session_id> <source>`: add or update a remote lock.
- `session locks remove <session_id> <source>`: remove a remote lock.
Unknown commands print usage and exit non-zero.
For config semantics, see [docs/config.md](./config.md). For operator lifecycle and recovery, see [docs/operations.md](./operations.md).
- `session init <session_id>`
- `session plan <session_id>`
- `session validate <session_id>`
- `session status <session_id>`
- `session restore <session_id>`
- `session artifacts <session_id>`
- `session locks <session_id>`
- `session locks add <session_id> <source>`
- `session locks remove <session_id> <source>`
## Common Flags
Most session-aware commands accept:
- `--config <path>`: optional explicit `pipeline.yml` path.
- `--campaign <id>`: optional campaign ID selector.
- `--campaign-file <path>`: optional explicit `campaign.yml` path.
- `--session <path>`: optional explicit concrete `session.yml` path.
- `--previous-session-id <value>`: expected previous session identifier.
- `--config <pipeline.yml>`
- `--campaign <id>`
- `--campaign-file <campaign.yml>`
- `--session <session.yml>`
- `--previous-session-id <id>`
The positional `<session_id>` is required even when `--session` is provided. It is used as the expected session identity and as the remote session lookup value when local session discovery misses.
`--campaign` and `--campaign-file` are mutually exclusive.
## Command Reference
### `run`
```bash
narratio run <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>] [--force] [--artifacts <name[,name...]>]
narratio run <session_id> [--force] [--artifacts <name[,name...]>] [...common flags]
```
Purpose:
- Execute configured stages in canonical order.
Success output:
- `narratio run: session <session_id>; executed=<n> skipped=<n>; manifest=<path>`
Common failure cases:
- missing system default config/session paths when flags are omitted.
- missing selected campaign under `pipeline.campaigns.root`.
- missing local session plus missing/unavailable remote `session.yml`.
- templated `session.yml`; run `narratio session init` to generate concrete YAML.
- concrete session identity mismatch.
- unknown configured artifact key in `--artifacts`.
Runs stages in canonical order and writes manifest state.
### `resume`
```bash
narratio resume <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>] [--force] [--artifacts <name[,name...]>]
narratio resume <session_id> [--force] [--artifacts <name[,name...]>] [...common flags]
```
Purpose:
- Continue from session-manifest stage status.
Success output:
- `narratio resume: session <session_id> has no remaining stages`
- or `narratio resume: session <session_id>; executed=<n> skipped=<n>; manifest=<path>`
Starts at the first non-succeeded stage from the session manifest.
### `run-stage`
```bash
narratio run-stage <stage> <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>] [--force] [--artifacts <name[,name...]>]
narratio run-stage <stage> <session_id> [--force] [--artifacts <name[,name...]>] [...common flags]
```
Valid stage names:
@@ -102,175 +75,130 @@ Valid stage names:
- `normalize`
- `trim`
- `analyze`
- `archive`
- `publish`
- `notify`
Success output:
- `narratio run-stage: stage=<name> executed=<n> skipped=<n> force=<true|false>; manifest=<path>`
`--artifacts` is accepted only for `analyze` and `archive`.
`--artifacts` is accepted only for `analyze` and `publish`.
### `analyze`
```bash
narratio analyze <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>] [--artifacts <name[,name...]>]
narratio analyze <session_id> [--artifacts <name[,name...]>] [...common flags]
```
Purpose:
- Force-rerun the analyze stage.
- Shorter equivalent for `narratio run-stage analyze <session_id> --force`.
`analyze` is force-by-design and does not accept `--force`.
Equivalent to `narratio run-stage analyze <session_id> --force`.
### `publish`
```bash
narratio publish <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>] [--artifacts <name[,name...]>]
narratio publish <session_id> [--artifacts <name[,name...]>] [...common flags]
```
Purpose:
- Force-rerun the archive stage.
- Shorter equivalent for `narratio run-stage archive <session_id> --force`.
`publish` is force-by-design and does not accept `--force` or a stage positional argument.
Equivalent to `narratio run-stage publish <session_id> --force`.
### `clean`
```bash
narratio clean <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>] [--dry-run] [--clear-cache]
narratio clean --all [--config <pipeline.yml>] [--dry-run] [--clear-cache]
narratio clean <session_id> [--dry-run] [--clear-cache] [...common flags]
narratio clean --all [--dry-run] [--clear-cache] [--config <pipeline.yml>]
```
Session cleanup deletes:
- `{workspace.root}/work/{campaign}/{session_id}`
- `{spool.root}/{campaign}/{session_id}`
All-session cleanup deletes:
- `{workspace.root}/work`
- the contents of `{spool.root}`, while preserving the spool root directory itself.
Cache behavior:
- cache is preserved by default.
- `--clear-cache` in session mode removes cached S3 audio files for the resolved session.
- `--all --clear-cache` removes the configured Narratio S3 audio cache namespace for the configured bucket/root prefix.
- `--clear-cache` does not delete arbitrary files under `pipeline.cache.root`.
- session mode deletes `{workspace.root}/work/{campaign}/{session_id}` and `{spool.root}/{campaign}/{session_id}`.
- `--all` deletes all session work and spool children.
- cache is preserved unless `--clear-cache` is passed.
### `session plan`
```bash
narratio session plan <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>] [--force]
narratio session plan <session_id> [--force] [...common flags]
```
Purpose:
- Validate config, load secrets if configured, prepare workdir, and print stage run/skip decisions.
Success output includes:
- `narratio session plan: workdir prepared at <path>`
- one line per stage (`<stage>: run|skip`)
- `totals: run=<n> skip=<n>`
### `session status`
```bash
narratio session status <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>]
```
Output includes:
- session ID, campaign, workspace, and session config source.
- local manifest state when present.
- remote current archive state when storage is configured.
- catalog-based promoted output availability for expected transcript and artifact sources.
- effective archive locks and conservative next actions.
Validates config and session inputs, prepares workdir layout, and prints stage run/skip decisions.
### `session validate`
```bash
narratio session validate <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>]
narratio session validate <session_id> [...common flags]
```
Checks include:
- effective config and session source.
- stable input files.
- local or remote audio availability.
- previous-session requirements.
- archive promotions and effective locks.
Read-only preflight checks for config, inputs, audio availability, previous-session requirements, publish outputs, and effective locks.
Warnings do not fail the command. Any `ERROR` finding exits non-zero.
### `session status`
```bash
narratio session status <session_id> [...common flags]
```
Shows local manifest state, remote current state (when storage is configured), published-output availability, and effective locks.
### `session init`
```bash
narratio session init <session_id> --output ./session.yml
narratio session init <session_id> --remote
narratio session init <session_id> --config <pipeline.yml> --campaign icewind --remote
narratio session init <session_id> --config <pipeline.yml> --campaign-file ./campaign.yml --remote
narratio session init <session_id> --remote --force
```
Additional flags:
Flags:
- `--previous-session-id <value>`
- `--date <value>`
- `--title <value>`
- `--audio-s3-prefix <prefix>`: defaults to `audio/` when neither audio flag is provided.
- `--audio-dir <path>`: local audio directory; mutually exclusive with `--audio-s3-prefix`.
- `--force`: overwrite existing local or remote target.
Behavior:
- exactly one of `--output` or `--remote` is required.
- `--config`, `--campaign`, and `--campaign-file` are optional overrides; omitted campaign selection uses `pipeline.campaigns.default_campaign_id`.
- `--campaign <id>` selects a campaign under `pipeline.campaigns.root`.
- `--campaign-file <path>` loads an explicit campaign file.
- if `campaign.yml` sets `session_template_file`, the template path is resolved relative to `campaign.yml` and rendered from init flags.
- if no session template is configured, a minimal concrete session file is generated directly.
- template variables must be supplied by matching flags, and supplied template-related flags must be used by the template.
- remote writes target `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/session.yml`.
- existing local or remote targets fail unless `--force` is passed.
- remote writes use existence checks, not compare-and-swap.
- `--output <path>` or `--remote` (exactly one is required)
- `--previous-session-id <id>`
- `--date <date>`
- `--title <title>`
- `--audio-dir <path>`
- `--audio-s3-prefix <prefix>`
- `--force`
- common config/campaign flags
### `session restore`
```bash
narratio session restore <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>] [--dry-run] [--force] [--include-audio]
narratio session restore <session_id> [--dry-run] [--force] [--include-audio] [...common flags]
```
Purpose:
- Restore durable session state from the committed remote archive current state.
- Default restore installs `manifest.json`, `transcripts/**`, and `artifacts/**` from the current session archive.
- When configured previous-session inputs require it, restore reconstructs `previous/**` from the previous session's committed current archive.
- `audio/**` is restored only with `--include-audio`.
Restores durable local session files from committed remote current state.
Dry-run output may include planned previous-cache downloads. Existing differing files under `previous/**` follow the normal restore conflict policy and require `--force` to overwrite.
Default restore scope:
When `--include-audio` is set, S3 audio files are restored through the shared audio cache. Cache hits avoid re-downloading large audio objects.
- `manifest.json`
- `transcripts/**`
- `artifacts/**`
- `previous/**` when required by configured previous-session artifact inputs
`audio/**` is restored only when `--include-audio` is set.
### `session artifacts`
```bash
narratio session artifacts <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>] [--remote]
narratio session artifacts <session_id> [--remote] [...common flags]
```
Purpose:
- List built-in, configured, previous-session, promoted, and locked artifact sources.
`--remote` checks promoted top-level object availability through the storage adapter. Remote markers appear only in the `Promoted` section, which reports each configured archive promotion destination and includes `dest=<path>` when that destination differs from the source's canonical path.
Lists built-in sources, configured artifact sources, previous-session sources, publish output rules, and lock status. With `--remote`, includes remote published-state markers.
### `session locks`
```bash
narratio session locks <session_id> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>]
narratio session locks add <session_id> <source> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>] [--reason <text>] [--force]
narratio session locks remove <session_id> <source> [--config <pipeline.yml>] [--campaign <id>] [--campaign-file <campaign.yml>] [--session <session.yml>] [--previous-session-id <id>]
narratio session locks <session_id> [...common flags]
narratio session locks add <session_id> <source> [--reason <text>] [--force] [...common flags]
narratio session locks remove <session_id> <source> [...common flags]
```
Behavior:
- list mode prints effective locks from static `pipeline.archive.locks` and remote `{session_prefix}/locks.yml`.
- `locks add` writes only the remote lock store and fails if the source is already locked by pipeline config.
- `locks remove` removes only remote locks and cannot remove static pipeline locks.
- `locks add --force` is required to update an existing remote lock reason.
- list mode prints effective locks from static `pipeline.publish.locks` and remote `{session_prefix}/locks.yml`.
- add/remove mutate only the remote lock store.
- static pipeline locks cannot be removed by lock commands.
## `--artifacts` Rules
- accepted on `run`, `resume`, `run-stage`, `analyze`, and `publish`.
- on `run-stage`, only valid for `analyze` and `publish`.
- filters configured analyze artifact execution.
- filters configured `pipeline.publish.outputs` entries for `narratio.artifact.<key>` sources.
- does not suppress built-in transcript/bounds publish outputs.
- does not imply `--force` for `run`, `resume`, or `run-stage`.
## Common Workflows
Default-discovery run:
Run full pipeline:
```bash
narratio run 2026-04-04
@@ -282,60 +210,21 @@ Run only selected analyze artifacts:
narratio run 2026-04-04 --artifacts session_recap,player_handout
```
Resume with selected analyze artifacts:
```bash
narratio resume 2026-04-04 --artifacts player_handout
```
Force-rerun analyze with selected artifacts:
Force analyze only:
```bash
narratio analyze 2026-04-04 --artifacts player_handout
```
Force-rerun archive publishing:
Force publish only:
```bash
narratio publish 2026-04-04
```
Preview restore actions without writes:
Restore preview then apply:
```bash
narratio session restore 2026-04-04 --dry-run
```
Restore and then force analyze:
```bash
narratio session restore 2026-04-04
narratio analyze 2026-04-04
```
Rehydrate canonical previous-session inputs after artifact-input changes:
```bash
narratio run-stage prepare 2026-04-04 --force
```
Reset local state before testing restore:
```bash
narratio clean 2026-04-04 --dry-run
narratio clean 2026-04-04
narratio session restore 2026-04-04 --include-audio
```
Clean all local sessions while keeping cached S3 audio:
```bash
narratio clean --all
```
## `--artifacts` and `--force`
- `--artifacts` filters which configured artifacts are executable when analyze runs and which configured artifact promotions archive publishes.
- `--artifacts` does not imply `--force`.
- if analyze is already `succeeded` and `--force` is not set, runner-level skip still applies.
- `--artifacts` does not suppress built-in transcript or bounds promotions.