Rewrite user and operator documentation for current CLI and config behavior
This commit is contained in:
@@ -1,72 +1,156 @@
|
||||
# Operations
|
||||
# Operations Guide
|
||||
|
||||
This guide covers the implemented operator lifecycle for Narratio.
|
||||
Operator workflow for running, recovering, and publishing Narratio sessions.
|
||||
|
||||
For field-level settings, see [docs/config.md](./config.md). For syntax/flags, see [docs/cli.md](./cli.md).
|
||||
For command syntax, see [docs/cli.md](./cli.md). For field-level config, see [docs/config.md](./config.md).
|
||||
|
||||
## Normal Workflow
|
||||
## Standard Session Workflow
|
||||
|
||||
1. Ensure `pipeline.yml`, `campaign.yml`, and `session.yml` are available.
|
||||
2. Ensure session audio is available (local `audio_dir`/`audio_files` or S3 prefix).
|
||||
3. Run:
|
||||
1. Select pipeline/campaign/session config.
|
||||
2. Validate session readiness:
|
||||
|
||||
```bash
|
||||
narratio session validate 2026-04-04
|
||||
```
|
||||
|
||||
3. (Optional) inspect stage decisions:
|
||||
|
||||
```bash
|
||||
narratio session plan 2026-04-04
|
||||
```
|
||||
|
||||
4. Run the pipeline:
|
||||
|
||||
```bash
|
||||
narratio run 2026-04-04
|
||||
```
|
||||
|
||||
4. Inspect status:
|
||||
5. Check state:
|
||||
|
||||
```bash
|
||||
narratio session status 2026-04-04
|
||||
```
|
||||
|
||||
## Campaign and Session Selection
|
||||
|
||||
Campaign selection priority:
|
||||
|
||||
- `--campaign-file`
|
||||
- `--campaign`
|
||||
- `pipeline.campaigns.default_campaign_id`
|
||||
|
||||
Session source priority:
|
||||
|
||||
- `--session`
|
||||
- local default search paths
|
||||
- remote session object (S3) when local session file is not found and storage is configured
|
||||
|
||||
## Session Initialization
|
||||
|
||||
Use `session init` to generate a concrete session file for local or remote use.
|
||||
|
||||
Local file:
|
||||
|
||||
```bash
|
||||
narratio session init 2026-04-04 --output ./session.yml --date 2026-04-04 --title "Session 12"
|
||||
```
|
||||
|
||||
Remote session object:
|
||||
|
||||
```bash
|
||||
narratio session init 2026-04-04 --remote --force
|
||||
```
|
||||
|
||||
If `campaign.yml` sets `session_template_file`, `session init` renders it. Template variables must resolve to concrete values.
|
||||
|
||||
## Stage Execution and Resume Behavior
|
||||
|
||||
Canonical stage order:
|
||||
|
||||
1. `prepare`
|
||||
2. `transcribe`
|
||||
3. `merge`
|
||||
4. `polish`
|
||||
5. `normalize`
|
||||
6. `trim`
|
||||
7. `analyze`
|
||||
8. `publish`
|
||||
9. `notify`
|
||||
|
||||
Execution rules:
|
||||
|
||||
- succeeded stages are skipped unless `--force` is set;
|
||||
- `resume` starts at first non-succeeded stage;
|
||||
- force rerunning a succeeded upstream stage marks succeeded downstream stages as `stale`.
|
||||
|
||||
Single-stage execution:
|
||||
|
||||
```bash
|
||||
narratio run-stage normalize 2026-04-04 --force
|
||||
```
|
||||
|
||||
## Artifact Selection
|
||||
|
||||
`--artifacts` can be used on `run`, `resume`, `run-stage`, `analyze`, and `publish`.
|
||||
|
||||
Selection behavior:
|
||||
|
||||
- validates names against `pipeline.scriptorium.artifacts`;
|
||||
- filters analyze execution to selected configured artifacts;
|
||||
- filters publish rules for `narratio.artifact.<name>` sources only;
|
||||
- does not suppress built-in transcript or bounds publish sources.
|
||||
|
||||
## Publish Workflow
|
||||
|
||||
Publish is the stage that commits remote current state.
|
||||
Run publish only:
|
||||
|
||||
```bash
|
||||
narratio publish 2026-04-04
|
||||
```
|
||||
|
||||
Equivalent command:
|
||||
Equivalent:
|
||||
|
||||
```bash
|
||||
narratio run-stage publish 2026-04-04 --force
|
||||
```
|
||||
|
||||
Publish uploads:
|
||||
Publish commit model:
|
||||
|
||||
- run history files under `{session_prefix}/runs/{run_id}/` (excluding `audio/`)
|
||||
- configured published outputs from `pipeline.publish.outputs`
|
||||
- `previous/**` cache files when present
|
||||
- `current/manifest.json`
|
||||
- `current/run_id.txt` last
|
||||
- uploads run files under `{session_prefix}/runs/{run_id}/`;
|
||||
- uploads configured published outputs;
|
||||
- uploads `previous/**` cache files when present;
|
||||
- writes `current/manifest.json`;
|
||||
- writes `current/run_id.txt` last.
|
||||
|
||||
`current/run_id.txt` is the remote commit marker.
|
||||
`current/run_id.txt` is the remote current-state commit marker.
|
||||
|
||||
## Published Outputs and Locks
|
||||
## Publish Locks
|
||||
|
||||
Published output behavior:
|
||||
Lock sources:
|
||||
|
||||
- outputs are source-based rules in `pipeline.publish.outputs`.
|
||||
- required missing unlocked sources fail publish.
|
||||
- optional missing unlocked sources are skipped.
|
||||
- selected artifacts (`--artifacts`) only filter configured `narratio.artifact.<key>` output rules.
|
||||
- built-in transcript and bounds output rules are not filtered by `--artifacts`.
|
||||
- static locks in `pipeline.publish.locks`
|
||||
- mutable remote locks in `{session_prefix}/locks.yml`
|
||||
|
||||
Lock behavior:
|
||||
Effective lock rules:
|
||||
|
||||
- static locks: `pipeline.publish.locks`.
|
||||
- mutable locks: `{session_prefix}/locks.yml`.
|
||||
- effective lock set is static + mutable; static wins on duplicate sources.
|
||||
- locked outputs are intentional skips and do not fail publish.
|
||||
- lock commands mutate only remote mutable locks.
|
||||
- static and remote locks are merged;
|
||||
- static locks win on source collisions;
|
||||
- locked outputs are intentional skips;
|
||||
- lock add/remove commands mutate only remote lock state.
|
||||
|
||||
Examples:
|
||||
|
||||
```bash
|
||||
narratio session locks 2026-04-04
|
||||
narratio session locks add 2026-04-04 narratio.artifact.session_recap --reason "manual edits" --force
|
||||
narratio session locks remove 2026-04-04 narratio.artifact.session_recap
|
||||
```
|
||||
|
||||
## Restore Workflow
|
||||
|
||||
Use restore when local durable session state is missing/stale and committed remote current state is authoritative.
|
||||
Use restore when local durable session state is missing or stale and remote committed current state is authoritative.
|
||||
|
||||
Preview:
|
||||
Dry run:
|
||||
|
||||
```bash
|
||||
narratio session restore 2026-04-04 --dry-run
|
||||
@@ -83,21 +167,22 @@ Default restore scope:
|
||||
- `manifest.json`
|
||||
- `transcripts/**`
|
||||
- `artifacts/**`
|
||||
- `previous/**` when required by configured previous-session artifact inputs
|
||||
- `previous/**` when needed by configured previous-session artifact inputs
|
||||
|
||||
Optional:
|
||||
|
||||
- add `--include-audio` to restore `audio/**`.
|
||||
- `--include-audio` to include `audio/**`
|
||||
- `--force` to overwrite local conflicts
|
||||
|
||||
Restore reads committed current state only (`current/run_id.txt`, `current/manifest.json`).
|
||||
Restore writes an execution report at `reports/restore-latest.json`.
|
||||
|
||||
## Workspace and State Layout
|
||||
## Local State Layout
|
||||
|
||||
Session root:
|
||||
|
||||
- `{workspace.root}/work/{campaign}/{session_id}/`
|
||||
- `{workspace.root}/work/{campaign}/{session_id}`
|
||||
|
||||
Durable session state:
|
||||
Durable session paths:
|
||||
|
||||
- `manifest.json`
|
||||
- `inputs/**`
|
||||
@@ -110,61 +195,55 @@ Durable session state:
|
||||
- `config/**`
|
||||
- `runs/**`
|
||||
|
||||
Run-local stage layout:
|
||||
Run-local layout:
|
||||
|
||||
- `runs/{run_id}/{stage}/outputs|logs|reports|config|scratch`
|
||||
- `runs/{run_id}/{stage}/outputs`
|
||||
- `runs/{run_id}/{stage}/logs`
|
||||
- `runs/{run_id}/{stage}/reports`
|
||||
- `runs/{run_id}/{stage}/config`
|
||||
- `runs/{run_id}/{stage}/scratch`
|
||||
|
||||
Stages typically write run-local outputs first, then materialize canonical session outputs on success.
|
||||
Spool layout (runtime/transient):
|
||||
|
||||
## Resume and Force Rules
|
||||
- `{spool.root}/{campaign}/{session_id}/{run_id}/...`
|
||||
- restore audio spool under `{spool.root}/{campaign}/{session_id}/restore/audio`
|
||||
|
||||
- `run` and `run-stage` skip succeeded stages unless `--force` is set.
|
||||
- `resume` starts at the first non-succeeded stage.
|
||||
- force-rerunning an upstream succeeded stage marks downstream succeeded stages as `stale`.
|
||||
- `--force` does not bypass publish locks.
|
||||
Cache layout (durable S3 audio cache):
|
||||
|
||||
- `{cache.root}/s3/{bucket}/...`
|
||||
|
||||
## Cleanup
|
||||
|
||||
Automatic post-publish cleanup is considered only when publish executes successfully and commits current state.
|
||||
|
||||
Config toggles:
|
||||
|
||||
- `pipeline.spool.delete_audio_after_publish=true`
|
||||
- `pipeline.workspace.cleanup_after_publish=true`
|
||||
|
||||
Manual cleanup:
|
||||
Session-scoped cleanup:
|
||||
|
||||
```bash
|
||||
narratio clean 2026-04-04
|
||||
```
|
||||
|
||||
Global cleanup:
|
||||
|
||||
```bash
|
||||
narratio clean --all
|
||||
```
|
||||
|
||||
Cache is preserved by default. Use `--clear-cache` to remove matching S3 audio cache entries.
|
||||
|
||||
## Failure and Recovery
|
||||
|
||||
After stage failure, Narratio keeps manifests and run-local files for inspection.
|
||||
|
||||
Standard recovery flow:
|
||||
|
||||
1. inspect status:
|
||||
Dry-run and cache variants:
|
||||
|
||||
```bash
|
||||
narratio session status 2026-04-04
|
||||
narratio clean 2026-04-04 --dry-run --clear-cache
|
||||
narratio clean --all --dry-run --clear-cache
|
||||
```
|
||||
|
||||
2. if needed, inspect restore plan:
|
||||
Rules:
|
||||
|
||||
```bash
|
||||
narratio session restore 2026-04-04 --dry-run
|
||||
```
|
||||
|
||||
3. fix root cause.
|
||||
4. continue with `resume`, or rerun a stage with `--force` then `resume`.
|
||||
- `clean` deletes work/spool session state;
|
||||
- cache is preserved unless `--clear-cache` is set;
|
||||
- automatic post-publish cleanup is gated by successful publish commit plus:
|
||||
- `pipeline.spool.delete_audio_after_publish=true`
|
||||
- `pipeline.workspace.cleanup_after_publish=true`
|
||||
|
||||
## Operational Caveats
|
||||
|
||||
- local and S3 audio modes are mutually exclusive.
|
||||
- publish requires prerequisite stages through `analyze` to be `succeeded`.
|
||||
- restore requires configured object storage and committed current state.
|
||||
- `session status` and `session artifacts --remote` both report remote published-output availability when storage is configured.
|
||||
- Local and S3 audio modes are mutually exclusive.
|
||||
- Publish requires prerequisite stages through analyze to be succeeded.
|
||||
- Restore requires configured object storage and committed remote current state.
|
||||
- Storage-backed commands load filesystem secrets before object-store initialization.
|
||||
|
||||
Reference in New Issue
Block a user