214 lines
6.7 KiB
Markdown
214 lines
6.7 KiB
Markdown
# Operations
|
|
|
|
This guide describes the implemented operator lifecycle for Narratio.
|
|
|
|
For field-level configuration, see [docs/config.md](./config.md). For full command/flag reference, see [docs/cli.md](./cli.md).
|
|
|
|
## Normal workflow (S3-first path)
|
|
|
|
1. Upload session `.flac` files to object storage under the configured session audio prefix.
|
|
2. Run Narratio:
|
|
|
|
```bash
|
|
narratio run --session-id 2026-04-04
|
|
```
|
|
|
|
3. Read success output:
|
|
- `narratio run: session <session_id>; executed=<n> skipped=<n>; manifest=<path>`
|
|
- use `manifest=<path>` with `status` for inspection.
|
|
|
|
Notes:
|
|
- default config/session discovery applies unless `--config` and `--session` are passed.
|
|
- S3 audio mode requires `session.inputs.audio_s3.prefix` and valid object-store access.
|
|
|
|
## Restore workflow
|
|
|
|
Use restore when local durable session state is missing or stale and archive current state is authoritative.
|
|
|
|
Dry-run (no local writes):
|
|
|
|
```bash
|
|
narratio restore --session-id 2026-04-04 --dry-run
|
|
```
|
|
|
|
Execution:
|
|
|
|
```bash
|
|
narratio restore --session-id 2026-04-04
|
|
```
|
|
|
|
Post-restore analyze rerun pattern:
|
|
|
|
```bash
|
|
narratio run-stage --session-id 2026-04-04 --force analyze
|
|
```
|
|
|
|
Restore source-of-truth:
|
|
- remote commit marker: `current/run_id.txt`
|
|
- remote current manifest: `current/manifest.json`
|
|
|
|
Restore default scope:
|
|
- includes `manifest.json`, `transcripts/**`, `artifacts/**`
|
|
- includes `audio/**` only with `--include-audio`
|
|
- excludes `runs/**`, `logs/**`, `reports/**`, `config/**`, `inputs/**`, and `current/**` (except remote `current/manifest.json` as source)
|
|
|
|
## Local filesystem layout and state artifacts
|
|
|
|
Session root:
|
|
- `{workspace.root}/work/{campaign}/{session_id}/`
|
|
|
|
Primary state:
|
|
- `manifest.json`: session-level stage state.
|
|
- `runs/{run_id}/manifest.json`: invocation-level state.
|
|
- `.lock`: session lock while a modifying command is active.
|
|
|
|
Canonical session directories:
|
|
- `inputs/`
|
|
- `audio/`
|
|
- `transcripts/`
|
|
- `artifacts/`
|
|
- `reports/`
|
|
- `logs/`
|
|
- `config/`
|
|
- `current/`
|
|
- `runs/`
|
|
|
|
Run-local stage directories:
|
|
- `runs/{run_id}/{stage}/` with stage-local `outputs/`, `logs/`, `reports/`, `config/`, `scratch/`.
|
|
|
|
Behavior:
|
|
- directory creation is idempotent.
|
|
- stage outputs are generally generated run-local first, then promoted to canonical paths on success.
|
|
- restore installs downloaded files to canonical session paths and does not recreate historical run sandboxes.
|
|
|
|
## Analyze artifact execution lifecycle
|
|
|
|
Analyze executes configured artifacts from `pipeline.scriptorium.artifacts`.
|
|
|
|
Execution model:
|
|
- executable set = enabled artifacts, filtered by `--artifacts` when provided.
|
|
- artifact-to-artifact dependencies are declared via `depends_on`.
|
|
- selected artifacts run in deterministic dependency order.
|
|
- after each successful artifact run, output is promoted to configured canonical `output_path`.
|
|
|
|
Configured artifact source reuse:
|
|
- a non-executable configured artifact can satisfy inputs if its configured output file already exists and is valid.
|
|
- reused configured artifact provenance is `filesystem.disabled_artifact_output`.
|
|
|
|
`--artifacts` behavior:
|
|
- accepted on `run`, `resume`, and `run-stage analyze`.
|
|
- filters analyze execution only; does not force stage rerun.
|
|
|
|
## Remote archive layout and publish contract
|
|
|
|
When archive is enabled and run upload is enabled, archive publishes under:
|
|
|
|
- session prefix: `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/`
|
|
- run prefix: `{session_prefix}/runs/{run_id}/`
|
|
|
|
Archive uploads:
|
|
- run record files from run root (excluding `audio/`).
|
|
- promoted files from explicit `archive.promote_artifacts` rules.
|
|
|
|
Publish order:
|
|
1. upload `current/manifest.json`
|
|
2. upload `current/run_id.txt` last
|
|
|
|
`current/run_id.txt` is the remote commit marker.
|
|
|
|
Archive promotion is explicit and source-based:
|
|
- Narratio does not auto-promote all generated analyze artifacts.
|
|
- each rule resolves `source` through the artifact resolver/catalog model, then uploads to `dest`.
|
|
- missing required promotion sources fail archive stage.
|
|
- missing optional promotion sources are skipped.
|
|
- invalid resolved artifacts fail archive stage.
|
|
|
|
## Resume, retry, restore, and safe rerun behavior
|
|
|
|
Default skip:
|
|
- `run` and `run-stage` skip already-succeeded stages unless `--force` is set.
|
|
|
|
Resume:
|
|
- `resume` starts at first non-succeeded stage.
|
|
- `resume --force` runs full stage order.
|
|
|
|
Restore conflict policy:
|
|
- restore classifies local differences as conflicts.
|
|
- without `--force`, restore fails when conflicts exist.
|
|
- with `--force`, conflicting local files are overwritten by remote archive files.
|
|
|
|
Forced reruns:
|
|
- force-rerunning an upstream succeeded stage marks downstream succeeded stages as `stale`.
|
|
|
|
Safe rerun pattern:
|
|
1. rerun the changed stage with `--force`.
|
|
2. run `resume` to rebuild downstream stages.
|
|
|
|
## Cleanup behavior
|
|
|
|
Cleanup is considered only when archive stage executed and succeeded.
|
|
|
|
Cleanup toggles:
|
|
- `pipeline.spool.delete_audio_after_archive=true` deletes run-scoped spool audio.
|
|
- `pipeline.workspace.cleanup_after_archive=true` deletes run-scoped local run directory.
|
|
|
|
Cleanup eligibility gates:
|
|
- archive enabled
|
|
- archive run upload enabled
|
|
- run record upload completed
|
|
- current pointer write completed (`current/run_id.txt` written)
|
|
|
|
No cleanup for failed/incomplete/unarchived/archive-skipped runs.
|
|
|
|
## Failure and recovery playbooks
|
|
|
|
After run failure, Narratio keeps:
|
|
- session manifest
|
|
- run manifest
|
|
- run-local artifacts/logs/config/reports
|
|
|
|
Failed or incomplete runs remain local-only.
|
|
|
|
After restore failure:
|
|
- already-installed restore files remain in place.
|
|
- restore does not roll back prior successful installs.
|
|
- existing local manifest is preserved if restored manifest validation/install fails.
|
|
|
|
Recommended recovery:
|
|
|
|
1. inspect state:
|
|
|
|
```bash
|
|
narratio status --manifest <manifest-path>
|
|
```
|
|
|
|
2. for restore-specific checks, run:
|
|
|
|
```bash
|
|
narratio restore --session-id 2026-04-04 --dry-run
|
|
```
|
|
|
|
3. fix root cause (config/input/credentials/storage/service availability).
|
|
4. continue with `resume`, or targeted `run-stage --force` followed by `resume`.
|
|
|
|
## Restore report
|
|
|
|
Non-dry-run restore writes a durable report at:
|
|
- `reports/restore-latest.json`
|
|
|
|
Report content includes:
|
|
- identity (`campaign`, `session_id`, `run_id`)
|
|
- mode flags (`dry_run`, `force`, `include_audio`)
|
|
- plan counts and execution counts
|
|
- per-action status
|
|
|
|
Dry-run does not write restore report files.
|
|
|
|
## Operational caveats
|
|
|
|
- `status` requires explicit `--manifest`; there is no session-id lookup command.
|
|
- local and S3 audio input modes are mutually exclusive.
|
|
- archive publish requires upstream stages through `analyze` to be `succeeded`.
|
|
- required promotion rules can fail when selected analyze artifacts did not generate a required file path.
|
|
- restore requires configured remote object storage and committed remote current state.
|