eric/narratio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 15 Wiki Activity

Update documentation for the new analyze stage and artifact registry

Browse Source
This commit is contained in:
Eric Rakestraw
2026-05-19 19:42:28 -05:00
parent ebb21b9201
commit 574b1cde6c
11 changed files with 464 additions and 344 deletions
Download Patch File Download Diff File Expand all files Collapse all files

120
docs/cli.md
View File

@@ -6,44 +6,46 @@
narratio run --session-id 2026-04-04 narratio run --session-id 2026-04-04
``` ```
This uses default config discovery for `pipeline.yml` and `session.yml`; both files must be discoverable for this command to run. This command uses default config discovery for `pipeline.yml` and `session.yml`; both files must be discoverable unless you pass explicit `--config` and `--session` paths.
## Command Overview ## Command Overview
Implemented commands: Implemented commands:
- `run`: execute the full stage plan and persist manifest state. - `run`: execute pipeline stages and persist manifest state.
- `plan`: validate config, prepare workdir, and print run/skip decisions. - `plan`: validate config, prepare workspace layout, and print stage run/skip decisions.
- `resume`: continue from the first non-succeeded stage in the manifest. - `resume`: continue from first non-succeeded stage unless forced.
- `status`: read and print stage statuses from an existing manifest file. - `status`: read and print stage statuses from an existing manifest.
- `run-stage`: execute exactly one selected stage. - `run-stage`: execute exactly one stage.
Unknown commands print usage (`Usage: narratio <run|plan|status|resume|run-stage>`) and exit non-zero. Unknown commands print usage and exit non-zero.
For configuration field details, see [docs/config.md](./config.md). For operational lifecycle details, see [docs/operations.md](./operations.md). For config semantics, see [docs/config.md](./config.md). For operator lifecycle and recovery, see [docs/operations.md](./operations.md).
## Complete Flag Reference ## Complete Flag Reference
### `run` ### `run`
- `--config <path>`: optional explicit `pipeline.yml` path; if omitted, default locations are searched. - `--config <path>`: optional explicit `pipeline.yml` path.
- `--session <path>`: optional explicit `session.yml` path; if omitted, default locations are searched. - `--session <path>`: optional explicit `session.yml` path.
- `--session-id <value>`: session template variable value for `session.yml` rendering. - `--session-id <value>`: session template variable value.
- `--force`: force stage execution (prevents skip of already-succeeded stages). - `--force`: force stage execution.
- `--artifacts <names>`: analyze artifact keys to execute (repeatable or comma-separated).
### `plan` ### `plan`
- `--config <path>` - `--config <path>`
- `--session <path>` - `--session <path>`
- `--session-id <value>` - `--session-id <value>`
- `--force`: show forced run decisions instead of normal skip behavior. - `--force`
### `resume` ### `resume`
- `--config <path>` - `--config <path>`
- `--session <path>` - `--session <path>`
- `--session-id <value>` - `--session-id <value>`
- `--force`: run full stage order rather than starting at first non-succeeded stage. - `--force`
- `--artifacts <names>`: analyze artifact keys to execute (repeatable or comma-separated).
### `run-stage` ### `run-stage`
@@ -51,6 +53,7 @@ For configuration field details, see [docs/config.md](./config.md). For operatio
- `--session <path>` - `--session <path>`
- `--session-id <value>` - `--session-id <value>`
- `--force` - `--force`
- `--artifacts <names>`: analyze artifact keys to execute (repeatable or comma-separated).
- positional `<stage>`: required stage name. - positional `<stage>`: required stage name.
Valid stage names: Valid stage names:
@@ -74,31 +77,27 @@ Valid stage names:
### `run` ### `run`
Purpose: Purpose:
- Execute configured stages in canonical order.
- Validate configuration and execute all stages in canonical order.
Syntax: Syntax:
```bash ```bash
narratio run [--config <pipeline.yml>] [--session <session.yml>] [--session-id <id>] [--force] narratio run [--config <pipeline.yml>] [--session <session.yml>] [--session-id <id>] [--force] [--artifacts <name[,name...]>]
``` ```
Success output: Success output:
- `narratio run: session <session_id>; executed=<n> skipped=<n>; manifest=<path>` - `narratio run: session <session_id>; executed=<n> skipped=<n>; manifest=<path>`
Common failure cases: Common failure cases:
- missing default config/session paths when flags omitted.
- no pipeline config found in default search paths when `--config` is omitted. - invalid template/rendered session mismatch.
- no session config found in default search paths when `--session` is omitted. - unknown/invalid `--artifacts` value.
- invalid flags or unexpected positional arguments. - `--artifacts` with unknown configured artifact key.
- config/template/validation errors.
### `plan` ### `plan`
Purpose: Purpose:
- Validate config, load secrets (if configured), prepare workdir, and print stage run/skip decisions.
- Validate config, load secrets (if configured), prepare workspace layout, and print per-stage run/skip decisions.
Syntax: Syntax:
@@ -107,43 +106,38 @@ narratio plan [--config <pipeline.yml>] [--session <session.yml>] [--session-id
``` ```
Success output includes: Success output includes:
- `narratio plan: workdir prepared at <path>` - `narratio plan: workdir prepared at <path>`
- one line per stage (`<stage>: run|skip`) - one line per stage (`<stage>: run|skip`)
- `totals: run=<n> skip=<n>` - `totals: run=<n> skip=<n>`
Common failure cases: Common failure cases:
- same config/session discovery and validation failures as `run`.
- same discovery, template, and validation failures as `run`.
- secrets directory read failures when `pipeline.secrets.env_dir` is configured. - secrets directory read failures when `pipeline.secrets.env_dir` is configured.
### `resume` ### `resume`
Purpose: Purpose:
- Continue from session-manifest stage status.
- Continue execution from manifest state for the same session.
Syntax: Syntax:
```bash ```bash
narratio resume [--config <pipeline.yml>] [--session <session.yml>] [--session-id <id>] [--force] narratio resume [--config <pipeline.yml>] [--session <session.yml>] [--session-id <id>] [--force] [--artifacts <name[,name...]>]
``` ```
Success output: Success output:
- `narratio resume: session <session_id> has no remaining stages`
- either `narratio resume: session <session_id> has no remaining stages`
- or `narratio resume: session <session_id>; executed=<n> skipped=<n>; manifest=<path>` - or `narratio resume: session <session_id>; executed=<n> skipped=<n>; manifest=<path>`
Common failure cases: Common failure cases:
- same discovery/template/validation failures as `run`. - same discovery/template/validation failures as `run`.
- manifest load errors when an existing manifest is unreadable. - manifest load errors when existing manifest is unreadable.
- invalid or unknown artifact selections.
### `status` ### `status`
Purpose: Purpose:
- Inspect one manifest file without executing stages.
- Inspect an existing manifest file without running stages.
Syntax: Syntax:
@@ -152,37 +146,37 @@ narratio status --manifest <manifest.json>
``` ```
Success output includes: Success output includes:
- `session_id: <id>` - `session_id: <id>`
- `updated_at: <timestamp>` - `updated_at: <timestamp>`
- `stages:` section with `- <stage>: <status>` entries. - `stages:` entries (`- <stage>: <status>`)
Common failure cases: Common failure cases:
- missing `--manifest`. - missing `--manifest`.
- manifest path unreadable or invalid JSON shape. - unreadable or invalid manifest path.
### `run-stage` ### `run-stage`
Purpose: Purpose:
- Execute exactly one stage.
- Execute exactly one stage from the supported stage set.
Syntax: Syntax:
```bash ```bash
narratio run-stage [--config <pipeline.yml>] [--session <session.yml>] [--session-id <id>] [--force] <stage> narratio run-stage [--config <pipeline.yml>] [--session <session.yml>] [--session-id <id>] [--force] [--artifacts <name[,name...]>] <stage>
``` ```
Success output: Success output:
- `narratio run-stage: stage=<name> executed=<n> skipped=<n> force=<true|false>; manifest=<path>` - `narratio run-stage: stage=<name> executed=<n> skipped=<n> force=<true|false>; manifest=<path>`
Common failure cases: `--artifacts` behavior:
- accepted only when `<stage>` is `analyze`.
- names are normalized (trimmed, deduplicated, sorted).
- unknown configured artifact keys fail.
- missing stage positional argument. Common failure cases:
- missing stage positional arg.
- unknown stage name. - unknown stage name.
- same discovery/template/validation failures as `run`. - using `--artifacts` with any non-`analyze` stage.
## Common Workflows ## Common Workflows
@@ -192,39 +186,37 @@ Default-discovery run:
narratio run --session-id 2026-04-04 narratio run --session-id 2026-04-04
``` ```
Explicit config/session run: Run only selected analyze artifacts:
```bash ```bash
narratio run --config /etc/narratio/pipeline.yml --session ./session.yml --session-id 2026-04-04 narratio run --session-id 2026-04-04 --artifacts session_recap,player_handout
``` ```
Plan before run: Resume with selected analyze artifacts:
```bash ```bash
narratio plan --config /etc/narratio/pipeline.yml --session ./session.yml --session-id 2026-04-04 narratio resume --session-id 2026-04-04 --artifacts player_handout
``` ```
Resume interrupted work: Run only analyze stage with selected artifacts:
```bash ```bash
narratio resume --config /etc/narratio/pipeline.yml --session ./session.yml --session-id 2026-04-04 narratio run-stage --session-id 2026-04-04 --artifacts player_handout analyze
```
Run one stage:
```bash
narratio run-stage --config /etc/narratio/pipeline.yml --session ./session.yml --session-id 2026-04-04 polish
``` ```
## Diagnostic / Recovery Commands ## Diagnostic / Recovery Commands
Read stage status from a manifest: Inspect stage status:
```bash ```bash
narratio status --manifest <manifest.json> narratio status --manifest <manifest.json>
``` ```
How to get manifest path: Get manifest path from previous output:
- `run`, `resume`, and `run-stage` print `manifest=<path>` on success.
- `run`, `resume`, and `run-stage` success output includes `manifest=<path>`. ## `--artifacts` and `--force`
- use that path with `status` for direct inspection.
- `--artifacts` filters which configured artifacts are executable when analyze runs.
- `--artifacts` does not imply `--force`.
- If analyze is already `succeeded` and `--force` is not set, runner-level skip still applies.

94
docs/config.md
View File

@@ -14,45 +14,42 @@ These commands load and validate both files before running:
- `narratio resume` - `narratio resume`
- `narratio run-stage` - `narratio run-stage`
Configuration behavior: Behavior:
- strict YAML decode is enabled (`KnownFields(true)`): unknown fields fail. - strict YAML decode is enabled (`KnownFields(true)`): unknown fields fail.
- session templates are rendered before session YAML decode. - session templates render before session YAML decode.
- defaults are applied for many optional pipeline fields. - defaults are applied for optional pipeline fields.
- validation enforces required fields, value formats, and cross-field constraints. - validation enforces required fields, value formats, and cross-field constraints.
## 2. Config file discovery ## 2. Config file discovery
Pipeline config lookup for `run`, `plan`, `resume`, and `run-stage`: Pipeline config lookup for `run`, `plan`, `resume`, and `run-stage`:
- If `--config <path>` is provided, that explicit path is used. - If `--config <path>` is provided, that path is used.
- If `--config` is omitted, Narratio searches in order: - If omitted, Narratio searches in order:
1. `/usr/local/etc/narratio/pipeline.yml` 1. `/usr/local/etc/narratio/pipeline.yml`
2. `/etc/narratio/pipeline.yml` 2. `/etc/narratio/pipeline.yml`
- The first existing file wins. - First existing file wins.
- If none exist, the command fails with a searched-paths error.
## 3. Session file discovery and templating ## 3. Session file discovery and templating
Session config lookup for `run`, `plan`, `resume`, and `run-stage`: Session config lookup for `run`, `plan`, `resume`, and `run-stage`:
- If `--session <path>` is provided, that explicit path is used. - If `--session <path>` is provided, that path is used.
- If `--session` is omitted, Narratio searches in order: - If omitted, Narratio searches in order:
1. `./session.yml` 1. `./session.yml`
2. `/usr/local/etc/narratio/session.yml` 2. `/usr/local/etc/narratio/session.yml`
3. `/etc/narratio/session.yml` 3. `/etc/narratio/session.yml`
- The first existing file wins. - First existing file wins.
- If none exist, the command fails and asks you to pass `--session`.
Session templating: Template behavior:
- Supported placeholders: - Supported placeholders:
- `{{session_id}}` - `{{session_id}}`
- `{{ session_id }}` - `{{ session_id }}`
- `--session-id <value>` supplies the template value. - `--session-id <value>` supplies the placeholder value.
- Unresolved placeholders fail with a template-rendering error. - unresolved placeholders fail load.
- If `--session-id` is provided and rendered `session_id` differs, load fails with a mismatch error. - if rendered `session_id` mismatches `--session-id`, load fails.
- Strict YAML decode still applies after template rendering.
## 4. Minimal pipeline config ## 4. Minimal pipeline config
@@ -64,9 +61,8 @@ whisperx:
Why this is sufficient: Why this is sufficient:
- `whisperx.transcribe_url` is required. - `whisperx.transcribe_url` is required.
- `workspace.root` is optional and defaults to `/var/lib/narratio`. - `workspace.root` defaults to `/var/lib/narratio`.
- Seriatim and Audita sections may be omitted; defaults are applied. - optional sections (`seriatim`, `audita`, `archive`, `scriptorium`, `trim`, `normalize`, etc.) receive defaults or stay inactive.
- Archive, storage, spool, normalize, and other optional sections get defaults when omitted.
## 5. Minimal session template ## 5. Minimal session template
@@ -119,18 +115,26 @@ archive:
whisperx: whisperx:
transcribe_url: "https://transcription.example.com/transcribe" transcribe_url: "https://transcription.example.com/transcribe"
scriptorium:
artifacts:
session_recap:
enabled: true
prompt_id: dnd.session_recap
output_path: artifacts/session_recap.md
inputs:
transcript:
source: narratio.transcript.trimmed
required: true
``` ```
Operational notes: Operational notes:
- `workspace.cleanup_after_archive` controls run-scoped workspace cleanup after successful archive commit. - archive promotion is explicit and path-based via `archive.promote_artifacts`.
- `spool.delete_audio_after_archive` controls run-scoped spool-audio cleanup after successful archive commit. - Narratio does not auto-promote all generated analyze artifacts.
- S3 archive/session-audio workflows require `storage.s3.bucket`.
## 7. Full pipeline reference ## 7. Full pipeline reference
Defaults listed here are effective runtime defaults after load.
| Path | Type | Required | Default | | Path | Type | Required | Default |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| `pipeline.workspace.root` | string | No | `/var/lib/narratio` | | `pipeline.workspace.root` | string | No | `/var/lib/narratio` |
@@ -150,7 +154,7 @@ Defaults listed here are effective runtime defaults after load.
| `pipeline.spool.delete_audio_after_archive` | bool | No | `false` | | `pipeline.spool.delete_audio_after_archive` | bool | No | `false` |
| `pipeline.archive.enabled` | bool | No | `true` | | `pipeline.archive.enabled` | bool | No | `true` |
| `pipeline.archive.upload_run` | bool | No | `true` | | `pipeline.archive.upload_run` | bool | No | `true` |
| `pipeline.archive.promote_artifacts[]` | list | No | two default rules | | `pipeline.archive.promote_artifacts[]` | list | No | trimmed + session_recap rules |
| `pipeline.archive.promote_artifacts[].from` | string | Yes (per rule) | none | | `pipeline.archive.promote_artifacts[].from` | string | Yes (per rule) | none |
| `pipeline.archive.promote_artifacts[].to` | string | Yes (per rule) | none | | `pipeline.archive.promote_artifacts[].to` | string | Yes (per rule) | none |
| `pipeline.archive.promote_artifacts[].required` | bool | No | `true` | | `pipeline.archive.promote_artifacts[].required` | bool | No | `true` |
@@ -203,6 +207,7 @@ Defaults listed here are effective runtime defaults after load.
| `pipeline.scriptorium.render_debug` | bool | No | `false` | | `pipeline.scriptorium.render_debug` | bool | No | `false` |
| `pipeline.scriptorium.artifacts` | map | No | empty | | `pipeline.scriptorium.artifacts` | map | No | empty |
| `pipeline.scriptorium.artifacts.<name>.enabled` | bool | No | `false` | | `pipeline.scriptorium.artifacts.<name>.enabled` | bool | No | `false` |
| `pipeline.scriptorium.artifacts.<name>.depends_on[]` | list[string] | No | empty |
| `pipeline.scriptorium.artifacts.<name>.render_debug` | bool | No | unset | | `pipeline.scriptorium.artifacts.<name>.render_debug` | bool | No | unset |
| `pipeline.scriptorium.artifacts.<name>.prompt_id` | string | Conditional | none | | `pipeline.scriptorium.artifacts.<name>.prompt_id` | string | Conditional | none |
| `pipeline.scriptorium.artifacts.<name>.profile_id` | string | No | empty | | `pipeline.scriptorium.artifacts.<name>.profile_id` | string | No | empty |
@@ -221,6 +226,18 @@ Defaults listed here are effective runtime defaults after load.
| `pipeline.notification.recipient` | string | No | empty | | `pipeline.notification.recipient` | string | No | empty |
| `pipeline.notification.timeout` | duration string | No | empty | | `pipeline.notification.timeout` | duration string | No | empty |
Scriptorium artifact-key and dependency rules:
- artifact keys must match `^[a-z][a-z0-9_]*$`.
- enabled artifacts require `prompt_id` and `output_path`.
- `output_path` must be relative, traversal-safe, and under `artifacts/`.
- configured artifact input sources use `narratio.artifact.<name>`.
- if input source references `narratio.artifact.<name>`, artifact `<name>` must exist and must be listed in `depends_on`.
- every `depends_on` entry must be a configured artifact key.
- self-dependency is rejected.
- enabled dependency cycles are rejected.
- any artifact referenced by `depends_on` or `narratio.artifact.<name>` source must define `output_path` (even if not enabled).
Allowed `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source` values: Allowed `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source` values:
- `previous_session_artifact` - `previous_session_artifact`
@@ -229,7 +246,7 @@ Allowed `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source` values:
- `narratio.transcript.full` - `narratio.transcript.full`
- `narratio.transcript.trimmed` - `narratio.transcript.trimmed`
- `narratio.bounds.session` - `narratio.bounds.session`
- `narratio.artifact.session_recap` - `narratio.artifact.<configured_artifact_key>`
## 8. Full session reference ## 8. Full session reference
@@ -248,11 +265,11 @@ Allowed `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source` values:
Audio-source rule: Audio-source rule:
- You must configure exactly one audio source mode: - configure exactly one mode:
- `audio_dir`, or - `audio_dir`, or
- `audio_files` (at least one), or - `audio_files` (at least one), or
- `audio_s3.prefix` - `audio_s3.prefix`
- `audio_s3` cannot be combined with `audio_dir` or `audio_files`. - `audio_s3` cannot be combined with local audio fields.
## 9. Secrets ## 9. Secrets
@@ -261,22 +278,21 @@ Narratio supports filesystem-based secret injection via `pipeline.secrets.env_di
Behavior: Behavior:
- `env_dir` may be absolute or relative. - `env_dir` may be absolute or relative.
- Relative `env_dir` is resolved from Narratio’s current working directory. - relative `env_dir` resolves from current working directory.
- Each top-level file with a valid env-var filename (`[A-Za-z_][A-Za-z0-9_]*`) is loaded. - files with valid env-var names (`[A-Za-z_][A-Za-z0-9_]*`) are loaded.
- File contents become env-var values, with trailing `\n` / `\r\n` trimmed. - values are loaded from file contents with trailing newline trimming.
- Existing process environment variables are preserved and not overwritten. - existing process env vars are preserved.
- Invalid names and directories inside `env_dir` are skipped. - invalid names and subdirectories are skipped.
- Missing/unreadable `env_dir` fails command execution. - missing/unreadable `env_dir` fails command execution.
Guidance: Guidance:
- Store secret values in secret files or pre-set environment variables. - do not put secret values directly in YAML.
- Do not put secret values directly in `pipeline.yml` or `session.yml`. - configure env var names in config and provide values via env/secrets files.
- Use config fields like `llm_api_key_env` and S3 credential env names to reference secret variable names, not secret data.
## 10. Examples ## 10. Examples
Maintained config examples: Maintained examples:
- `docs/examples/pipeline.minimal.yml` - `docs/examples/pipeline.minimal.yml`
- `docs/examples/pipeline.production.yml` - `docs/examples/pipeline.production.yml`
@@ -285,4 +301,4 @@ Maintained config examples:
- `docs/examples/session.local-audio.yml` - `docs/examples/session.local-audio.yml`
- `docs/examples/session.s3-audio.yml` - `docs/examples/session.s3-audio.yml`
These examples are covered by configuration load/validate tests in `internal/config`. These examples are validated by `internal/config` tests.

35
docs/examples/pipeline.full.annotated.yml
View File

@@ -2,8 +2,8 @@
# Values are safe placeholders and must be adapted per environment. # Values are safe placeholders and must be adapted per environment.
workspace: workspace:
# Required: local workspace root. # Optional: defaults to /var/lib/narratio.
root: ./tmp/narratio-workspace root: /var/lib/narratio/workspace
# Optional: remove run-scoped workdir after successful archive commit. # Optional: remove run-scoped workdir after successful archive commit.
cleanup_after_archive: false cleanup_after_archive: false
@@ -14,7 +14,7 @@ workspace:
storage: storage:
# Optional storage backend selector; use "s3" for archive + S3 audio workflows. # Optional storage backend selector; use "s3" for archive + S3 audio workflows.
backend: s3 backend: s3
# Legacy fields retained in schema for compatibility. # Compatibility fields retained in schema.
bucket: "" bucket: ""
prefix: "" prefix: ""
s3: s3:
@@ -40,7 +40,7 @@ archive:
# Optional booleans; defaults are true. # Optional booleans; defaults are true.
enabled: true enabled: true
upload_run: true upload_run: true
# Optional promotions; defaults shown explicitly. # Optional promotion rules; required files fail archive if missing.
promote_artifacts: promote_artifacts:
- from: transcripts/trimmed.json - from: transcripts/trimmed.json
to: transcripts/trimmed.json to: transcripts/trimmed.json
@@ -48,6 +48,9 @@ archive:
- from: artifacts/session_recap.md - from: artifacts/session_recap.md
to: artifacts/session_recap.md to: artifacts/session_recap.md
required: true required: true
- from: artifacts/player_handout.md
to: artifacts/player_handout.md
required: false
whisperx: whisperx:
# Required. # Required.
@@ -118,6 +121,7 @@ scriptorium:
timeout: 10m timeout: 10m
render_debug: false render_debug: false
artifacts: artifacts:
# Configured artifact keys map to source IDs narratio.artifact.<key>.
session_recap: session_recap:
enabled: true enabled: true
prompt_id: dnd.session_recap prompt_id: dnd.session_recap
@@ -140,6 +144,29 @@ scriptorium:
previous_session_id: true previous_session_id: true
output_kind: session_recap output_kind: session_recap
# Example dependent artifact:
# - depends_on entries use artifact keys.
# - narratio.artifact.<key> sources require matching depends_on membership.
player_handout:
enabled: true
depends_on:
- session_recap
prompt_id: dnd.player_handout
profile_id: local-fast
output_path: artifacts/player_handout.md
timeout: 10m
inputs:
recap:
source: narratio.artifact.session_recap
required: true
transcript:
source: narratio.transcript.trimmed
required: true
vars:
session_id: true
campaign_name: true
output_kind: player_handout
analyzer: analyzer:
# Optional adapter settings. # Optional adapter settings.
binary_path: "" binary_path: ""

21
docs/examples/pipeline.production.yml
View File

@@ -25,6 +25,9 @@ archive:
- from: artifacts/session_recap.md - from: artifacts/session_recap.md
to: artifacts/session_recap.md to: artifacts/session_recap.md
required: true required: true
- from: artifacts/player_handout.md
to: artifacts/player_handout.md
required: false
whisperx: whisperx:
transcribe_url: "https://transcription.example.com/transcribe" transcribe_url: "https://transcription.example.com/transcribe"
@@ -87,6 +90,24 @@ scriptorium:
campaign_name: true campaign_name: true
previous_session_id: true previous_session_id: true
output_kind: session_recap output_kind: session_recap
player_handout:
enabled: true
depends_on:
- session_recap
prompt_id: dnd.player_handout
profile_id: local-fast
output_path: artifacts/player_handout.md
timeout: 10m
inputs:
recap:
source: narratio.artifact.session_recap
required: true
transcript:
source: narratio.transcript.trimmed
required: true
vars:
session_id: true
output_kind: player_handout
analyzer: analyzer:
timeout: 2m timeout: 2m

6
docs/internal/README.md
View File

@@ -4,13 +4,13 @@
Developers and LLM coding agents changing Narratio internals. Developers and LLM coding agents changing Narratio internals.
## Scope ## Scope
Implementation-accurate contracts for workspace/state, stages, and external adapter boundaries. Implementation-accurate contracts for workspace/state, manifests, stages, artifact resolution, and adapter boundaries.
## Component Docs ## Component Docs
- `adapters.md`: external adapter map, runtime wiring, and boundary ownership. - `adapters.md`: external adapter map, runtime wiring, and boundary ownership.
- `storage.md`: remote storage backend contracts and object-store invariants. - `storage.md`: remote storage backend contracts and object-store invariants.
- `manifest.md`: session/run manifest schemas, lifecycle transitions, and persistence semantics. - `manifest.md`: session/run manifest schemas, lifecycle transitions, and persistence semantics.
- `artifacts.md`: supported artifact IDs, transcript tiers, and source-resolution behavior. - `artifacts.md`: built-in artifact registry, runtime artifact catalog, and source-resolution behavior.
- `workspace.md`: local state model, manifests, run-local layout, promotion, and cleanup invariants. - `workspace.md`: local state model, manifests, run-local layout, promotion, and cleanup invariants.
- `stage-prepare.md`: input materialization and provenance capture. - `stage-prepare.md`: input materialization and provenance capture.
- `stage-transcribe.md`: WhisperX transcript generation. - `stage-transcribe.md`: WhisperX transcript generation.
@@ -18,7 +18,7 @@ Implementation-accurate contracts for workspace/state, stages, and external adap
- `stage-polish.md`: Audita transcript polishing. - `stage-polish.md`: Audita transcript polishing.
- `stage-normalize.md`: post-polish normalization. - `stage-normalize.md`: post-polish normalization.
- `stage-trim.md`: bounds-driven transcript trimming. - `stage-trim.md`: bounds-driven transcript trimming.
- `stage-analyze.md`: Scriptorium session recap generation. - `stage-analyze.md`: dependency-ordered Scriptorium artifact generation for selected configured artifacts.
- `stage-archive.md`: archive upload and current-pointer publish contract. - `stage-archive.md`: archive upload and current-pointer publish contract.
## External Integration Notes ## External Integration Notes

83
docs/internal/artifacts.md
View File

@@ -1,39 +1,40 @@
# Internal: Artifacts # Internal: Artifacts
## Purpose ## Purpose
Describe supported session artifact IDs, transcript tiers, and artifact resolution/provenance behavior used by stage logic and Scriptorium input configuration. Define Narratio's artifact identity and resolution model for built-in transcript/bounds artifacts and runtime-configured analyze artifacts.
## Inputs and outputs ## Inputs and outputs
Inputs: Inputs:
- Artifact source identifiers from stage config/runtime (for example `pipeline.scriptorium.artifacts.*.inputs.*.source`). - artifact sources from config/runtime (`pipeline.scriptorium.artifacts.*.inputs.*.source`)
- Session paths and optional session manifest stage outputs. - session paths and optional session manifest stage outputs
- runtime artifact catalog state for configured artifact sources
Outputs: Outputs:
- Resolved local artifact path + provenance (`ResolvedSessionArtifact`). - resolved local artifact path and provenance (`ResolvedSessionArtifact`)
- Validation errors for unsupported or unreadable artifact sources. - runtime catalog entries for planned/executable/available artifacts
- validation errors for unsupported, missing, or invalid artifact sources
## Boundaries ## Boundaries
Owns: Owns:
- Canonical artifact ID registry and metadata (`internal/artifacts/artifact_resolver.go`). - built-in artifact registry and content validation rules
- Alias normalization for legacy source names. - runtime artifact catalog for configured artifact source IDs
- Resolution order and artifact content validation. - source resolution behavior for built-in and configured artifact sources
Does not own: Does not own:
- Artifact generation (stages produce files). - artifact generation (stages produce files)
- Manifest transition policy. - manifest transition policy
- Remote archive publishing behavior. - archive promotion behavior
## Config fields used ## Config fields used
Artifact source usage is driven by: - `pipeline.scriptorium.artifacts.<name>.enabled`
- `pipeline.scriptorium.artifacts.<name>.output_path`
- `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source` - `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source`
- Optional source-specific fields for previous artifact input (`artifact`, `path`, `required`).
## External adapters used ## External adapters used
- No external service adapters. - none
- Resolver relies on local filesystem checks + session manifest state.
## State and manifest behavior ## State and manifest behavior
Supported canonical IDs and current mappings: Built-in registry entries:
| Artifact ID | Canonical file | Producer stage | Output kind | | Artifact ID | Canonical file | Producer stage | Output kind |
| --- | --- | --- | --- | | --- | --- | --- | --- |
@@ -42,39 +43,45 @@ Supported canonical IDs and current mappings:
| `narratio.transcript.full` | `transcripts/normalized.json` | `normalize` | `transcript_normalized` | | `narratio.transcript.full` | `transcripts/normalized.json` | `normalize` | `transcript_normalized` |
| `narratio.transcript.trimmed` | `transcripts/trimmed.json` | `trim` | `transcript_trimmed` | | `narratio.transcript.trimmed` | `transcripts/trimmed.json` | `trim` | `transcript_trimmed` |
| `narratio.bounds.session` | `artifacts/session_bounds.json` | `trim` | `session_bounds` | | `narratio.bounds.session` | `artifacts/session_bounds.json` | `trim` | `session_bounds` |
| `narratio.artifact.session_recap` | `artifacts/session_recap.md` | `analyze` | `session_recap` |
Resolution order: Runtime catalog entries include built-ins and configured `narratio.artifact.<name>` sources.
1. Session manifest producer-stage outputs (if readable/valid).
2. Canonical session path fallback.
Provenance fields: Catalog states:
- `ProducerStage` - `planned`: source is registered and known for this run
- `OutputKind` - `executable`: configured artifact is selected for analyze execution
- `ProducerRunID` (when resolved from manifest output) - `available`: artifact has a usable file path (generated this run or reused from disk)
- `Provenance` (`manifest.<stage>.outputs` or `fallback.canonical_path`)
Content validation by artifact type: Resolution behavior:
- Transcript artifacts: JSON with top-level `segments` array. - built-in sources resolve via manifest producer outputs first, then canonical fallback path
- `narratio.bounds.session`: valid JSON. - configured `narratio.artifact.<name>` sources resolve through runtime catalog availability
- `narratio.artifact.session_recap`: non-empty text. - configured source lookup requires catalog context
Configured artifact provenance values:
- `generated.current_analyze_run`
- `filesystem.disabled_artifact_output`
Content validation:
- transcript built-ins: JSON with top-level `segments` array
- bounds built-in: valid JSON
- configured artifacts: non-empty text file
## Skip and resume behavior ## Skip and resume behavior
- Resolver has no direct skip/resume logic. - resolver and catalog have no direct skip/resume decisions
- Resolver output influences stage behavior (for example analyze input resolution and required-input failures). - stage/runner skip-resume behavior consumes catalog/resolver results
## Failure behavior ## Failure behavior
- Unsupported or empty artifact source -> normalization error. - unsupported source -> source validation error
- Known source not found/readable -> `ErrSessionArtifactNotFound` wrapped error. - known source unavailable -> `ErrSessionArtifactNotFound`
- Found but invalid content -> validation error. - configured source without catalog -> resolution error
- resolved file with invalid content -> validation error
## Tests to inspect before changing ## Tests to inspect before changing
- `internal/artifacts/artifact_resolver_test.go` - `internal/artifacts/artifact_resolver_test.go`
- `internal/artifacts/resolve_test.go` - `internal/artifacts/catalog_test.go`
- `internal/stage/analyze_test.go` - `internal/stage/analyze_test.go`
- `internal/config/scriptorium_test.go` - `internal/config/scriptorium_test.go`
## Architectural invariants ## Architectural invariants
- Artifact IDs are canonical interface values for stage/config integration. - built-in IDs are static and registry-backed
- Alias support is compatibility behavior layered on top of canonical IDs. - configured artifact IDs are runtime-derived (`narratio.artifact.<name>`) and catalog-backed
- Manifest producer outputs are preferred over canonical fallback when both exist. - built-in/source resolution remains deterministic and validation-gated

1
docs/internal/manifest.md
View File

@@ -55,6 +55,7 @@ Relationship during execution:
- Runner updates both manifests for every stage transition. - Runner updates both manifests for every stage transition.
- Session manifest is the durable pipeline-progress ledger. - Session manifest is the durable pipeline-progress ledger.
- Run manifest is invocation history and audit record. - Run manifest is invocation history and audit record.
- Analyze stage outputs are persisted as `kind=scriptorium_artifact` with `source_id=narratio.artifact.<name>` for configured artifact identity.
## Skip and resume behavior ## Skip and resume behavior
- Resume and skip decisions are based on session-manifest stage statuses. - Resume and skip decisions are based on session-manifest stage statuses.

63
docs/internal/stage-analyze.md
View File

@@ -1,25 +1,31 @@
# Stage: analyze # Stage: analyze
## Purpose ## Purpose
Generate the session recap artifact using configured Scriptorium artifact settings. Execute selected configured Scriptorium artifacts in deterministic dependency order and promote successful outputs to canonical session artifact paths.
## Inputs and Outputs ## Inputs and Outputs
Inputs: Inputs:
- transcript inputs as requested by selected artifact config (processed/normalized/trimmed/current recap, depending on `pipeline.scriptorium.artifacts.session_recap.inputs`) - configured artifact definitions from `pipeline.scriptorium.artifacts`
- selected artifact filter from runtime (`--artifacts`) when provided
- resolved artifact input sources declared per artifact (`inputs.*.source`)
- optional previous-session file inputs (`previous_session_artifact`)
Outputs: Outputs:
- `artifacts/session_recap.md` - one promoted output file per executed configured artifact at that artifact's configured `output_path`
- stage metadata containing generated artifact entries and reused disabled-artifact entries
## Boundaries ## Boundaries
Owns: Owns:
- Selecting supported analyze artifact (`session_recap` only) - runtime artifact catalog construction for analyze execution
- Resolving transcript/reference inputs and vars - selected-artifact planning and dependency ordering
- Optional render-debug execution before run - per-artifact input resolution, var resolution, timeout/render-debug resolution
- Main Scriptorium run and output promotion - Scriptorium run/render invocation for each selected artifact
- run-local output generation and canonical promotion
Does not own: Does not own:
- Transcript processing pipeline stages - transcript generation/processing stages
- Archive publish/pointer behavior - archive promotion policy
- per-artifact resume semantics
## Config Fields Used ## Config Fields Used
- `session.session_id` - `session.session_id`
@@ -29,8 +35,9 @@ Does not own:
- `pipeline.scriptorium.config_path` - `pipeline.scriptorium.config_path`
- `pipeline.scriptorium.timeout` - `pipeline.scriptorium.timeout`
- `pipeline.scriptorium.render_debug` - `pipeline.scriptorium.render_debug`
- `pipeline.scriptorium.artifacts.session_recap.*` - `pipeline.scriptorium.artifacts.<name>.*`
- `enabled` - `enabled`
- `depends_on`
- `prompt_id` - `prompt_id`
- `profile_id` - `profile_id`
- `timeout` - `timeout`
@@ -41,29 +48,37 @@ Does not own:
## External Adapters Used ## External Adapters Used
- Scriptorium adapter: - Scriptorium adapter:
- optional `RenderArtifact` (debug diagnostics) - optional `RenderArtifact` (render debug)
- `RunArtifact` (actual recap generation) - `RunArtifact` (artifact generation)
## State and Manifest Behavior ## State and Manifest Behavior
- If `pipeline.scriptorium` is nil, stage returns success metadata with `skipped=true`. - If `pipeline.scriptorium` is absent, stage returns success metadata with `skipped=true`.
- If no enabled artifacts exist, stage returns success metadata with `skipped=true`. - If no artifacts are configured, stage returns success metadata with `skipped=true`.
- If enabled artifacts exist but any artifact other than `session_recap` is enabled, stage fails. - If zero artifacts are executable after `enabled` + `--artifacts` filtering, stage returns success metadata with `skipped=true`.
- Uses run-local output/log/config/reports paths when run layout is enabled. - Builds runtime catalog with built-ins and configured artifacts.
- Promotes canonical recap output and records adapter metadata. - Non-executable configured artifacts are marked available only when their configured output file exists and is valid on disk.
- Executes selected configured artifacts in topological order with deterministic tie-breaking.
- For each generated artifact, records metadata fields including `name`, `source_id`, `output_kind`, `path`, `prompt_id`, `profile_id`, and `provenance`.
- Reused disabled artifacts are recorded separately in `reused_artifacts` with provenance `filesystem.disabled_artifact_output`.
## Skip and Resume Behavior ## Skip and Resume Behavior
- Runner-level skip applies when already succeeded and not forced. - Runner-level skip applies when analyze is already `succeeded` and `--force` is not set.
- Forced reruns can stale downstream succeeded stages. - Analyze remains stage-scoped for resume/skip; there is no per-artifact resume state.
- Stage-local "skipped" metadata is distinct from runner-level stage status skip. - `--artifacts` filters which configured artifacts are executable when analyze runs; it does not imply `--force`.
## Failure Behavior ## Failure Behavior
- Fails on missing required resolved inputs, invalid transcript inputs, render/run adapter failures, or validation-failed run results. - Fails on invalid dependency ordering, unavailable required configured inputs, invalid built-in input prerequisites, render/run adapter failures, validation-failed adapter results, or missing/empty outputs.
- Required configured dependency missing from catalog availability fails clearly before invocation.
- Optional missing inputs are omitted.
## Tests to Inspect Before Changing ## Tests to Inspect Before Changing
- `internal/stage/analyze_test.go` - `internal/stage/analyze_test.go`
- `internal/artifacts/catalog_test.go`
- `internal/artifacts/artifact_resolver_test.go`
- `internal/adapters/scriptorium/subprocess_test.go` - `internal/adapters/scriptorium/subprocess_test.go`
## Architectural Invariants ## Architectural Invariants
- Analyze implementation supports only `artifacts.session_recap` as executable artifact. - Configured artifacts are identified by `narratio.artifact.<name>` source IDs.
- Optional inputs may be omitted; required inputs must resolve. - Artifact-to-artifact references rely on explicit `depends_on` declarations validated in config.
- Successful output must exist and be non-empty before promotion. - Generated analyze outputs are treated uniformly as Scriptorium artifacts.
- Successful outputs must exist and be non-empty before promotion.

171
docs/operations.md
View File

@@ -6,8 +6,7 @@ For field-level configuration, see [docs/config.md](./config.md). For full comma
## Normal workflow (S3-first path) ## Normal workflow (S3-first path)
1. Upload session `.flac` files to the session audio prefix in object storage: 1. Upload session `.flac` files to object storage under the session audio prefix.
- `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/{audio_s3.prefix}`
2. Run Narratio: 2. Run Narratio:
```bash ```bash
@@ -15,28 +14,24 @@ narratio run --session-id 2026-04-04
``` ```
3. Read success output: 3. Read success output:
- `narratio run: session <session_id>; executed=<n> skipped=<n>; manifest=<path>` - `narratio run: session <session_id>; executed=<n> skipped=<n>; manifest=<path>`
- `manifest=<path>` is the local session manifest path to use with `status`. - use `manifest=<path>` with `status` for inspection.
Notes: Notes:
- default config/session discovery applies unless `--config` and `--session` are passed.
- This command relies on discoverable `pipeline.yml` and `session.yml` unless `--config` and `--session` are passed explicitly. - S3 audio mode requires `session.inputs.audio_s3.prefix` and valid object-store access.
- For S3 audio input, `session.inputs.audio_s3.prefix` must be configured and audio files must already exist remotely.
## Local filesystem layout and state artifacts ## Local filesystem layout and state artifacts
Session root: Session root:
- `{workspace.root}/work/{campaign}/{session_id}/` - `{workspace.root}/work/{campaign}/{session_id}/`
Primary state: Primary state:
- `manifest.json`: session-level stage state.
- `manifest.json`: session-level manifest (authoritative local stage state). - `runs/{run_id}/manifest.json`: invocation-level state.
- `runs/{run_id}/manifest.json`: run-level manifest for one invocation. - `.lock`: session lock while a run is active.
- `.lock`: session lock file while a run is active.
Canonical session directories: Canonical session directories:
- `inputs/` - `inputs/`
- `audio/` - `audio/`
- `transcripts/` - `transcripts/`
@@ -48,123 +43,107 @@ Canonical session directories:
- `runs/` - `runs/`
Run-local stage directories: Run-local stage directories:
- `runs/{run_id}/{stage}/` with stage-local `outputs/`, `logs/`, `reports/`, `config/`, `scratch/`.
- `runs/{run_id}/{stage}/` Behavior:
- Stage runtime files are written under deterministic run-local subdirectories such as: - directory creation is idempotent.
- `outputs/`, `logs/`, `reports/`, `config/`, `scratch/` - stage outputs are generally generated run-local first, then promoted to canonical paths on success.
Behavior notes: ## Analyze artifact execution lifecycle
- Layout creation is idempotent. Analyze executes configured artifacts from `pipeline.scriptorium.artifacts`.
- Durable outputs are promoted to canonical session paths after stage success.
- Run-local artifacts remain in `runs/{run_id}/...` unless configured post-archive cleanup removes that run scope. Execution model:
- executable set = enabled artifacts, filtered by `--artifacts` when provided.
- artifact-to-artifact dependencies are declared via `depends_on`.
- selected artifacts run in deterministic dependency order.
- after each successful artifact run, output is promoted to configured canonical `output_path`.
Configured artifact source reuse:
- a non-executable configured artifact can satisfy inputs if its configured output file already exists and is valid.
- reused configured artifact provenance is `filesystem.disabled_artifact_output`.
`--artifacts` behavior:
- accepted on `run`, `resume`, and `run-stage analyze`.
- filters analyze execution only; does not force stage rerun.
## Remote archive layout and publish contract ## Remote archive layout and publish contract
When archive is enabled and run upload is enabled, archive publishes to object storage under: When archive is enabled and run upload is enabled, archive publishes under:
- Session prefix: `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/` - session prefix: `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/`
- Run prefix: `{session_prefix}/runs/{run_id}/` - run prefix: `{session_prefix}/runs/{run_id}/`
Archive uploads: Archive uploads:
- run record files from run root (excluding `audio/`).
- promoted files from explicit `archive.promote_artifacts` rules.
- Run record files from run root (including stage subtrees and run manifest), excluding local `audio/`. Publish order:
- Promoted artifacts from `archive.promote_artifacts` to session-level keys. 1. upload `current/manifest.json`
2. upload `current/run_id.txt` last
Publish order (commit contract): `current/run_id.txt` is the remote commit marker.
1. Upload `current/manifest.json` Archive promotion is explicit and path-based:
2. Upload `current/run_id.txt` last - Narratio does not auto-promote all generated analyze artifacts.
- missing required promotion sources fail archive stage.
Meaning of `current/run_id.txt`: - missing optional promotion sources are skipped.
- It is the effective remote commit marker for published session state.
- It is written only after required run uploads and required promotions succeed.
## Resume, retry, and safe rerun behavior ## Resume, retry, and safe rerun behavior
Default skip behavior: Default skip:
- `run` and `run-stage` skip already-succeeded stages unless `--force` is set.
- `run` and `run-stage` skip stages already marked `succeeded` unless `--force` is set. Resume:
- `resume` starts at first non-succeeded stage.
- `resume --force` runs full stage order.
Resume behavior: Forced reruns:
- force-rerunning an upstream succeeded stage marks downstream succeeded stages as `stale`.
- `resume` starts at the first non-`succeeded` stage in canonical stage order. Safe rerun pattern:
- If all stages are `succeeded`, `resume` prints that no stages remain. 1. rerun the changed stage with `--force`.
- `resume --force` runs full stage order rather than starting at first non-succeeded. 2. run `resume` to rebuild downstream stages.
Forced rerun behavior:
- Successful forced rerun of an upstream stage marks downstream previously `succeeded` stages as `stale`.
- `stale` stages are not treated as complete and are eligible to run in subsequent commands.
Targeted rerun with one stage:
```bash
narratio run-stage --force <stage>
```
Valid stage names:
- `prepare`, `transcribe`, `merge`, `polish`, `normalize`, `trim`, `analyze`, `archive`, `notify`
Safe operator pattern:
1. Force-rerun the stage that changed.
2. Run `resume` to rebuild downstream stages in order.
## Cleanup behavior ## Cleanup behavior
Cleanup is considered only after run execution completes and only when archive stage both executed and succeeded. Cleanup is considered only when archive stage executed and succeeded.
Configured cleanup toggles: Cleanup toggles:
- `pipeline.spool.delete_audio_after_archive=true` deletes run-scoped spool audio.
- `pipeline.workspace.cleanup_after_archive=true` deletes run-scoped local run directory.
- `pipeline.spool.delete_audio_after_archive=true` Cleanup eligibility gates:
- deletes only run-scoped spool audio directory: `{spool.root}/{campaign}/{session_id}/{run_id}/audio/` - archive enabled
- `pipeline.workspace.cleanup_after_archive=true` - archive run upload enabled
- deletes only run-scoped local run directory: `{workspace.root}/work/{campaign}/{session_id}/runs/{run_id}/` - run record upload completed
- current pointer write completed (`current/run_id.txt` written)
Eligibility gates for cleanup: No cleanup for failed/incomplete/unarchived/archive-skipped runs.
- archive is enabled
- archive run upload is enabled
- archive metadata indicates run record upload happened
- archive metadata indicates `current` pointer write completed (`current/run_id.txt` written)
Cleanup does not run for:
- failed runs
- incomplete runs
- unarchived runs
- archive-skipped runs (`archive.enabled=false` or `archive.upload_run=false`)
## Failure and recovery playbooks ## Failure and recovery playbooks
What remains after failure: After failure, Narratio keeps:
- session manifest
- run manifest
- run-local artifacts/logs/config/reports
- Session manifest remains on disk. Failed or incomplete runs remain local-only.
- Run manifest remains under `runs/{run_id}/manifest.json`.
- Run-local stage artifacts/logs/config/reports remain under `runs/{run_id}/...`.
- Failed/incomplete runs remain local-only.
- Remote current pointer is not committed if archive prerequisite or pointer-write steps fail.
Recommended recovery flow: Recommended recovery:
1. Inspect current state: 1. inspect state:
```bash ```bash
narratio status --manifest <manifest-path-from-run-output> narratio status --manifest <manifest-path>
``` ```
2. Fix the root cause (config, input, credentials, adapter availability, etc.). 2. fix root cause (config/input/credentials/service availability).
3. Continue with: 3. continue with `resume`, or targeted `run-stage --force` followed by `resume`.
- `narratio resume --session-id <id>` for ordered continuation, or
- `narratio run-stage --force <stage>` for targeted correction, then `resume`.
## Operational caveats ## Operational caveats
- `status` requires an explicit manifest path; there is no direct session-id lookup command. - `status` requires explicit `--manifest`; there is no session-id lookup command.
- S3 audio mode and local audio mode are mutually exclusive in session config. - local and S3 audio input modes are mutually exclusive.
- Archive verifies stage prerequisites (`prepare` through `analyze`) before publishing. - archive publish requires upstream stages through `analyze` to be `succeeded`.
- By default, archive does not upload local `audio/` into run history. - required promotion rules can fail when selected analyze artifacts did not generate a required file path.
- Unknown CLI commands fail and print usage.

2
docs/roadmap/runtime-artifacts.md
View File

@@ -663,6 +663,8 @@ Tests:
### Phase 8: Documentation and Examples ### Phase 8: Documentation and Examples
Status: complete.
Update documentation after the implementation is complete. Update documentation after the implementation is complete.
Recommended documentation changes: Recommended documentation changes:

212
docs/troubleshooting.md
View File

@@ -6,11 +6,11 @@ Canonical operator troubleshooting guide for recurring implemented Narratio fail
## Config file discovery failure ## Config file discovery failure
Symptom: Symptom:
- `run`, `plan`, `resume`, or `run-stage` fails saying config/session file was not found. - `run`, `plan`, `resume`, or `run-stage` fails with config/session not found.
Likely Cause: Likely Cause:
- `pipeline.yml` or `session.yml` is missing from default search paths. - `pipeline.yml` or `session.yml` is missing from discovery paths.
- Wrong working directory when relying on `./session.yml`. - wrong working directory when relying on `./session.yml`.
Diagnostics: Diagnostics:
@@ -21,8 +21,8 @@ ls -l /usr/local/etc/narratio/pipeline.yml /etc/narratio/pipeline.yml
``` ```
Safe Fix: Safe Fix:
- Pass explicit paths with `--config` and `--session`. - pass explicit `--config` and `--session`.
- Or place files in documented discovery paths. - or place files in documented discovery paths.
Links: Links:
- [docs/config.md](./config.md) - [docs/config.md](./config.md)
@@ -31,11 +31,11 @@ Links:
## Session template rendering failure ## Session template rendering failure
Symptom: Symptom:
- Load fails with unresolved template placeholder or `session_id` mismatch. - load fails with unresolved placeholder or `session_id` mismatch.
Likely Cause: Likely Cause:
- `session.yml` contains `{{session_id}}`/`{{ session_id }}` but `--session-id` was omitted. - templated `session.yml` used without `--session-id`.
- Provided `--session-id` does not match rendered `session_id`. - rendered `session_id` differs from passed `--session-id`.
Diagnostics: Diagnostics:
@@ -44,8 +44,8 @@ narratio plan --session ./session.yml --session-id 2026-04-04
``` ```
Safe Fix: Safe Fix:
- Always pass `--session-id` when using template placeholders. - pass `--session-id` when template placeholders are present.
- Ensure rendered `session_id` equals intended run session id. - ensure rendered `session_id` matches intended run session id.
Links: Links:
- [docs/config.md](./config.md) - [docs/config.md](./config.md)
@@ -53,12 +53,11 @@ Links:
## Strict YAML decode or validation failure ## Strict YAML decode or validation failure
Symptom: Symptom:
- Config load fails with unknown field, missing required field, invalid duration, or invalid cross-field constraint. - config load fails with unknown field or validation error.
Likely Cause: Likely Cause:
- YAML key typo or stale field name. - typo/stale field name.
- Required fields missing. - missing required fields or invalid constraints.
- Invalid value format (for example duration/URL/env var name).
Diagnostics: Diagnostics:
@@ -67,22 +66,114 @@ narratio plan --config /path/to/pipeline.yml --session /path/to/session.yml --se
``` ```
Safe Fix: Safe Fix:
- Correct fields/values to match canonical reference and examples. - align fields/values to canonical config reference and examples.
- Validate against `docs/examples/` shapes.
Links: Links:
- [docs/config.md](./config.md) - [docs/config.md](./config.md)
- [docs/examples/](./examples/) - [docs/examples/](./examples/)
## `--artifacts` selection failure
Symptom:
- `run`/`resume`/`run-stage` fails with invalid or unknown artifact selection.
Likely Cause:
- `--artifacts` contains blank names or unknown artifact keys.
- `pipeline.scriptorium.artifacts` missing while using `--artifacts`.
Diagnostics:
```bash
narratio run --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --artifacts player_handout
```
Safe Fix:
- use configured artifact keys only.
- ensure `pipeline.scriptorium.artifacts` is defined.
Links:
- [docs/cli.md](./cli.md)
- [docs/config.md](./config.md)
## `run-stage --artifacts` on non-analyze stage
Symptom:
- `run-stage` fails with `--artifacts is only supported for stage "analyze"`.
Likely Cause:
- `--artifacts` was used with a non-`analyze` stage.
Diagnostics:
```bash
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --artifacts session_recap polish
```
Safe Fix:
- use `--artifacts` only with `run-stage ... analyze`.
Links:
- [docs/cli.md](./cli.md)
## Configured artifact dependency/input validation failure
Symptom:
- config validation fails for `depends_on`, `narratio.artifact.<name>` source, or artifact output path.
Likely Cause:
- `narratio.artifact.<name>` source missing matching `depends_on` key.
- dependency references unknown artifact key.
- dependency self-reference or enabled dependency cycle.
- artifact output path missing/invalid/outside `artifacts/` root.
Diagnostics:
```bash
narratio plan --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04
```
Safe Fix:
- ensure artifact-to-artifact inputs have explicit `depends_on` entries using artifact keys.
- ensure referenced artifacts exist and define valid `output_path` values.
- keep output paths relative and under `artifacts/`.
Links:
- [docs/config.md](./config.md)
- [docs/internal/stage-analyze.md](./internal/stage-analyze.md)
## Required configured artifact input unavailable at analyze time
Symptom:
- analyze fails because configured input source is unavailable.
Likely Cause:
- required upstream configured artifact was not selected/executed this run.
- non-executable dependency output file is missing or invalid on disk.
Diagnostics:
```bash
narratio status --manifest /path/to/manifest.json
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 --artifacts player_handout analyze
```
Safe Fix:
- run analyze with needed artifacts selected.
- or ensure dependency output file exists at configured path and is valid.
Links:
- [docs/operations.md](./operations.md)
- [docs/config.md](./config.md)
## Manifest/status path failure ## Manifest/status path failure
Symptom: Symptom:
- `status` fails because manifest path is missing, unreadable, or invalid. - `status` fails because manifest path is missing, unreadable, or invalid.
Likely Cause: Likely Cause:
- Wrong manifest path. - wrong manifest path.
- Manifest removed after cleanup. - manifest removed after cleanup.
- Trying to run `status` without `--manifest`. - `--manifest` omitted.
Diagnostics: Diagnostics:
@@ -92,8 +183,7 @@ ls -l /path/to/manifest.json
``` ```
Safe Fix: Safe Fix:
- Use manifest path printed by `run`, `resume`, or `run-stage` output. - use manifest path printed by `run`, `resume`, or `run-stage`.
- Re-run with correct session/config if inspecting a different session.
Links: Links:
- [docs/cli.md](./cli.md) - [docs/cli.md](./cli.md)
@@ -102,11 +192,11 @@ Links:
## Session lock conflict (`.lock`) ## Session lock conflict (`.lock`)
Symptom: Symptom:
- Run fails with lock conflict indicating session workdir is already locked. - run fails with lock conflict for session workdir.
Likely Cause: Likely Cause:
- Another Narratio process is actively running the same session. - another Narratio process is running same session.
- Prior run exited unexpectedly and left a stale lock file. - stale lock from interrupted prior run.
Diagnostics: Diagnostics:
@@ -117,21 +207,21 @@ ps aux | grep narratio
``` ```
Safe Fix: Safe Fix:
- If another run is active, wait for it to finish. - wait for active run to finish.
- If no process is active and lock is stale, remove only that session `.lock` file and retry. - if no process is active, remove only stale session `.lock` file.
Links: Links:
- [docs/operations.md](./operations.md) - [docs/operations.md](./operations.md)
- [docs/internal/workspace.md](./internal/workspace.md) - [docs/internal/workspace.md](./internal/workspace.md)
## Secrets env-dir or credential env failure ## Secrets env-dir or credential-env failure
Symptom: Symptom:
- Startup fails loading secrets directory, or a stage fails because required credential env var is missing. - startup fails loading secrets directory, or stage fails due to missing credential env vars.
Likely Cause: Likely Cause:
- `pipeline.secrets.env_dir` path is wrong/unreadable. - invalid `pipeline.secrets.env_dir` path/permissions.
- Credential env var referenced in config is unset or empty. - required credential env var unset/empty.
Diagnostics: Diagnostics:
@@ -141,24 +231,22 @@ env | grep -E 'AUDITA|OBJECT_STORAGE|AWS|SCRIPTORIUM'
``` ```
Safe Fix: Safe Fix:
- Fix `pipeline.secrets.env_dir` path/permissions. - fix secrets directory and credential env vars.
- Ensure required env vars are set to non-empty values. - keep secret values out of YAML.
- Keep secrets out of YAML; use env references only.
Links: Links:
- [docs/config.md](./config.md) - [docs/config.md](./config.md)
- [docs/operations.md](./operations.md)
## S3-audio prepare failure ## S3-audio prepare failure
Symptom: Symptom:
- `prepare` fails in S3 mode (no audio found, list/download failure, backend missing, path conflict). - `prepare` fails in S3 mode (listing/downloading/no audio/backend error).
Likely Cause: Likely Cause:
- Wrong `session.inputs.audio_s3.prefix`. - wrong `session.inputs.audio_s3.prefix`.
- No `.flac` files at expected prefix. - no `.flac` files at resolved prefix.
- Missing or invalid S3 backend credentials/config. - invalid/missing object-store credentials or backend config.
- Conflicting audio-source settings (`audio_s3` plus local audio fields). - mixed local+S3 audio input config.
Diagnostics: Diagnostics:
@@ -167,24 +255,21 @@ narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml
``` ```
Safe Fix: Safe Fix:
- Ensure `audio_s3` is the only audio source configured for that session. - configure exactly one audio source mode.
- Confirm `.flac` objects exist under the resolved session audio prefix. - verify `.flac` files and storage access.
- Fix S3 storage configuration and credentials.
Links: Links:
- [docs/config.md](./config.md) - [docs/config.md](./config.md)
- [docs/operations.md](./operations.md) - [docs/operations.md](./operations.md)
- [docs/internal/stage-prepare.md](./internal/stage-prepare.md)
## Archive prerequisite or promotion/current-pointer failure ## Archive promotion/current-pointer failure
Symptom: Symptom:
- `archive` fails due to prerequisite stage status, missing required promotion source, or pointer write failure. - archive fails on required promotion source missing or pointer write failure.
Likely Cause: Likely Cause:
- One or more prerequisite stages are not `succeeded`. - required promoted file absent (including analyze outputs not generated for this run).
- Required promoted artifact does not exist. - storage upload failed before `current/run_id.txt` commit marker write.
- Remote upload failure before `current/run_id.txt` write.
Diagnostics: Diagnostics:
@@ -194,36 +279,11 @@ narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml
``` ```
Safe Fix: Safe Fix:
- Resume or rerun failed upstream stage(s). - rerun or resume upstream stages to generate required files.
- Ensure required promoted artifact paths exist locally before archive. - adjust promotion rules to match files that must exist.
- Retry archive after storage/connectivity issue is resolved. - retry after storage issue is resolved.
Links: Links:
- [docs/operations.md](./operations.md) - [docs/operations.md](./operations.md)
- [docs/config.md](./config.md) - [docs/config.md](./config.md)
- [docs/internal/stage-archive.md](./internal/stage-archive.md) - [docs/internal/stage-archive.md](./internal/stage-archive.md)
## `run-stage` invalid stage name or invalid flags
Symptom:
- `run-stage` fails with unknown stage or invalid flag/argument usage.
Likely Cause:
- Stage name typo.
- Missing positional stage argument.
- Unsupported/incorrect flag syntax.
Diagnostics:
```bash
narratio run-stage --config /path/to/pipeline.yml --session /path/to/session.yml --session-id 2026-04-04 normalize
```
Safe Fix:
- Use only supported stage names.
- Provide exactly one positional stage argument.
- Align flags to documented command reference.
Links:
- [docs/cli.md](./cli.md)
- [docs/operations.md](./operations.md)