narratio/docs/config.md

# Configuration

## 1. Overview

Narratio loads two YAML files:

- `pipeline.yml`: pipeline-level runtime configuration.
- `session.yml`: per-session metadata and input selection.

These commands load and validate both files before running:

- `narratio run`
- `narratio plan`
- `narratio resume`
- `narratio run-stage`

Configuration behavior:

- strict YAML decode is enabled (`KnownFields(true)`): unknown fields fail.
- session templates are rendered before session YAML decode.
- defaults are applied for many optional pipeline fields.
- validation enforces required fields, value formats, and cross-field constraints.

## 2. Config file discovery

Pipeline config lookup for `run`, `plan`, `resume`, and `run-stage`:

- If `--config <path>` is provided, that explicit path is used.
- If `--config` is omitted, Narratio searches in order:
  1. `/usr/local/etc/narratio/pipeline.yml`
  2. `/etc/narratio/pipeline.yml`
- The first existing file wins.
- If none exist, the command fails with a searched-paths error.

## 3. Session file discovery and templating

Session config lookup for `run`, `plan`, `resume`, and `run-stage`:

- If `--session <path>` is provided, that explicit path is used.
- If `--session` is omitted, Narratio searches in order:
  1. `./session.yml`
  2. `/usr/local/etc/narratio/session.yml`
  3. `/etc/narratio/session.yml`
- The first existing file wins.
- If none exist, the command fails and asks you to pass `--session`.

Session templating:

- Supported placeholders:
  - `{{session_id}}`
  - `{{ session_id }}`
- `--session-id <value>` supplies the template value.
- Unresolved placeholders fail with a template-rendering error.
- If `--session-id` is provided and rendered `session_id` differs, load fails with a mismatch error.
- Strict YAML decode still applies after template rendering.

## 4. Minimal pipeline config

```yaml
workspace:
  root: ./tmp/narratio-workspace

whisperx:
  transcribe_url: "https://transcription.example.com/transcribe"
```

Why this is sufficient:

- `workspace.root` and `whisperx.transcribe_url` are the core required pipeline fields.
- Seriatim and Audita sections may be omitted; defaults are applied.
- Archive, storage, spool, normalize, and other optional sections get defaults when omitted.

## 5. Minimal session template

```yaml
session_id: "{{ session_id }}"
campaign: sample-campaign
inputs:
  audio_dir: ./audio
  speakers_file: ./examples/speakers.yml
  autocorrect_file: ./examples/autocorrect.yml
  glossary_file: ./examples/glossary.yml
```

Usage:

```bash
narratio run --config /path/to/pipeline.yml --session ./session.yml --session-id 2026-05-03
```

## 6. Production-oriented config

```yaml
workspace:
  root: /var/lib/narratio/workspace
  cleanup_after_archive: true

storage:
  backend: s3
  s3:
    bucket: my-dnd-archive
    root_prefix: dnd
    region: us-east-1
    access_key_id_env: OBJECT_STORAGE_KEY_ID
    secret_access_key_env: OBJECT_STORAGE_KEY

spool:
  root: /var/spool/narratio
  delete_audio_after_archive: true

archive:
  enabled: true
  upload_run: true
  promote_artifacts:
    - from: transcripts/trimmed.json
      to: transcripts/trimmed.json
      required: true
    - from: artifacts/session_recap.md
      to: artifacts/session_recap.md
      required: true

whisperx:
  transcribe_url: "https://transcription.example.com/transcribe"
```

Operational notes:

- `workspace.cleanup_after_archive` controls run-scoped workspace cleanup after successful archive commit.
- `spool.delete_audio_after_archive` controls run-scoped spool-audio cleanup after successful archive commit.
- S3 archive/session-audio workflows require `storage.s3.bucket`.

## 7. Full pipeline reference

Defaults listed here are effective runtime defaults after load.

| Path | Type | Required | Default | Constraints / Notes |
| --- | --- | --- | --- | --- |
| `pipeline.workspace.root` | string | Yes | none | Must be non-empty. |
| `pipeline.workspace.cleanup_after_archive` | bool | No | `false` | Run-scoped workdir cleanup after successful archive commit. |
| `pipeline.secrets.env_dir` | string | Conditional | none | If `pipeline.secrets` is set, `env_dir` must be non-empty. |
| `pipeline.storage.backend` | string | No | empty | `s3` enables S3 archive decision path checks. |
| `pipeline.storage.bucket` | string | No | empty | Accepted by schema; compatibility field. |
| `pipeline.storage.prefix` | string | No | empty | Accepted by schema; compatibility field. |
| `pipeline.storage.s3.bucket` | string | Conditional | empty | Required when S3 session audio is used or S3 archive upload is enabled. |
| `pipeline.storage.s3.root_prefix` | string | No | `dnd` | Must be relative, non-empty, no `..`. |
| `pipeline.storage.s3.region` | string | No | empty | Optional region hint for backend wiring. |
| `pipeline.storage.s3.endpoint` | string | No | empty | If provided, must not be all-whitespace. |
| `pipeline.storage.s3.force_path_style` | bool | No | `false` | S3-compatible endpoint toggle. |
| `pipeline.storage.s3.access_key_id_env` | string | No | `OBJECT_STORAGE_KEY_ID` | Must be a valid env var name. |
| `pipeline.storage.s3.secret_access_key_env` | string | No | `OBJECT_STORAGE_KEY` | Must be a valid env var name. |
| `pipeline.spool.root` | string | No | `/var/spool/narratio` | Local spool root. |
| `pipeline.spool.delete_audio_after_archive` | bool | No | `false` | Cleanup toggle for run-scoped spool audio. |
| `pipeline.archive.enabled` | bool | No | `true` | Archive stage enablement. |
| `pipeline.archive.upload_run` | bool | No | `true` | Run-record upload toggle when archive enabled. |
| `pipeline.archive.promote_artifacts[]` | list | No | two default rules | Defaults: `transcripts/trimmed.json` and `artifacts/session_recap.md`. |
| `pipeline.archive.promote_artifacts[].from` | string | Yes (per rule) | none | Must be relative, non-empty, no `..`. |
| `pipeline.archive.promote_artifacts[].to` | string | Yes (per rule) | none | Must be relative, non-empty, no `..`. |
| `pipeline.archive.promote_artifacts[].required` | bool | No | `true` | Defaults per rule if omitted. |
| `pipeline.whisperx.transcribe_url` | string | Yes | none | Must be a valid URL. |
| `pipeline.whisperx.language` | string | No | `en` | Passed to WhisperX adapter. |
| `pipeline.whisperx.timeout` | duration string | No | `30m` | Must parse as duration. |
| `pipeline.whisperx.retries` | int | No | `3` | Must be `>= 0`. |
| `pipeline.whisperx.retry_delay` | duration string | No | `2s` | Must parse as duration. |
| `pipeline.whisperx.concurrency` | int | No | `2` | Must be `> 0`. |
| `pipeline.seriatim.binary` | string | No | `seriatim` | Must be non-empty after defaults. |
| `pipeline.seriatim.timeout` | duration string | No | `10m` | Must parse as duration. |
| `pipeline.seriatim.output_schema` | string | No | `seriatim-intermediate` | Allowed: `seriatim-minimal`, `seriatim-intermediate`, `seriatim-full`. |
| `pipeline.seriatim.coalesce_gap` | float | No | `3.0` | Must be `>= 0`. |
| `pipeline.seriatim.report` | bool | No | `true` | Enables report output in Seriatim calls. |
| `pipeline.seriatim.env.overlap_word_run_gap` | float | No | unset | If set, must be `> 0`. |
| `pipeline.seriatim.env.overlap_word_run_reorder_window` | float | No | unset | If set, must be `> 0`. |
| `pipeline.seriatim.env.backchannel_max_duration` | float | No | unset | If set, must be `> 0`. |
| `pipeline.seriatim.env.filler_max_duration` | float | No | unset | If set, must be `> 0`. |
| `pipeline.audita.binary` | string | No | `audita` | Must be non-empty after defaults. |
| `pipeline.audita.timeout` | duration string | No | `3h` | Must parse as duration. |
| `pipeline.audita.llm_api_key_env` | string | No | empty | Name of env var to forward to Audita. |
| `pipeline.audita.modules[]` | list[string] | No | empty | If set, each must be one of `glossary`, `homophones`, `spoken_word`, `grammar`. |
| `pipeline.audita.base_url` | string | No | empty | If non-empty, must be a valid URL. |
| `pipeline.audita.model` | string | No | empty | Optional model override passed to Audita. |
| `pipeline.audita.total_llm_concurrency` | int | No | unset | If set, must be `> 0`. |
| `pipeline.audita.proposal_llm_concurrency` | int | No | unset | If set, must be `> 0`. |
| `pipeline.audita.validation_model` | string | No | empty | Optional validation model override. |
| `pipeline.audita.validation_llm_concurrency` | int | No | unset | If set, must be `> 0`. |
| `pipeline.audita.transcript_description` | string | No | empty | If provided, must not be all-whitespace. |
| `pipeline.audita.config_path` | string | No | empty | If provided, must not be all-whitespace. |
| `pipeline.audita.output_schema` | string | No | empty | Allowed: empty, `bare-segments`, `audita-v1`. |
| `pipeline.audita.work_dir_retention` | string | No | empty | Allowed: empty, `always`, `auto`, `never`. |
| `pipeline.audita.report` | bool | No | `true` | Enables Audita report output. |
| `pipeline.normalize.output_path` | string | No | `transcripts/normalized.json` | Must be non-empty after defaults. |
| `pipeline.normalize.output_schema` | string | No | `seriatim-intermediate` | Allowed: `seriatim-minimal`, `seriatim-intermediate`, `seriatim-full`. |
| `pipeline.normalize.report` | bool | No | `true` | Enables normalize report output. |
| `pipeline.trim.enabled` | bool | No | `false` | When `false`, trim bounds fields are not required. |
| `pipeline.trim.output_path` | string | Conditional | none | Required when `pipeline.trim.enabled=true`. |
| `pipeline.trim.bounds.prompt_id` | string | Conditional | none | Required when `pipeline.trim.enabled=true`. |
| `pipeline.trim.bounds.profile_id` | string | No | empty | Optional profile override. |
| `pipeline.trim.bounds.transcript_input_name` | string | Conditional | none | Required when `pipeline.trim.enabled=true`. |
| `pipeline.trim.bounds.output_path` | string | Conditional | none | Required when `pipeline.trim.enabled=true`. |
| `pipeline.trim.bounds.timeout` | duration string | No | `10m` | Must parse as duration when set. |
| `pipeline.trim.bounds.render_debug` | bool | No | `false` | Enables render-debug output for bounds prompt. |
| `pipeline.trim.bounds.render_output_path` | string | Conditional | none | Required when `render_debug=true`. |
| `pipeline.trim.seriatim.report` | bool | No | `false` | Trim-stage Seriatim report toggle. |
| `pipeline.scriptorium.binary` | string | No | `scriptorium` | Required only when `pipeline.scriptorium` is configured. |
| `pipeline.scriptorium.config_path` | string | No | empty | If provided, must not be all-whitespace. |
| `pipeline.scriptorium.timeout` | duration string | No | `10m` | Must parse as duration when set. |
| `pipeline.scriptorium.render_debug` | bool | No | `false` | Global render-debug toggle for Scriptorium adapter usage. |
| `pipeline.scriptorium.artifacts` | map | No | empty | Artifact-generation map keyed by artifact name. |
| `pipeline.scriptorium.artifacts.<name>.enabled` | bool | No | `false` | If `true`, `prompt_id` and `output_path` are required. |
| `pipeline.scriptorium.artifacts.<name>.render_debug` | bool | No | unset | Per-artifact render-debug override. |
| `pipeline.scriptorium.artifacts.<name>.prompt_id` | string | Conditional | none | Required when artifact is enabled. |
| `pipeline.scriptorium.artifacts.<name>.profile_id` | string | No | empty | Optional profile override. |
| `pipeline.scriptorium.artifacts.<name>.output_path` | string | Conditional | none | Required when artifact is enabled. |
| `pipeline.scriptorium.artifacts.<name>.timeout` | duration string | No | empty | If set, must parse as duration. |
| `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source` | string | Conditional | none | Required when input is present; allowed values listed below. |
| `pipeline.scriptorium.artifacts.<name>.inputs.<key>.artifact` | string | No | empty | Used by `previous_session_artifact` source. |
| `pipeline.scriptorium.artifacts.<name>.inputs.<key>.path` | string | No | empty | Optional explicit path metadata. |
| `pipeline.scriptorium.artifacts.<name>.inputs.<key>.required` | bool | No | `false` | Input requirement flag for artifact generation. |
| `pipeline.scriptorium.artifacts.<name>.vars.<key>` | map value | No | empty | Value must be string or boolean. |
| `pipeline.analyzer.binary_path` | string | No | empty | Optional analyzer binary override. |
| `pipeline.analyzer.timeout` | duration string | No | empty | If set, must parse as duration. |
| `pipeline.analyzer.artifacts.output_dir` | string | No | empty | Optional analyzer output directory hint. |
| `pipeline.analyzer.artifacts.types[]` | list[string] | No | empty | Optional analyzer artifact type list. |
| `pipeline.notification.backend` | string | No | empty | Optional notifier backend selector. |
| `pipeline.notification.recipient` | string | No | empty | Optional notification recipient target. |
| `pipeline.notification.timeout` | duration string | No | empty | If set, must parse as duration. |

Allowed `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source` values:

- `previous_session_artifact`
- `processed_transcript`
- `normalized_transcript`
- `trimmed_transcript`
- `narratio.transcript.merged`
- `narratio.transcript.polished`
- `narratio.transcript.full`
- `narratio.transcript.trimmed`
- `narratio.bounds.session`
- `narratio.artifact.session_recap`

## 8. Full session reference

| Path | Type | Required | Default | Constraints / Notes |
| --- | --- | --- | --- | --- |
| `session.session_id` | string | Yes | none | Must be non-empty after template rendering. |
| `session.campaign` | string | Yes | none | Must be non-empty. Used in local/remote path modeling. |
| `session.date` | string | No | empty | Optional session metadata for prompts/artifacts. |
| `session.title` | string | No | empty | Optional session metadata for prompts/artifacts. |
| `session.inputs.audio_dir` | string | Conditional | empty | One audio-source option. Mutually exclusive with `audio_s3`. |
| `session.inputs.audio_files[]` | list[string] | Conditional | empty | One audio-source option. At least one entry can satisfy audio-source requirement. Mutually exclusive with `audio_s3`. |
| `session.inputs.audio_s3.prefix` | string | Conditional | none | Required if `audio_s3` object is present; must be relative, non-empty, no `..`. Mutually exclusive with `audio_dir` and `audio_files`. |
| `session.inputs.speakers_file` | string | Yes | none | Must be non-empty. |
| `session.inputs.autocorrect_file` | string | Yes | none | Must be non-empty. |
| `session.inputs.glossary_file` | string | Yes | none | Must be non-empty. |

Audio-source rule:

- You must configure exactly one audio source mode:
  - `audio_dir`, or
  - `audio_files` (at least one), or
  - `audio_s3.prefix`
- `audio_s3` cannot be combined with `audio_dir` or `audio_files`.

## 9. Secrets

Narratio supports filesystem-based secret injection via `pipeline.secrets.env_dir`.

Behavior:

- `env_dir` may be absolute or relative.
- Relative `env_dir` is resolved from Narratio’s current working directory.
- Each top-level file with a valid env-var filename (`[A-Za-z_][A-Za-z0-9_]*`) is loaded.
- File contents become env-var values, with trailing `\n` / `\r\n` trimmed.
- Existing process environment variables are preserved and not overwritten.
- Invalid names and directories inside `env_dir` are skipped.
- Missing/unreadable `env_dir` fails command execution.

Guidance:

- Store secret values in secret files or pre-set environment variables.
- Do not put secret values directly in `pipeline.yml` or `session.yml`.
- Use config fields like `llm_api_key_env` and S3 credential env names to reference secret variable names, not secret data.

## 10. Examples

Maintained config examples:

- `docs/examples/pipeline.minimal.yml`
- `docs/examples/pipeline.production.yml`
- `docs/examples/pipeline.full.annotated.yml`
- `docs/examples/session.template.yml`
- `docs/examples/session.local-audio.yml`
- `docs/examples/session.s3-audio.yml`

These examples are covered by configuration load/validate tests in `internal/config`.