narratio/docs/config.md

# Configuration

## 1. Overview

Narratio loads two YAML files:

- `pipeline.yml`: pipeline-level runtime configuration.
- `session.yml`: per-session metadata and input selection.

These commands load and validate both files before running:

- `narratio run`
- `narratio plan`
- `narratio resume`
- `narratio run-stage`

Configuration behavior:

- strict YAML decode is enabled (`KnownFields(true)`): unknown fields fail.
- session templates are rendered before session YAML decode.
- defaults are applied for many optional pipeline fields.
- validation enforces required fields, value formats, and cross-field constraints.

## 2. Config file discovery

Pipeline config lookup for `run`, `plan`, `resume`, and `run-stage`:

- If `--config <path>` is provided, that explicit path is used.
- If `--config` is omitted, Narratio searches in order:
  1. `/usr/local/etc/narratio/pipeline.yml`
  2. `/etc/narratio/pipeline.yml`
- The first existing file wins.
- If none exist, the command fails with a searched-paths error.

## 3. Session file discovery and templating

Session config lookup for `run`, `plan`, `resume`, and `run-stage`:

- If `--session <path>` is provided, that explicit path is used.
- If `--session` is omitted, Narratio searches in order:
  1. `./session.yml`
  2. `/usr/local/etc/narratio/session.yml`
  3. `/etc/narratio/session.yml`
- The first existing file wins.
- If none exist, the command fails and asks you to pass `--session`.

Session templating:

- Supported placeholders:
  - `{{session_id}}`
  - `{{ session_id }}`
- `--session-id <value>` supplies the template value.
- Unresolved placeholders fail with a template-rendering error.
- If `--session-id` is provided and rendered `session_id` differs, load fails with a mismatch error.
- Strict YAML decode still applies after template rendering.

## 4. Minimal pipeline config

```yaml
whisperx:
  transcribe_url: "https://transcription.example.com/transcribe"
```

Why this is sufficient:

- `whisperx.transcribe_url` is required.
- `workspace.root` is optional and defaults to `/var/lib/narratio`.
- Seriatim and Audita sections may be omitted; defaults are applied.
- Archive, storage, spool, normalize, and other optional sections get defaults when omitted.

## 5. Minimal session template

```yaml
session_id: "{{ session_id }}"
campaign: sample-campaign
inputs:
  audio_dir: ./audio
  speakers_file: ./examples/speakers.yml
  autocorrect_file: ./examples/autocorrect.yml
  glossary_file: ./examples/glossary.yml
```

Usage:

```bash
narratio run --config /path/to/pipeline.yml --session ./session.yml --session-id 2026-05-03
```

## 6. Production-oriented config

```yaml
workspace:
  root: /var/lib/narratio/workspace
  cleanup_after_archive: true

storage:
  backend: s3
  s3:
    bucket: my-dnd-archive
    root_prefix: dnd
    region: us-east-1
    access_key_id_env: OBJECT_STORAGE_KEY_ID
    secret_access_key_env: OBJECT_STORAGE_KEY

spool:
  root: /var/spool/narratio
  delete_audio_after_archive: true

archive:
  enabled: true
  upload_run: true
  promote_artifacts:
    - from: transcripts/trimmed.json
      to: transcripts/trimmed.json
      required: true
    - from: artifacts/session_recap.md
      to: artifacts/session_recap.md
      required: true

whisperx:
  transcribe_url: "https://transcription.example.com/transcribe"
```

Operational notes:

- `workspace.cleanup_after_archive` controls run-scoped workspace cleanup after successful archive commit.
- `spool.delete_audio_after_archive` controls run-scoped spool-audio cleanup after successful archive commit.
- S3 archive/session-audio workflows require `storage.s3.bucket`.

## 7. Full pipeline reference

Defaults listed here are effective runtime defaults after load.

| Path | Type | Required | Default |
| --- | --- | --- | --- |
| `pipeline.workspace.root` | string | No | `/var/lib/narratio` |
| `pipeline.workspace.cleanup_after_archive` | bool | No | `false` |
| `pipeline.secrets.env_dir` | string | Conditional | none |
| `pipeline.storage.backend` | string | No | empty |
| `pipeline.storage.bucket` | string | No | empty |
| `pipeline.storage.prefix` | string | No | empty |
| `pipeline.storage.s3.bucket` | string | Conditional | empty |
| `pipeline.storage.s3.root_prefix` | string | No | `dnd` |
| `pipeline.storage.s3.region` | string | No | empty |
| `pipeline.storage.s3.endpoint` | string | No | empty |
| `pipeline.storage.s3.force_path_style` | bool | No | `false` |
| `pipeline.storage.s3.access_key_id_env` | string | No | `OBJECT_STORAGE_KEY_ID` |
| `pipeline.storage.s3.secret_access_key_env` | string | No | `OBJECT_STORAGE_KEY` |
| `pipeline.spool.root` | string | No | `/var/spool/narratio` |
| `pipeline.spool.delete_audio_after_archive` | bool | No | `false` |
| `pipeline.archive.enabled` | bool | No | `true` |
| `pipeline.archive.upload_run` | bool | No | `true` |
| `pipeline.archive.promote_artifacts[]` | list | No | two default rules |
| `pipeline.archive.promote_artifacts[].from` | string | Yes (per rule) | none |
| `pipeline.archive.promote_artifacts[].to` | string | Yes (per rule) | none |
| `pipeline.archive.promote_artifacts[].required` | bool | No | `true` |
| `pipeline.whisperx.transcribe_url` | string | Yes | none |
| `pipeline.whisperx.language` | string | No | `en` |
| `pipeline.whisperx.timeout` | duration string | No | `30m` |
| `pipeline.whisperx.retries` | int | No | `3` |
| `pipeline.whisperx.retry_delay` | duration string | No | `2s` |
| `pipeline.whisperx.concurrency` | int | No | `2` |
| `pipeline.seriatim.binary` | string | No | `seriatim` |
| `pipeline.seriatim.timeout` | duration string | No | `10m` |
| `pipeline.seriatim.output_schema` | string | No | `seriatim-intermediate` |
| `pipeline.seriatim.coalesce_gap` | float | No | `3.0` |
| `pipeline.seriatim.report` | bool | No | `true` |
| `pipeline.seriatim.env.overlap_word_run_gap` | float | No | unset |
| `pipeline.seriatim.env.overlap_word_run_reorder_window` | float | No | unset |
| `pipeline.seriatim.env.backchannel_max_duration` | float | No | unset |
| `pipeline.seriatim.env.filler_max_duration` | float | No | unset |
| `pipeline.audita.binary` | string | No | `audita` |
| `pipeline.audita.timeout` | duration string | No | `3h` |
| `pipeline.audita.llm_api_key_env` | string | No | empty |
| `pipeline.audita.modules[]` | list[string] | No | empty |
| `pipeline.audita.base_url` | string | No | empty |
| `pipeline.audita.model` | string | No | empty |
| `pipeline.audita.total_llm_concurrency` | int | No | unset |
| `pipeline.audita.proposal_llm_concurrency` | int | No | unset |
| `pipeline.audita.validation_model` | string | No | empty |
| `pipeline.audita.validation_llm_concurrency` | int | No | unset |
| `pipeline.audita.transcript_description` | string | No | empty |
| `pipeline.audita.config_path` | string | No | empty |
| `pipeline.audita.output_schema` | string | No | empty |
| `pipeline.audita.work_dir_retention` | string | No | empty |
| `pipeline.audita.report` | bool | No | `true` |
| `pipeline.normalize.output_path` | string | No | `transcripts/normalized.json` |
| `pipeline.normalize.output_schema` | string | No | `seriatim-intermediate` |
| `pipeline.normalize.report` | bool | No | `true` |
| `pipeline.trim.enabled` | bool | No | `false` |
| `pipeline.trim.output_path` | string | Conditional | none |
| `pipeline.trim.bounds.prompt_id` | string | Conditional | none |
| `pipeline.trim.bounds.profile_id` | string | No | empty |
| `pipeline.trim.bounds.transcript_input_name` | string | Conditional | none |
| `pipeline.trim.bounds.output_path` | string | Conditional | none |
| `pipeline.trim.bounds.timeout` | duration string | No | `10m` |
| `pipeline.trim.bounds.render_debug` | bool | No | `false` |
| `pipeline.trim.bounds.render_output_path` | string | Conditional | none |
| `pipeline.trim.seriatim.report` | bool | No | `false` |
| `pipeline.scriptorium.binary` | string | No | `scriptorium` |
| `pipeline.scriptorium.config_path` | string | No | empty |
| `pipeline.scriptorium.timeout` | duration string | No | `10m` |
| `pipeline.scriptorium.render_debug` | bool | No | `false` |
| `pipeline.scriptorium.artifacts` | map | No | empty |
| `pipeline.scriptorium.artifacts.<name>.enabled` | bool | No | `false` |
| `pipeline.scriptorium.artifacts.<name>.render_debug` | bool | No | unset |
| `pipeline.scriptorium.artifacts.<name>.prompt_id` | string | Conditional | none |
| `pipeline.scriptorium.artifacts.<name>.profile_id` | string | No | empty |
| `pipeline.scriptorium.artifacts.<name>.output_path` | string | Conditional | none |
| `pipeline.scriptorium.artifacts.<name>.timeout` | duration string | No | empty |
| `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source` | string | Conditional | none |
| `pipeline.scriptorium.artifacts.<name>.inputs.<key>.artifact` | string | No | empty |
| `pipeline.scriptorium.artifacts.<name>.inputs.<key>.path` | string | No | empty |
| `pipeline.scriptorium.artifacts.<name>.inputs.<key>.required` | bool | No | `false` |
| `pipeline.scriptorium.artifacts.<name>.vars.<key>` | map value | No | empty |
| `pipeline.analyzer.binary_path` | string | No | empty |
| `pipeline.analyzer.timeout` | duration string | No | empty |
| `pipeline.analyzer.artifacts.output_dir` | string | No | empty |
| `pipeline.analyzer.artifacts.types[]` | list[string] | No | empty |
| `pipeline.notification.backend` | string | No | empty |
| `pipeline.notification.recipient` | string | No | empty |
| `pipeline.notification.timeout` | duration string | No | empty |

Allowed `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source` values:

- `previous_session_artifact`
- `narratio.transcript.merged`
- `narratio.transcript.polished`
- `narratio.transcript.full`
- `narratio.transcript.trimmed`
- `narratio.bounds.session`
- `narratio.artifact.session_recap`

## 8. Full session reference

| Path | Type | Required | Default |
| --- | --- | --- | --- |
| `session.session_id` | string | Yes | none |
| `session.campaign` | string | Yes | none |
| `session.date` | string | No | empty |
| `session.title` | string | No | empty |
| `session.inputs.audio_dir` | string | Conditional | empty |
| `session.inputs.audio_files[]` | list[string] | Conditional | empty |
| `session.inputs.audio_s3.prefix` | string | Conditional | none |
| `session.inputs.speakers_file` | string | Yes | none |
| `session.inputs.autocorrect_file` | string | Yes | none |
| `session.inputs.glossary_file` | string | Yes | none |

Audio-source rule:

- You must configure exactly one audio source mode:
  - `audio_dir`, or
  - `audio_files` (at least one), or
  - `audio_s3.prefix`
- `audio_s3` cannot be combined with `audio_dir` or `audio_files`.

## 9. Secrets

Narratio supports filesystem-based secret injection via `pipeline.secrets.env_dir`.

Behavior:

- `env_dir` may be absolute or relative.
- Relative `env_dir` is resolved from Narratio’s current working directory.
- Each top-level file with a valid env-var filename (`[A-Za-z_][A-Za-z0-9_]*`) is loaded.
- File contents become env-var values, with trailing `\n` / `\r\n` trimmed.
- Existing process environment variables are preserved and not overwritten.
- Invalid names and directories inside `env_dir` are skipped.
- Missing/unreadable `env_dir` fails command execution.

Guidance:

- Store secret values in secret files or pre-set environment variables.
- Do not put secret values directly in `pipeline.yml` or `session.yml`.
- Use config fields like `llm_api_key_env` and S3 credential env names to reference secret variable names, not secret data.

## 10. Examples

Maintained config examples:

- `docs/examples/pipeline.minimal.yml`
- `docs/examples/pipeline.production.yml`
- `docs/examples/pipeline.full.annotated.yml`
- `docs/examples/session.template.yml`
- `docs/examples/session.local-audio.yml`
- `docs/examples/session.s3-audio.yml`

These examples are covered by configuration load/validate tests in `internal/config`.