narratio/docs/config.md

# Configuration

## 1. Overview

Narratio loads three YAML files:

- `pipeline.yml`: pipeline-level runtime configuration.
- `campaign.yml`: stable campaign identity and campaign-level input defaults.
- `session.yml`: per-session metadata and input selection, loaded locally or from the configured S3 backend.

These commands load and validate all three files before running:

- `narratio run`
- `narratio plan`
- `narratio resume`
- `narratio run-stage`
- `narratio restore`

Behavior:

- strict YAML decode is enabled (`KnownFields(true)`): unknown fields fail.
- ordinary local and remote `session.yml` files must be concrete YAML; template placeholders are rejected.
- defaults are applied for optional pipeline fields.
- campaign-level stable input paths fill missing session input paths.
- session-level stable input paths override campaign-level input paths.
- campaign config may point `session init` to a session template.
- validation enforces required fields, value formats, and cross-field constraints.

## 2. Config file discovery

These commands use the same config discovery behavior:

- `narratio run`
- `narratio plan`
- `narratio resume`
- `narratio run-stage`
- `narratio restore`

Pipeline config lookup:

- if `--config <path>` is provided, that path is used.
- if omitted, Narratio searches in order:
  1. `/usr/local/etc/narratio/pipeline.yml`
  2. `/etc/narratio/pipeline.yml`
- first existing file wins.

Campaign config lookup:

- if `--campaign <path>` is provided, that path is used.
- if omitted, Narratio searches in order:
  1. `/usr/local/etc/narratio/campaign.yml`
  2. `/etc/narratio/campaign.yml`
- first existing file wins.

Session config lookup:

- if `--session <path>` is provided, that path is used.
- if `--session` is omitted, Narratio searches locally in order:
  1. `/usr/local/etc/narratio/session.yml`
  2. `/etc/narratio/session.yml`
- first existing local file wins.
- if no local session file is found, `--session-id <value>` is present, storage is configured, and campaign identity is resolved, Narratio loads remote `session.yml` from:
  - `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/session.yml`
- local discovery always runs before remote fallback.
- local files in the current working directory are used only when passed explicitly, for example `--config ./pipeline.yml --campaign ./campaign.yml --session ./session.yml`.

## 3. Session templating

Template behavior for local and remote `session.yml` loaded by downstream commands:

- downstream commands do not render templates.
- local and remote `session.yml` must be concrete.
- any `{{ ... }}` placeholder in loaded `session.yml` fails with guidance to run `narratio session init`.
- if concrete `session_id` mismatches `--session-id`, load fails.
- if concrete `previous_session_id` mismatches `--previous-session-id`, load fails.

Template behavior for `narratio session init`:

- `campaign.yml` may set `session_template_file`.
- relative template paths resolve relative to `campaign.yml`.
- supported init template variables:
  - `{{ session_id }}`
  - `{{ previous_session_id }}`
  - `{{ date }}`
  - `{{ title }}`
  - `{{ audio_s3_prefix }}`
  - `{{ audio_dir }}`
- each template variable must be supplied by the matching `session init` flag.
- template-related flags such as `--date`, `--title`, `--audio-s3-prefix`, `--audio-dir`, and `--previous-session-id` fail if the configured template does not use them.
- rendered output is strict-decoded and validated before it is written locally or remotely.
- if `session_template_file` is omitted, `session init` generates the minimal concrete session YAML directly.

## 4. Minimal config set

### `pipeline.yml`

```yaml
whisperx:
  transcribe_url: "https://transcription.example.com/transcribe"
```

Why this is sufficient:

- `whisperx.transcribe_url` is required.
- `workspace.root` defaults to `/var/lib/narratio`.
- optional sections (`seriatim`, `audita`, `archive`, `scriptorium`, `trim`, `normalize`, etc.) receive defaults or stay inactive.

### `campaign.yml`

```yaml
campaign: sample-campaign
session_template_file: ./session.template.yml
inputs:
  speakers_file: ./speakers.yml
  autocorrect_file: ./autocorrect.yml
  glossary_file: ./glossary.yml
```

Why this is sufficient:

- `campaign` supplies the stable campaign identity.
- stable input files are required and resolve relative to `campaign.yml` when copied during `prepare`.

### `session.yml`

```yaml
session_id: 2026-05-03
inputs:
  audio_dir: ./audio
```

Why this is sufficient:

- `session_id` is required.
- `campaign` can be omitted because it is supplied by `campaign.yml`.
- stable input paths can be omitted because `campaign.yml` supplies defaults.
- local `audio_dir` resolves relative to `session.yml`.

Minimal local-file usage:

```bash
narratio run --config /path/to/pipeline.yml --campaign ./campaign.yml --session ./session.yml --session-id 2026-05-03
```

Previous-session-enabled variant:

```yaml
session_id: 2026-05-03
previous_session_id: 2026-04-26
inputs:
  audio_dir: ./audio
```

```bash
narratio run --config /path/to/pipeline.yml --campaign ./campaign.yml --session ./session.yml --session-id 2026-05-03 --previous-session-id 2026-04-26
```

## 5. Production-oriented config set

### `pipeline.yml`

```yaml
workspace:
  root: /var/lib/narratio/workspace
  cleanup_after_archive: true

storage:
  backend: s3
  s3:
    bucket: my-dnd-archive
    root_prefix: dnd
    region: us-east-1
    access_key_id_env: OBJECT_STORAGE_KEY_ID
    secret_access_key_env: OBJECT_STORAGE_KEY

spool:
  root: /var/spool/narratio
  delete_audio_after_archive: true

cache:
  root: /var/cache/narratio
  s3_audio: true

archive:
  enabled: true
  upload_run: true
  promote_artifacts:
    - source: narratio.transcript.final_trimmed
      dest: transcripts/final.trimmed.json
      required: true
    - source: narratio.artifact.session_recap
      dest: artifacts/session_recap.md
      required: true
  locks:
    - source: narratio.artifact.session_recap
      reason: Final recap was manually edited.

whisperx:
  transcribe_url: "https://transcription.example.com/transcribe"

scriptorium:
  artifacts:
    session_recap:
      enabled: true
      prompt_id: dnd.session_recap
      output_path: artifacts/session_recap.md
      inputs:
        transcript:
          source: narratio.transcript.final_trimmed
          required: true
        previous_recap:
          source: narratio.previous_session.artifact.session_recap
          required: false
```

### `campaign.yml`

```yaml
campaign: forsaken
inputs:
  speakers_file: /srv/narratio/campaigns/forsaken/speakers.yml
  autocorrect_file: /srv/narratio/campaigns/forsaken/autocorrect.yml
  glossary_file: /srv/narratio/campaigns/forsaken/glossary.yml
```

### Local `session.yml`

```yaml
session_id: 2026-05-03
previous_session_id: 2026-04-26
date: 2026-05-03
title: The Black Cabin
inputs:
  audio_s3:
    prefix: audio/
```

### S3-first session config

For S3-first operation, upload the same `session.yml` content to:

```text
{root_prefix}/campaigns/{campaign}/sessions/{session_id}/session.yml
```

Then run with explicit or discovered pipeline/campaign config and no `--session`:

```bash
narratio run --config /usr/local/etc/narratio/pipeline.yml --campaign /usr/local/etc/narratio/campaign.yml --session-id 2026-05-03 --previous-session-id 2026-04-26
```

Operational notes:

- archive promotion is explicit and source-based via `archive.promote_artifacts`.
- `source` is required; `dest` is optional and derived when omitted.
- `archive.locks` skips top-level promotion overwrites for static locked sources while preserving run-local uploads.
- operator-created mutable locks are stored at `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/locks.yml` and are merged with static locks.
- Narratio does not auto-promote all generated analyze artifacts.
- `restore` reads the same config/campaign/session inputs and restore scope is bounded by committed archive current state.
- `clean` removes workspace/spool state by default and preserves `pipeline.cache.root` unless `--clear-cache` is passed.

## 6. Full pipeline reference

| Path | Type | Required | Default |
| --- | --- | --- | --- |
| `pipeline.workspace.root` | string | No | `/var/lib/narratio` |
| `pipeline.workspace.cleanup_after_archive` | bool | No | `false` |
| `pipeline.secrets.env_dir` | string | Conditional | none |
| `pipeline.storage.backend` | string | No | empty |
| `pipeline.storage.s3.bucket` | string | Conditional | empty |
| `pipeline.storage.s3.root_prefix` | string | No | `dnd` |
| `pipeline.storage.s3.region` | string | No | empty |
| `pipeline.storage.s3.endpoint` | string | No | empty |
| `pipeline.storage.s3.force_path_style` | bool | No | `false` |
| `pipeline.storage.s3.access_key_id_env` | string | No | `OBJECT_STORAGE_KEY_ID` |
| `pipeline.storage.s3.secret_access_key_env` | string | No | `OBJECT_STORAGE_KEY` |
| `pipeline.spool.root` | string | No | `/var/spool/narratio` |
| `pipeline.spool.delete_audio_after_archive` | bool | No | `false` |
| `pipeline.cache.root` | string | No | `/var/cache/narratio` |
| `pipeline.cache.s3_audio` | bool | No | `true` |
| `pipeline.archive.enabled` | bool | No | `true` |
| `pipeline.archive.upload_run` | bool | No | `true` |
| `pipeline.archive.promote_artifacts[]` | list | No | final-trimmed transcript rule |
| `pipeline.archive.promote_artifacts[].source` | string | Yes (per rule) | none |
| `pipeline.archive.promote_artifacts[].dest` | string | No | derived from source |
| `pipeline.archive.promote_artifacts[].required` | bool | No | `true` |
| `pipeline.archive.locks[]` | list | No | empty |
| `pipeline.archive.locks[].source` | string | Yes (per lock) | none |
| `pipeline.archive.locks[].reason` | string | No | empty |
| `pipeline.whisperx.transcribe_url` | string | Yes | none |
| `pipeline.whisperx.language` | string | No | `en` |
| `pipeline.whisperx.timeout` | duration string | No | `30m` |
| `pipeline.whisperx.retries` | int | No | `3` |
| `pipeline.whisperx.retry_delay` | duration string | No | `2s` |
| `pipeline.whisperx.concurrency` | int | No | `2` |
| `pipeline.seriatim.binary` | string | No | `seriatim` |
| `pipeline.seriatim.timeout` | duration string | No | `10m` |
| `pipeline.seriatim.output_schema` | string | No | `seriatim-intermediate` |
| `pipeline.seriatim.coalesce_gap` | float | No | `3.0` |
| `pipeline.seriatim.report` | bool | No | `true` |
| `pipeline.seriatim.env.overlap_word_run_gap` | float | No | unset |
| `pipeline.seriatim.env.overlap_word_run_reorder_window` | float | No | unset |
| `pipeline.seriatim.env.backchannel_max_duration` | float | No | unset |
| `pipeline.seriatim.env.filler_max_duration` | float | No | unset |
| `pipeline.audita.binary` | string | No | `audita` |
| `pipeline.audita.timeout` | duration string | No | `3h` |
| `pipeline.audita.llm_api_key_env` | string | No | empty |
| `pipeline.audita.modules[]` | list[string] | No | empty |
| `pipeline.audita.base_url` | string | No | empty |
| `pipeline.audita.model` | string | No | empty |
| `pipeline.audita.total_llm_concurrency` | int | No | unset |
| `pipeline.audita.proposal_llm_concurrency` | int | No | unset |
| `pipeline.audita.validation_model` | string | No | empty |
| `pipeline.audita.validation_llm_concurrency` | int | No | unset |
| `pipeline.audita.transcript_description` | string | No | empty |
| `pipeline.audita.config_path` | string | No | empty |
| `pipeline.audita.output_schema` | string | No | empty |
| `pipeline.audita.work_dir_retention` | string | No | empty |
| `pipeline.audita.report` | bool | No | `true` |
| `pipeline.normalize.output_path` | string | No | `transcripts/final.json` |
| `pipeline.normalize.output_schema` | string | No | `seriatim-intermediate` |
| `pipeline.normalize.report` | bool | No | `true` |
| `pipeline.trim.enabled` | bool | No | `false` |
| `pipeline.trim.output_path` | string | Conditional | none |
| `pipeline.trim.bounds.prompt_id` | string | Conditional | none |
| `pipeline.trim.bounds.profile_id` | string | No | empty |
| `pipeline.trim.bounds.transcript_input_name` | string | Conditional | none |
| `pipeline.trim.bounds.output_path` | string | Conditional | none |
| `pipeline.trim.bounds.timeout` | duration string | No | `10m` |
| `pipeline.trim.bounds.render_debug` | bool | No | `false` |
| `pipeline.trim.bounds.render_output_path` | string | Conditional | none |
| `pipeline.trim.seriatim.report` | bool | No | `false` |
| `pipeline.scriptorium.binary` | string | No | `scriptorium` |
| `pipeline.scriptorium.config_path` | string | No | empty |
| `pipeline.scriptorium.timeout` | duration string | No | `10m` |
| `pipeline.scriptorium.render_debug` | bool | No | `false` |
| `pipeline.scriptorium.artifacts` | map | No | empty |
| `pipeline.scriptorium.artifacts.<name>.enabled` | bool | No | `false` |
| `pipeline.scriptorium.artifacts.<name>.depends_on[]` | list[string] | No | empty |
| `pipeline.scriptorium.artifacts.<name>.render_debug` | bool | No | unset |
| `pipeline.scriptorium.artifacts.<name>.prompt_id` | string | Conditional | none |
| `pipeline.scriptorium.artifacts.<name>.profile_id` | string | No | empty |
| `pipeline.scriptorium.artifacts.<name>.output_path` | string | Conditional | none |
| `pipeline.scriptorium.artifacts.<name>.timeout` | duration string | No | empty |
| `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source` | string | Conditional | none |
| `pipeline.scriptorium.artifacts.<name>.inputs.<key>.artifact` | string | No | empty |
| `pipeline.scriptorium.artifacts.<name>.inputs.<key>.path` | string | No | empty |
| `pipeline.scriptorium.artifacts.<name>.inputs.<key>.required` | bool | No | `false` |
| `pipeline.scriptorium.artifacts.<name>.vars.<key>` | map value | No | empty |
| `pipeline.notification.backend` | string | No | empty |
| `pipeline.notification.recipient` | string | No | empty |
| `pipeline.notification.timeout` | duration string | No | empty |

Scriptorium artifact-key and dependency rules:

- artifact keys must match `^[a-z][a-z0-9_]*$`.
- enabled artifacts require `prompt_id` and `output_path`.
- `output_path` must be relative, traversal-safe, and under `artifacts/`.
- configured artifact input sources use `narratio.artifact.<name>`.
- if input source references `narratio.artifact.<name>`, artifact `<name>` must exist and must be listed in `depends_on`.
- every `depends_on` entry must be a configured artifact key.
- self-dependency is rejected.
- enabled dependency cycles are rejected.
- any artifact referenced by `depends_on` or `narratio.artifact.<name>` source must define `output_path` (even if not enabled).

Allowed `pipeline.scriptorium.artifacts.<name>.inputs.<key>.source` values:

- `narratio.previous_session.artifact.<configured_artifact_key>`
- `narratio.transcript.base`
- `narratio.transcript.polished`
- `narratio.transcript.final`
- `narratio.transcript.final_trimmed`
- `narratio.bounds.session`
- `narratio.artifact.<configured_artifact_key>`

`pipeline.archive.promote_artifacts[].source` values:

- `narratio.transcript.base`
- `narratio.transcript.polished`
- `narratio.transcript.final`
- `narratio.transcript.final_trimmed`
- `narratio.bounds.session`
- `narratio.artifact.<configured_artifact_key>`

`pipeline.archive.locks[].source` accepts the same source values as `pipeline.archive.promote_artifacts[].source`.

Archive promotion destination rules:

- `dest` must be a clean relative path (not absolute, no traversal).
- duplicate `dest` values are rejected.
- if `dest` is omitted:
  - built-in sources derive their canonical destination path;
  - configured sources derive from `pipeline.scriptorium.artifacts.<name>.output_path`;
  - derivation failure is a config validation error.

Archive lock rules:

- locks are source-based and do not accept `dest`.
- duplicate lock sources are rejected.
- static `pipeline.archive.locks` win over remote mutable locks for the same source.
- locked promotions are recorded as intentional skips in archive metadata.
- locked required promotions do not fail archive by default.
- ordinary `--force` reruns do not override locks.

Remote mutable lock store:

- path: `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/locks.yml`.
- strict YAML shape: top-level `locks`, each with `source` and optional `reason`.
- `narratio locks add` and `narratio locks remove` mutate only the remote lock store.
- writes use existence checks plus `--force` for updates; they are not compare-and-swap atomic.

Restore-related implications:

- restore remote identity requires archive S3 identity to resolve (`pipeline.storage.s3.bucket` and session prefix derivation inputs).
- restore scope considers committed current state and durable paths (`manifest.json`, `transcripts/**`, `artifacts/**`, `previous/**`, optional `audio/**`).
- S3 audio downloads use `pipeline.spool.root` for active downloads and `pipeline.cache.root` for reusable cached audio when `pipeline.cache.s3_audio` is true.
- `pipeline.cache.root` is durable local cache state. It is not workspace state and is preserved by default by `narratio clean`.

## 7. Full campaign reference

| Path | Type | Required | Default |
| --- | --- | --- | --- |
| `campaign.campaign` | string | Yes | none |
| `campaign.session_template_file` | string | No | none |
| `campaign.inputs.speakers_file` | string | Yes | none |
| `campaign.inputs.autocorrect_file` | string | Yes | none |
| `campaign.inputs.glossary_file` | string | Yes | none |

Campaign input paths and `campaign.session_template_file` may be absolute or relative. Relative paths resolve from the directory containing `campaign.yml`.

## 8. Full session reference

| Path | Type | Required | Default |
| --- | --- | --- | --- |
| `session.session_id` | string | Yes | none |
| `session.previous_session_id` | string | No | empty |
| `session.campaign` | string | No | `campaign.campaign` |
| `session.date` | string | No | empty |
| `session.title` | string | No | empty |
| `session.inputs.audio_dir` | string | Conditional | empty |
| `session.inputs.audio_files[]` | list[string] | Conditional | empty |
| `session.inputs.audio_s3.prefix` | string | Conditional | none |
| `session.inputs.speakers_file` | string | No | `campaign.inputs.speakers_file` |
| `session.inputs.autocorrect_file` | string | No | `campaign.inputs.autocorrect_file` |
| `session.inputs.glossary_file` | string | No | `campaign.inputs.glossary_file` |

Session input paths may be absolute or relative. Relative audio paths and session-level stable input overrides resolve from the directory containing `session.yml`. If both `campaign.yml` and `session.yml` specify campaign identity, the values must match.

Audio-source rule:

- configure exactly one mode:
  - `audio_dir`, or
  - `audio_files` (at least one), or
  - `audio_s3.prefix`
- `audio_s3` cannot be combined with local audio fields.

Previous-session rule:

- if `session.previous_session_id` is set, it must not equal `session.session_id`.
- canonical previous-session sources (`narratio.previous_session.artifact.<name>`) are hydrated during `prepare` from archive current state when required by enabled configured artifacts.

## 9. Secrets

Narratio supports filesystem-based secret injection via `pipeline.secrets.env_dir`.

Behavior:

- `env_dir` may be absolute or relative.
- relative `env_dir` resolves from current working directory.
- files with valid env-var names (`[A-Za-z_][A-Za-z0-9_]*`) are loaded.
- values are loaded from file contents with trailing newline trimming.
- existing process env vars are preserved.
- invalid names and subdirectories are skipped.
- missing/unreadable `env_dir` fails command execution.

Guidance:

- do not put secret values directly in YAML.
- configure env var names in config and provide values via env/secrets files.

## 10. Examples

Maintained examples:

- `examples/pipeline.minimal.yml`
- `examples/pipeline.production.yml`
- `examples/pipeline.full.annotated.yml`
- `examples/campaign.yml`
- `examples/session.template.yml`
- `examples/session.local-audio.yml`
- `examples/session.s3-audio.yml`

These examples are validated by `internal/config` tests.