narratio/README.md

# narratio

`narratio` is a Go-based orchestration application for processing D&D session audio into transcripts and downstream artifacts.

This repository currently contains a **working scaffold** with strict config loading, local workdir/manifest handling, resumable stage control, real WhisperX/Seriatim/Audita adapters, and real `prepare` + `transcribe` + `merge` + `polish` stages.

## Expected Config Files

`narratio` expects two YAML files:

- `pipeline.yml`: pipeline/workspace settings (`workspace`, `storage`, `whisperx`, `seriatim`, `audita`, optional `scriptorium`, `analyzer`, `notification`)
- `session.yml`: per-session settings (`session_id`, `inputs`, optional metadata)

WhisperX config contract in `pipeline.yml`:

- required: `whisperx.transcribe_url`
- defaulted when omitted: `whisperx.language` (`en`), `whisperx.timeout` (`30m`), `whisperx.retries` (`3`), `whisperx.retry_delay` (`2s`), `whisperx.concurrency` (`2`)

Seriatim config contract in `pipeline.yml`:

- required: `seriatim.binary` (name or path; existence is checked at execution time, not config validation time)
- defaulted when omitted: `seriatim.timeout` (`10m`), `seriatim.output_schema` (`seriatim-intermediate`), `seriatim.coalesce_gap` (`3.0`), `seriatim.report` (`true`)
- allowed `seriatim.output_schema` values: `seriatim-minimal`, `seriatim-intermediate`, `seriatim-full`
- optional tuning: `seriatim.env.*` (`overlap_word_run_gap`, `overlap_word_run_reorder_window`, `backchannel_max_duration`, `filler_max_duration`) must be `> 0` when provided

Audita config contract in `pipeline.yml`:

- required: `audita.binary` (name or path; existence is checked at execution time, not config validation time)
- optional: `audita.llm_api_key_env` (environment variable name holding the API key secret; no automatic default)
- defaulted when omitted: `audita.timeout` (`3h`), `audita.modules` (`glossary,homophones,glossary,spoken_word,grammar,homophones,glossary`), `audita.base_url` (`https://openrouter.ai/api/v1`), `audita.model` (`openrouter/google/gemma-4-31b-it`), `audita.llm_concurrency` (`1`), `audita.validation_model` (`""`), `audita.validation_llm_concurrency` (`1`), `audita.report` (`true`)
- allowed `audita.modules` values: `glossary`, `homophones`, `spoken_word`, `grammar` (order and repeats are allowed)
- `audita.base_url` must be a valid URL when provided
- `audita.llm_concurrency` and `audita.validation_llm_concurrency` must be `> 0`

Audita credentials note:

- store only the environment variable **name** in config (`audita.llm_api_key_env`), never the API key value itself
- API key values must not be written to pipeline config, generated configs, logs, or manifest metadata
- if `audita.llm_api_key_env` is configured and the named env var is not set (or is empty), Narratio fails before invocation with a redacted error
- if `audita.llm_api_key_env` is omitted/empty, Narratio does not require a credential and omits `AUDITA_LLM_API_KEY` from the subprocess overrides

Audita runtime note:

- Narratio currently passes primary LLM concurrency via `AUDITA_LLM_CONCURRENCY` subprocess environment override, not a `--llm-concurrency` flag.

Scriptorium config contract in `pipeline.yml` (optional until `analyze` is implemented):

- optional section: `scriptorium` (when omitted, current pipeline behavior is unchanged)
- required when `scriptorium` is present: `scriptorium.binary` (name on PATH or full path)
- optional: `scriptorium.config_path` (if set, it must be non-empty; file existence is validated at execution time, not basic config validation time)
- defaulted when omitted inside present `scriptorium` section: `scriptorium.timeout` (`10m`)
- optional: `scriptorium.render_debug` (`false` by default through YAML zero-value behavior)
- artifact definitions live under `scriptorium.artifacts.<artifact_name>` and support multiple configured artifacts
- enabled artifacts require `prompt_id` and `output_path`
- artifact `timeout` values must be valid Go durations when provided
- input definitions live under `scriptorium.artifacts.<artifact_name>.inputs.<input_name>` and require non-empty `source`
- `vars` values currently accept booleans and strings only
- Narratio does not store secrets in Scriptorium config; keep credentials in environment management, not YAML

Scriptorium initial artifact note:

- this pass does not auto-create `session_recap`; configure `scriptorium.artifacts.session_recap` explicitly in `pipeline.yml`

`speakers.yml` note:

- use Seriatim’s documented `match:` format (not the legacy direct mapping style used by older scripts/scaffolds)
- see [`examples/speakers.yml`](examples/speakers.yml) for a concrete `match:` example.

Prepared inputs note:

- `prepare` copies session inputs into the session workdir (`work/<session_id>/inputs`), and downstream stages consume those prepared copies, not the original source files.
- If you edit `speakers.yml`, `autocorrect.yml`, `glossary.yml`, or session input paths after `prepare` has run, rerun `prepare` with `--force` before rerunning downstream stages.

Decoding is strict (`KnownFields(true)`), so unknown YAML fields fail fast.

Example minimal files are available under `examples/`:

- `examples/pipeline.minimal.yml`
- `examples/session.minimal.yml`

## Current Scaffold Status

Implemented now:

- strict config load + validation
- local artifact/workdir creation and locking
- manifest create/load/save and stage status tracking
- stage framework with real `prepare`, `transcribe`, `merge`, and `polish` stages; placeholder downstream stages
- resumable run control (`run`, `resume`, `run-stage`, `plan` with run/skip decisions)
- real WhisperX HTTP adapter plus real Seriatim/Audita subprocess adapters (with fake/no-op adapters for test/scaffold usage)

Not implemented yet:

- real analyzer integration
- real remote archive/storage backend
- real notification backend

## Run Tests

```bash
go test ./...
```

## Run Plan

```bash
go run ./cmd/narratio plan --config examples/pipeline.minimal.yml --session examples/session.minimal.yml
```

## Run Pipeline (Current State)

The current `run` command executes `prepare` + real `transcribe` + real `merge` + real `polish` + placeholder downstream stages and records progress in `manifest.json`.

Default CLI wiring builds and uses:

- real WhisperX HTTP adapter from `pipeline.whisperx`
- real Seriatim subprocess adapter from `pipeline.seriatim`
- real Audita subprocess adapter from `pipeline.audita`

Subprocess runtime note:

- subprocesses inherit the parent environment by default, then apply Narratio override values (override values win).
- non-zero subprocess errors include stdout/stderr log paths and a short redacted stderr tail when available to speed diagnosis.

Real `polish` stage output paths:

- `transcripts/processed.json`
- `artifacts/audita.report.json` (when `audita.report: true`)
- `artifacts/audita-work`
- `logs/audita.stdout.log`
- `logs/audita.stderr.log`
- `config/audita.generated.yml`

```bash
go run ./cmd/narratio run --config examples/pipeline.minimal.yml --session examples/session.minimal.yml
```