narratio/README.md

narratio

narratio is a Go orchestration application for processing D&D session audio into transcripts and generated artifacts.

Current Implementation

Implemented now:

• strict config loading/validation (pipeline.yml and session.yml)
• local workspace/session layout, locking, and manifest persistence
• resumable stage control (run, plan, resume, run-stage, status)
• real prepare, transcribe, merge, polish, normalize, trim, and analyze stages
• real WhisperX, Seriatim, and Audita adapters
• real Scriptorium subprocess adapter
• optional Scriptorium render diagnostics (render_debug)

Not implemented yet:

• archive stage behavior
• notify stage behavior
• additional analyze artifacts beyond session_recap
• generic DAG orchestration

Config Files

Narratio expects two YAML files:

• pipeline.yml: pipeline/workspace settings
• session.yml: per-session settings

Pipeline config lookup for CLI commands:

• if --config <path> is provided, Narratio uses that path
• if --config is omitted, Narratio searches in this order:
  • /usr/local/etc/narratio/pipeline.yml
  • /etc/narratio/pipeline.yml

Optional secrets-from-files config:

• pipeline.secrets.env_dir may point to a directory of secret files
• each top-level file with an env-var-style name is loaded as an environment variable:
  • file name = env var name
  • file contents = env var value (trailing newline/CRLF trimmed)
• process environment wins: existing env vars are not overwritten
• if configured, Narratio fails fast when env_dir is missing/unreadable
• relative env_dir values resolve from Narratio’s current working directory

YAML decoding is strict (KnownFields(true)), so unknown fields fail fast.

Canonical Stage Order

1. prepare
2. transcribe
3. merge
4. polish
5. normalize
6. trim
7. analyze
8. archive
9. notify

Transcript Tiers

• transcripts/merged.json: canonical deterministic merged transcript from Seriatim merge
• transcripts/processed.json: full raw Audita-polished transcript output
• transcripts/normalized.json: Seriatim-normalized transcript from the normalize stage
• transcripts/trimmed.json: gameplay-only normalized polished transcript from trim stage

Normalize Configuration

pipeline.normalize is optional. When omitted, Narratio defaults to:

• output_path: transcripts/normalized.json
• output_schema: seriatim-intermediate
• report: true

Allowed normalize.output_schema values:

• seriatim-minimal
• seriatim-intermediate
• seriatim-full

normalize.output_path is treated as session-workdir-relative when not absolute.

Normalize stage behavior summary:

• normalize runs after polish and before trim
• normalize resolves transcripts/processed.json
• normalize runs Seriatim normalize to produce transcripts/normalized.json
• normalize diagnostics are written to:
  • artifacts/seriatim.normalize.report.json (when enabled)
  • logs/seriatim.normalize.stdout.log
  • logs/seriatim.normalize.stderr.log
  • config/seriatim.normalize.generated.yml

Trim Configuration

pipeline.trim is optional. If omitted, no trim config is loaded. If trim.enabled is omitted, it defaults to false.

When trim.enabled: true:

• trim.output_path is required
• trim.bounds.prompt_id is required
• trim.bounds.transcript_input_name is required
• trim.bounds.output_path is required
• trim.bounds.timeout must be a valid Go duration when provided
• trim.bounds.render_debug: true requires trim.bounds.render_output_path
• trim.bounds.profile_id may be empty to use the prompt default profile

Trim paths are treated as session-workdir-relative when not absolute.

Example trim config:

trim:
  enabled: true
  output_path: "transcripts/trimmed.json"
  bounds:
    prompt_id: "dnd_session.bounds"
    profile_id: ""
    transcript_input_name: "transcript"
    output_path: "artifacts/session_bounds.json"
    timeout: "10m"
    render_debug: false
    render_output_path: "artifacts/session_bounds.render.json"
  seriatim:
    report: false

Trim behavior summary:

• trim discovers and validates transcripts/normalized.json
• trim uses Scriptorium bounds (dnd_session.bounds by example config) to produce artifacts/session_bounds.json
• bounds IDs are validated against the same normalized transcript ID space that Seriatim trim will consume
• trim converts bounds to Seriatim keep selector (for example 10-868) and runs Seriatim trim
• if trim is disabled, Narratio copies normalized transcript to trimmed transcript and records trim_action=copy_disabled

Trim outputs and diagnostics:

• artifacts/session_bounds.json
• transcripts/trimmed.json
• logs/scriptorium.bounds.stdout.log
• logs/scriptorium.bounds.stderr.log
• config/scriptorium.bounds.generated.yml
• logs/seriatim.trim.stdout.log
• logs/seriatim.trim.stderr.log
• config/seriatim.trim.generated.yml
• optional bounds render-debug outputs:
  • artifacts/session_bounds.render.json
  • logs/scriptorium.bounds.render.stdout.log
  • logs/scriptorium.bounds.render.stderr.log
  • config/scriptorium.bounds.render.generated.yml

Render-debug files are diagnostics and are not treated as canonical stage output artifact refs.

Scriptorium Configuration

pipeline.scriptorium is optional. When present, Narratio validates and uses it for analyze-stage artifact generation.

Key points:

• scriptorium.binary is required when section is present
• scriptorium.config_path is optional
• scriptorium.timeout defaults to 10m when omitted
• scriptorium.render_debug enables render diagnostics globally
• artifacts are configured under scriptorium.artifacts (map shape supports multiple artifacts)
• enabled artifacts require prompt_id and output_path
• artifact render_debug may override global render setting
• vars currently support boolean and string values

Example session_recap artifact definition:

scriptorium:
  binary: "scriptorium"
  config_path: "/etc/scriptorium/config.yml"
  timeout: "10m"
  render_debug: false

  artifacts:
    session_recap:
      enabled: true
      prompt_id: "dnd.session_recap"
      profile_id: "local-quality" # optional
      output_path: "artifacts/session_recap.md"
      timeout: "10m"
      # render_debug: true # optional per-artifact override

      inputs:
        transcript:
          source: "trimmed_transcript"
          required: true

        previous_recap:
          source: "previous_session_artifact"
          artifact: "session_recap"
          path: "" # optional; set when available
          required: false

      vars:
        session_id: true
        session_date: true
        campaign_name: true
        previous_session_id: true
        output_kind: "session_recap"

Prompt IDs and profile IDs are configuration values. They are not hardcoded in analyze-stage logic.

Do not put secrets in pipeline.yml. If API-key behavior is configured, use env var names only.

If pipeline.secrets.env_dir is configured, keep only references and secret files there; secret values are still not written to manifests, generated configs, or Narratio-managed logs.

Scriptorium Runtime Behavior

Narratio integrates with Scriptorium through the public CLI subprocess contract:

• generation: scriptorium run
• diagnostics/testing: scriptorium render --format json when render_debug is enabled

For the initial implementation, only session_recap generation is supported.

Analyze-stage session recap behavior:

• available transcript input sources for configured artifacts: processed_transcript, normalized_transcript, trimmed_transcript
• session recap should use gameplay-only transcript input (source: trimmed_transcript)
• Narratio resolves trimmed_transcript from trim manifest output (transcript_trimmed) or fallback transcripts/trimmed.json
• Narratio resolves normalized_transcript from normalize manifest output (transcript_normalized) or fallback transcripts/normalized.json
• missing trimmed transcript fails clearly and advises running trim stage first
• normalized_transcript is the preferred full-transcript source for future table/meta-analysis artifacts
• processed_transcript remains supported for advanced/debug use cases
• optionally includes previous_recap when configured and resolvable
• omits optional previous recap when unavailable
• fails if required inputs are missing
• validates output file exists and is non-empty

Expected session output paths:

• artifacts/session_recap.md
• logs/scriptorium.session_recap.stdout.log
• logs/scriptorium.session_recap.stderr.log
• config/scriptorium.session_recap.generated.yml
• artifacts/session_recap.render.json when render diagnostics are enabled

Examples

Starter files:

• examples/pipeline.minimal.yml
• examples/session.minimal.yml
• examples/speakers.yml

Commands

Run tests:

go test ./...

Plan a run:

go run ./cmd/narratio plan --session examples/session.minimal.yml

Use --config <path> to override default pipeline lookup when needed.

Run full pipeline:

go run ./cmd/narratio run --config examples/pipeline.minimal.yml --session examples/session.minimal.yml

Run analyze only:

go run ./cmd/narratio run-stage --config examples/pipeline.minimal.yml --session examples/session.minimal.yml analyze

Operational Note

Checksum-based stale detection is not implemented yet.

If prepared inputs or prompt/runtime config change, rerun the appropriate upstream stages before relying on downstream artifacts.

Examples:

• glossary/autocorrect/speaker-context changes: rerun at least merge, polish, normalize, trim, and analyze
• trim bounds prompt/profile/config changes: rerun at least normalize, trim, and analyze
• session recap prompt/profile/input-source changes: rerun analyze

Roadmap

Near-term roadmap:

• extend analyze to additional configured artifacts
• support workflows where later artifacts consume earlier generated artifacts
• keep orchestration explicit without a generic DAG engine
• implement archive and notify backends