strict config loading/validation (pipeline.yml and session.yml)
local workspace/session layout, locking, and manifest persistence
resumable stage control (run, plan, resume, run-stage, status)
real prepare, transcribe, merge, polish, normalize, trim, and analyze stages
real WhisperX, Seriatim, and Audita adapters
real Scriptorium subprocess adapter
optional Scriptorium render diagnostics (render_debug)

Not implemented yet:

notify stage behavior
additional analyze artifacts beyond session_recap
generic DAG orchestration

Config Files

Narratio expects two YAML files:

pipeline.yml: pipeline/workspace settings
session.yml: per-session settings

Pipeline config lookup for CLI commands:

if --config <path> is provided, Narratio uses that path
if --config is omitted, Narratio searches in this order:
- /usr/local/etc/narratio/pipeline.yml
- /etc/narratio/pipeline.yml

Session config lookup for CLI commands:

if --session <path> is provided, Narratio uses that path
if --session is omitted, Narratio searches in this order:
- ./session.yml
- /usr/local/etc/narratio/session.yml
- /etc/narratio/session.yml

Session template support:

Narratio renders session.yml templates before strict YAML decode.
--session-id <value> provides the session_id template variable.
Supported placeholder forms:
- {{session_id}}
- {{ session_id }}
unresolved template placeholders fail with a clear error.
strict YAML validation still runs after rendering.
concrete session.yml files without templates remain fully supported.

Optional secrets-from-files config:

pipeline.secrets.env_dir may point to a directory of secret files
each top-level file with an env-var-style name is loaded as an environment variable:
- file name = env var name
- file contents = env var value (trailing newline/CRLF trimmed)
process environment wins: existing env vars are not overwritten
if configured, Narratio fails fast when env_dir is missing/unreadable
relative env_dir values resolve from Narratio’s current working directory

YAML decoding is strict (KnownFields(true)), so unknown fields fail fast.

Storage And Archive Foundations

Narratio now includes configuration and path-model foundations for archive support, plus implemented prepare-stage S3 audio input.

Implemented foundations:

pipeline.storage.s3 config shape (bucket, root_prefix, region, endpoint, force_path_style, access_key_id_env, secret_access_key_env)
pipeline.spool config shape (root, delete_audio_after_archive)
pipeline.archive config shape (enabled, upload_run, promote_artifacts)
promotion-rule validation (from/to required, relative-only paths, traversal rejected)
session.campaign requirement for campaign-aware path construction
optional session.inputs.audio_s3.prefix modeling and prepare-stage S3 audio download
run ID generation and S3/local path helper foundations
manifest run/path identity fields

Current defaults:

pipeline.storage.s3.root_prefix: dnd
pipeline.storage.s3.access_key_id_env: OBJECT_STORAGE_KEY_ID
pipeline.storage.s3.secret_access_key_env: OBJECT_STORAGE_KEY
pipeline.workspace.cleanup_after_archive: false
pipeline.spool.root: /var/spool/narratio
pipeline.spool.delete_audio_after_archive: false
pipeline.archive.enabled: true
pipeline.archive.upload_run: true
default pipeline.archive.promote_artifacts:
- transcripts/trimmed.json -> transcripts/trimmed.json (required: true)
- artifacts/session_recap.md -> artifacts/session_recap.md (required: true)

Current boundaries:

local development audio (audio_dir / audio_files) still works
audio_dir/audio_files and audio_s3 are mutually exclusive
real S3-compatible backend now exists in the storage adapter package
storage backend tests use fake storage and do not require live S3
archive uploads successful run records under runs/{run_id}/
archive does not upload local audio by default
archive uploads promoted outputs to session-level keys using archive.promote_artifacts
archive uploads current/manifest.json
archive uploads current/run_id.txt last as the effective commit marker
required missing promotions fail archive
optional missing promotions are skipped and recorded
cleanup remains conservative and opt-in:
- pipeline.spool.delete_audio_after_archive: true removes only the run-scoped spool audio directory after successful archive commit
- pipeline.workspace.cleanup_after_archive: true removes only the run-scoped local workdir after successful archive commit
- cleanup executes only after all selected stages for the command invocation succeed
- cleanup does not run for failed, incomplete, skipped, or unarchived runs
local development audio_dir/audio_files source inputs are never deleted by spool cleanup
S3 credentials are resolved from configured env-var names when both are present; if either is missing, Narratio falls back to the AWS SDK default credential chain

S3 input details and current boundaries are documented in docs/s3-audio-input.md.

Remote Storage Backend

Narratio includes an object-store backend layer for future prepare/archive work:

List(ctx, prefix)
Download(ctx, key, localPath)
Upload(ctx, localPath, key, opts)
Exists(ctx, key)

Implemented backends:

fake storage backend for deterministic tests
S3-compatible backend built from pipeline.storage.s3

Key invariant:

callers pass full bucket-relative object keys
storage backends do not prepend root_prefix and do not infer session/campaign paths

Current boundary:

prepare uses List + Download through the backend when session.inputs.audio_s3 is configured
archive uses Upload through the backend for successful run-record uploads
archive also uses Upload for promotion writes and current pointers
no failed or incomplete runs are uploaded
local audio is not re-uploaded by default

Archive run-upload details and boundaries are documented in docs/archive-storage.md.

Canonical Stage Order

prepare
transcribe
merge
polish
normalize
trim
analyze
archive
notify

Transcript Tiers

transcripts/merged.json: canonical deterministic merged transcript from Seriatim merge
transcripts/processed.json: full raw Audita-polished transcript output
transcripts/normalized.json: Seriatim-normalized transcript from the normalize stage
transcripts/trimmed.json: gameplay-only normalized polished transcript from trim stage

Seriatim Configuration

pipeline.seriatim configures the Seriatim subprocess adapter used by merge, normalize, and trim.

Minimal behavior:

pipeline.seriatim may be omitted entirely.
when omitted, Narratio defaults to:
- binary: seriatim
- timeout: 10m
- output_schema: seriatim-intermediate
- coalesce_gap: 3.0
- report: true

Optional overrides in pipeline.seriatim continue to work, including explicit binary paths and advanced env tuning values.

Audita Configuration

pipeline.audita configures the real Audita subprocess adapter used by polish.

Minimal behavior:

pipeline.audita may be omitted entirely.
when omitted, Narratio defaults to:
- binary: audita
- timeout: 3h
- report: true

Optional:

llm_api_key_env (when set, Narratio requires that env var and passes it to Audita as AUDITA_LLM_API_KEY)
modules override list (when empty/omitted, Narratio does not pass --modules)
base_url (when omitted, Narratio does not pass --base-url; Audita runtime defaults/config may apply)
model (when omitted, Narratio does not pass --model; Audita runtime defaults/config may apply)
transcript_description
config_path
output_schema (bare-segments or audita-v1)
work_dir_retention (always, auto, or never)
total_llm_concurrency (> 0 when provided)
proposal_llm_concurrency (> 0 when provided)
validation_model
validation_llm_concurrency (> 0 when provided)
report (defaults to true)

Narratio passes only configured optional Audita flags. Omitted optional values are left to Audita runtime defaults/config.

Normalize Configuration

pipeline.normalize is optional. When omitted, Narratio defaults to:

output_path: transcripts/normalized.json
output_schema: seriatim-intermediate
report: true

Allowed normalize.output_schema values:

seriatim-minimal
seriatim-intermediate
seriatim-full

normalize.output_path is treated as session-workdir-relative when not absolute.

Normalize stage behavior summary:

normalize runs after polish and before trim
normalize resolves transcripts/processed.json
normalize runs Seriatim normalize to produce transcripts/normalized.json
normalize diagnostics are written to:
- artifacts/seriatim.normalize.report.json (when enabled)
- logs/seriatim.normalize.stdout.log
- logs/seriatim.normalize.stderr.log
- config/seriatim.normalize.generated.yml

Trim Configuration

pipeline.trim is optional. If omitted, no trim config is loaded. If trim.enabled is omitted, it defaults to false.

When trim.enabled: true:

trim.output_path is required
trim.bounds.prompt_id is required
trim.bounds.transcript_input_name is required
trim.bounds.output_path is required
trim.bounds.timeout must be a valid Go duration when provided
trim.bounds.render_debug: true requires trim.bounds.render_output_path
trim.bounds.profile_id may be empty to use the prompt default profile

Trim paths are treated as session-workdir-relative when not absolute.

Example trim config:

trim:
  enabled: true
  output_path: "transcripts/trimmed.json"
  bounds:
    prompt_id: "dnd_session.bounds"
    profile_id: ""
    transcript_input_name: "transcript"
    output_path: "artifacts/session_bounds.json"
    timeout: "10m"
    render_debug: false
    render_output_path: "artifacts/session_bounds.render.json"
  seriatim:
    report: false

Trim behavior summary:

trim discovers and validates transcripts/normalized.json
trim uses Scriptorium bounds (dnd_session.bounds by example config) to produce artifacts/session_bounds.json
bounds IDs are validated against the same normalized transcript ID space that Seriatim trim will consume
trim converts bounds to Seriatim keep selector (for example 10-868) and runs Seriatim trim
if trim is disabled, Narratio copies normalized transcript to trimmed transcript and records trim_action=copy_disabled

Trim outputs and diagnostics:

artifacts/session_bounds.json
transcripts/trimmed.json
logs/scriptorium.bounds.stdout.log
logs/scriptorium.bounds.stderr.log
config/scriptorium.bounds.generated.yml
logs/seriatim.trim.stdout.log
logs/seriatim.trim.stderr.log
config/seriatim.trim.generated.yml
optional bounds render-debug outputs:
- artifacts/session_bounds.render.json
- logs/scriptorium.bounds.render.stdout.log
- logs/scriptorium.bounds.render.stderr.log
- config/scriptorium.bounds.render.generated.yml

Render-debug files are diagnostics and are not treated as canonical stage output artifact refs.

Scriptorium Configuration

pipeline.scriptorium is optional. When present, Narratio validates and uses it for analyze-stage artifact generation.

Key points:

scriptorium.binary defaults to scriptorium when section is present
scriptorium.config_path is optional
scriptorium.timeout defaults to 10m when omitted
scriptorium.render_debug enables render diagnostics globally
artifacts are configured under scriptorium.artifacts (map shape supports multiple artifacts)
enabled artifacts require prompt_id and output_path
artifact render_debug may override global render setting
vars currently support boolean and string values

Example session_recap artifact definition:

scriptorium:
  binary: "scriptorium"
  config_path: "/etc/scriptorium/config.yml"
  timeout: "10m"
  render_debug: false

  artifacts:
    session_recap:
      enabled: true
      prompt_id: "dnd.session_recap"
      profile_id: "local-quality" # optional
      output_path: "artifacts/session_recap.md"
      timeout: "10m"
      # render_debug: true # optional per-artifact override

      inputs:
        transcript:
          source: "trimmed_transcript"
          required: true

        previous_recap:
          source: "previous_session_artifact"
          artifact: "session_recap"
          path: "" # optional; set when available
          required: false

      vars:
        session_id: true
        session_date: true
        campaign_name: true
        previous_session_id: true
        output_kind: "session_recap"

Prompt IDs and profile IDs are configuration values. They are not hardcoded in analyze-stage logic.

Do not put secrets in pipeline.yml. If API-key behavior is configured, use env var names only.

If pipeline.secrets.env_dir is configured, keep only references and secret files there; secret values are still not written to manifests, generated configs, or Narratio-managed logs.

Scriptorium Runtime Behavior

Narratio integrates with Scriptorium through the public CLI subprocess contract:

generation: scriptorium run
diagnostics/testing: scriptorium render --format json when render_debug is enabled

For the initial implementation, only session_recap generation is supported.

Analyze-stage session recap behavior:

available transcript input sources for configured artifacts: processed_transcript, normalized_transcript, trimmed_transcript
session recap should use gameplay-only transcript input (source: trimmed_transcript)
Narratio resolves trimmed_transcript from trim manifest output (transcript_trimmed) or fallback transcripts/trimmed.json
Narratio resolves normalized_transcript from normalize manifest output (transcript_normalized) or fallback transcripts/normalized.json
missing trimmed transcript fails clearly and advises running trim stage first
normalized_transcript is the preferred full-transcript source for future table/meta-analysis artifacts
processed_transcript remains supported for advanced/debug use cases
optionally includes previous_recap when configured and resolvable
omits optional previous recap when unavailable
fails if required inputs are missing
validates output file exists and is non-empty

Expected session output paths:

artifacts/session_recap.md
logs/scriptorium.session_recap.stdout.log
logs/scriptorium.session_recap.stderr.log
config/scriptorium.session_recap.generated.yml
artifacts/session_recap.render.json when render diagnostics are enabled

Examples

Starter files:

examples/pipeline.minimal.yml
examples/pipeline.audita-overrides.yml
examples/session.minimal.yml
examples/session.template.yml
examples/speakers.yml

Commands

Run tests:

go test ./...

Plan a run:

go run ./cmd/narratio plan --session examples/session.minimal.yml

Use --config <path> to override default pipeline lookup when needed.

Run with a discoverable session template:

go run ./cmd/narratio run --session-id 2026-04-04

Run full pipeline:

go run ./cmd/narratio run --config examples/pipeline.minimal.yml --session examples/session.minimal.yml

Run analyze only:

go run ./cmd/narratio run-stage --config examples/pipeline.minimal.yml --session examples/session.minimal.yml analyze

Resume with a template session ID:

go run ./cmd/narratio resume --config examples/pipeline.minimal.yml --session examples/session.template.yml --session-id 2026-04-04

Operational Note

Checksum-based stale detection is not implemented yet.

If prepared inputs or prompt/runtime config change, rerun the appropriate upstream stages before relying on downstream artifacts.

Examples:

glossary/autocorrect/speaker-context changes: rerun at least merge, polish, normalize, trim, and analyze
trim bounds prompt/profile/config changes: rerun at least normalize, trim, and analyze
session recap prompt/profile/input-source changes: rerun analyze

Roadmap

Near-term roadmap:

extend analyze to additional configured artifacts
support workflows where later artifacts consume earlier generated artifacts
keep orchestration explicit without a generic DAG engine
implement archive and notify backends

README.md Unescape Escape

narratio

Current Implementation