narratio/docs/internal/stage-prepare.md

Stage: prepare

Purpose

Materialize canonical current-session input state and provenance before downstream stages run.

Prepare owns:

• local input file materialization (inputs/**);
• audio input materialization (audio/**);
• previous-session cache hydration (previous/**) for canonical previous-session artifact sources.

Inputs and outputs

Inputs:

• resolved config/campaign/session (pipeline.yml, campaign.yml, session.yml);
• remote session provenance when session.yml was loaded from S3;
• campaign or session input files (speakers, autocorrect, glossary);
• audio source:
  • local: session.inputs.audio_dir or session.inputs.audio_files;
  • S3: session.inputs.audio_s3.prefix;
• configured enabled Scriptorium artifact inputs (for previous-session requirement scanning);
• remote previous-session current archive state when previous hydration is required.

Outputs:

• inputs/campaign.yml;
• inputs/session.yml;
• inputs/pipeline.resolved.yml;
• inputs/speakers.yml;
• inputs/autocorrect.yml;
• inputs/glossary.yml;
• audio/*.flac in canonical session audio/;
• optional previous/manifest.json;
• optional previous/artifacts/**;
• deterministic manifest.Inputs records with checksums and provenance metadata.

Boundaries

Owns:

• input path resolution and materialization;
• S3 audio list/download/copy flow;
• previous-session artifact requirement collection from enabled configured artifacts;
• previous cache lifecycle when requirements exist (clear and rehydrate managed previous/ state).

Does not own:

• transcript or artifact generation;
• analyze-stage source resolution;
• archive commit behavior.

Config fields used

• session.session_id
• session.previous_session_id
• session.campaign
• session.inputs.speakers_file
• session.inputs.autocorrect_file
• session.inputs.glossary_file
• session.inputs.audio_dir
• session.inputs.audio_files
• session.inputs.audio_s3.prefix
• pipeline.workspace.root
• pipeline.spool.root
• pipeline.cache.root
• pipeline.cache.s3_audio
• pipeline.storage.s3.bucket
• pipeline.storage.s3.root_prefix
• pipeline.scriptorium.artifacts.<name>.enabled
• pipeline.scriptorium.artifacts.<name>.inputs.<key>.source
• pipeline.scriptorium.artifacts.<name>.inputs.<key>.required
• campaign.campaign_id
• campaign.inputs.speakers_file
• campaign.inputs.autocorrect_file
• campaign.inputs.glossary_file

External adapters used

• storage.ObjectStore for:
  • S3 audio listing/downloads;
  • previous-session current pointer/manifest/artifact object checks and downloads.

State and manifest behavior

• Ensures workspace layout exists.
• Materializes canonical input files and audio files.
• For S3 audio, uses run-scoped spool for active downloads and durable cache for reusable audio files; cache hits copy directly to work audio without downloading the object again.
• Records inputs/session.yml provenance as local session_config or remote session_config.s3.
• Resolves campaign-provided stable input paths relative to campaign.yml.
• Resolves session-provided stable input overrides relative to session.yml.
• Scans enabled configured artifact inputs for canonical sources:
  • narratio.previous_session.artifact.<artifact_key>
• If one or more canonical previous-session requirements exist:
  • clears managed previous/ state;
  • hydrates required/optional previous artifacts from the configured previous session’s committed archive current state;
  • writes previous/manifest.json and hydrated previous/artifacts/**;
  • stores archive-relative artifact paths such as artifacts/session_recap.md as previous/artifacts/session_recap.md, not previous/artifacts/artifacts/session_recap.md;
  • records hydrated previous inputs in manifest.Inputs with source previous_session_archive.current.
• If no canonical previous-session requirements exist, prepare does not manage previous/.
• manifest.Inputs is sorted deterministically by (kind, path).
• S3 audio manifest.Inputs retain S3 provenance and include cache_path; spool_path is present only when the current prepare invocation downloaded the file.

Required and optional previous-session behavior

• previous_session_id unset:
  • if any referenced previous artifact is required: fail;
  • if all referenced previous artifacts are optional: continue and omit them.
• Previous session archive current pointer or manifest missing:
  • if any referenced previous artifact is required: fail;
  • if all referenced previous artifacts are optional: continue and omit missing ones.
• Missing required previous artifact object: fail.
• Missing optional previous artifact object: omit.
• Downloaded previous artifacts must validate as non-empty files.

Skip and resume behavior

• Runner-level skip remains authoritative:
  • if prepare already succeeded and run is not forced, prepare does not run and no hydration/download occurs.
• If prepare runs (including with --force), it owns managed previous/ state for canonical previous-session inputs.

Failure behavior

• Fails on missing required input files, invalid audio-source combinations, empty/duplicate audio inputs, missing object store for S3 modes, and remote access/download/validation errors.
• For required canonical previous-session inputs, analyze-time missing-input guidance is to rerun:
  • narratio run-stage --force prepare

Tests to inspect before changing

• internal/stage/prepare_test.go
• internal/stage/prepare_previous_test.go
• internal/artifacts/previous_requirements_test.go
• internal/app/runner_test.go

Architectural invariants

• audio_dir/audio_files and audio_s3 are mutually exclusive.
• Storage keys are computed by callers using archive/path helpers; storage adapter receives explicit keys.
• prepare is the only stage that hydrates canonical previous-session cache state.