narratio/docs/internal/overview.md

Internal Overview

This document is the implemented component map for Narratio. Normative system boundaries and dependency direction belong in Architecture. User and operator contracts belong in the CLI, Configuration, Operations, and Troubleshooting. Externally observable tool and format contracts belong under Integrations.

Execution Path

cmd/narratio -> internal/app -> configuration and production composition
             -> internal/stage -> adapters and external systems
             -> manifests and artifact resolution -> durable local/remote output

The executable delegates process behavior to the application boundary. The application resolves configuration, composes concrete collaborators, acquires session safety controls, and runs commands. Pipeline commands execute the canonical stage sequence through adapter interfaces, while manifests record progress and artifact services resolve durable inputs and outputs.

The pipeline runner follows one explicit invocation lifecycle: inspect bounded prerequisites without mutation, prepare the workspace layout, acquire one session lock, load or create the authoritative session manifest, compose only the collaborators needed by the selected range, execute each selected stage in order, and then return the summary. Cleanup, when selected, is the ordinary final stage rather than runner finalization work. The lock remains held from manifest initialization through finalization, and a failed stage or required manifest save prevents later stages from starting.

Components

Area	Implemented owners	Responsibility
Executable	cmd/narratio	Process entry, standard stream wiring, argument handoff, and exit status.
Application orchestration	internal/app	Command dispatch, configuration selection, secret-file environment loading, production composition, session locking, planning, execution, restore, and user-facing reporting.
Configuration	internal/config	ConfigKit-backed pipeline imports and profiles, canonical party and family expansion, strict YAML loading, defaults, normalization, session templating, and validation.
Pipeline stages	internal/stage	Canonical stage registry, shared stage contract, execution dependencies, and implemented stage behavior.
External boundaries	internal/adapters, internal/audio	WhisperX HTTP, downstream subprocesses, notification, object storage, and S3 audio materialization behind Narratio contracts.
Manifests	internal/manifest	Durable session progress, stage transitions, validation, and atomic persistence.
Artifacts and paths	internal/artifacts, internal/pathsafe	Artifact identities and resolution, local and remote path/key models, current-state discovery, and confined relative destinations.
Previous-session cache	internal/previouscache	Deterministic planning and materialization requirements for configured previous-session inputs.
Artifact policy	internal/artifactpolicy	Source and destination policy, configured artifact identity validation, and publish destination safety.
Shared models and file operations	internal/artifactmodel, internal/contracts, internal/fileops	Transcript and artifact data contracts plus atomic single-file replacement, bounded reads, workspace modes, and root-scoped cleanup helpers.
Logging	internal/logging	Application logger construction and shared structured logging behavior.

The application boundary composes concrete implementations. Stages depend on Narratio-level contracts; external transport and SDK details remain in adapters. The normative rules for these relationships remain in Architecture.

Command execution normalizes one private dependency value per invocation. Configuration reads, secret-directory reads, remote downloads, object-store construction, and restore phase functions use that value throughout the command. Public command entry points supply production defaults, while focused tests replace only the external or sequencing boundary they exercise without mutating package state.

Pipeline execution and session plan share the same inclusive contiguous-range model and the same pure pre-execution decision over recorded status, force, and normalized resume validation. Planning applies that decision and modeled selected-stage transitions to cloned session state; it does not persist, create invocation state, initialize stage-execution adapters, or claim to observe an execution result. Command configuration loading can still retrieve a missing session file through configured remote storage. It retains the initially composed pipeline and selected campaign while resolving either a local or downloaded remote session, so one invocation cannot mix pipeline revisions. Analyze planning additionally exposes the artifact closure's targets, prerequisite rebuilds, execution order, and current reuse.

Pipeline Stage Set

The implemented canonical order is:

1. prepare
2. transcribe
3. merge
4. polish
5. normalize
6. trim
7. render
8. extract
9. analyze
10. publish
11. notify (no-op)
12. cleanup

notify currently has no persisted pipeline outputs and uses the explicit noop notification mode. Its versioned semantic evidence records that delivery mode and excludes adapter credentials and response data. The focused stage documents own implementation mechanics. The CLI and Operations own user-visible invocation and execution semantics.

Execution order and invalidation are separate application contracts. The stage registry owns the flat execution sequence. The application orchestration owner uses a fixed, validated dependency relation to find transitive dependents in canonical order. In particular, render and extract are sibling consumers of trimmed transcript state: neither invalidates the other, while either can stale analyze, publish, notify, and cleanup. Cleanup is a dependent of notify, so any earlier replacement can stale a prior cleanup result.

Focused Documentation

• Configuration Internals: pipeline imports and profiles, field ownership, decoding, and root-relative path semantics.
• Adapter Internals: external adapter boundaries, composition, failure behavior, and test surfaces.
• Artifact Internals: source identities, runtime catalog, resolution, previous requirements, and current-state helpers.
• Manifest Internals: session records, persistence, and execution transitions.
• Storage Internals: object-store interface and S3 behavior.
• Workspace Internals: local layout, locking, and cleanup guardrails.
• Restore Internals: discovery, planning, execution, and reporting.
• prepare
• transcribe
• merge
• polish
• normalize
• trim
• render
• extract
• analyze
• publish
• cleanup

Use this map to find an owner, then read its focused documentation and tests before changing behavior.

The stage registry is implemented in internal/stage/placeholders.go and its ordering is protected by internal/app/planner_test.go. Cross-invocation skip, force, failure, and invalidation behavior is exercised in internal/app/runner_test.go and internal/app/run_stage_test.go.