Stage: transcribe

Purpose

Generate raw per-speaker transcripts from prepared audio using WhisperX.

Inputs

audio/*.flac from prepare

Outputs

transcripts/raw/<speaker>.json

Key Behavior

discovers prepared audio from manifest inputs or the canonical audio directory and derives each speaker identity from its .flac filename.
hashes the current audio and reconciles it with the versioned per-speaker completion collection before starting HTTP work.
reuses only records whose audio identity, observable WhisperX semantics, canonical output location, JSON, checksum, and size remain current.
durably checkpoints the reconciled collection before dispatch, then sends only missing or obsolete speakers through the bounded worker group.
validates each successful run-local response, rechecks the audio bytes, materializes the canonical raw transcript, and checkpoints that speaker before accepting another completion.
rebuilds aggregate stage outputs in stable speaker order from the complete current collection, including records produced by earlier invocations.

Invariants

prepared audio identities must be unique; prepare disambiguates distinct source paths that share a basename.
output path returned by adapter must match requested output path.
an empty adapter result path means the requested path; adapters cannot select an alternate destination.
candidate collection construction, manifest save, and installation into live state are serialized. Expensive hashing and HTTP work remain outside that critical section.
a canonical file has no completion authority until its manifest checkpoint succeeds. A save failure preserves the last successfully committed collection for ordinary retry.
each successful output is validated before its checkpoint, and cancellation, incomplete dispatch, or a failed aggregate save cannot be reported as aggregate stage success.
removed audio inputs are omitted from the current collection and aggregate output list; reconciliation does not delete arbitrary transcript files.
merge cannot consume a partial versioned collection: it requires transcribe aggregate success and an exact match between current speaker records and declared aggregate outputs.

Resume Evidence

Each completed speaker record binds the speaker ID, session-relative audio path, streamed audio checksum and size, the Narratio-visible WhisperX service URL and recognition language, and the canonical transcript path, checksum, size, and producer run. Changed audio bytes select that speaker for replacement; a changed URL or language selects every speaker. Either case stales fixed descendants through the normal runner decision. Retry count/delay, concurrency, timeout, credentials, and diagnostic locations are operational and do not change this evidence.

An ordinary retry reuses every valid speaker checkpoint even when aggregate transcribe status is failed, running, or stale. A succeeded stage is skipped only when its stored speaker set and aggregate output evidence also match the current collection. Otherwise, execution repairs those records without retranscribing reusable speakers and stales dependents through the normal runner path. --force first durably replaces the collection with an empty versioned collection, so a later ordinary retry cannot resurrect pre-force results; speakers checkpointed by that forced attempt remain reusable.

WhisperX models or private service configuration not exposed by Narratio's adapter contract cannot be fingerprinted; use --force after changing them. An older aggregate-only transcribe record has no trustworthy audio-to-output binding. When transcribe is next selected, every current speaker is therefore generated once and recorded in the versioned collection; later invocations can reuse it normally.

WhisperX owns HTTP, retry, timeout, and cancellation semantics.
Configuration owns concurrency and other operator-selected values.
Implementation and tests: internal/stage/transcribe.go, internal/stage/transcribe_reconciliation.go, internal/stage/transcribe_execution_state.go, internal/stage/transcribe_resume.go, internal/stage/transcribe_test.go, internal/stage/transcribe_reconciliation_test.go, and internal/stage/transcribe_checkpoint_test.go

4.3 KiB Raw Blame History