184 lines
10 KiB
Markdown
184 lines
10 KiB
Markdown
# Resumable Speaker Transcription and Replayable Uploads
|
|
|
|
Status: Implemented.
|
|
|
|
## Goal and User Intent
|
|
|
|
Preserve completed speaker transcriptions when another speaker fails, so an
|
|
ordinary run or resume does not repeat expensive successful work. Separately,
|
|
allow Go's HTTP transport to replay WhisperX uploads when it encounters a
|
|
recoverable connection or stream failure.
|
|
|
|
The motivating incident exhausted four transcription attempts for one speaker
|
|
after three other speakers had completed. The final error reported an HTTP/2
|
|
peer `PROTOCOL_ERROR` and an inability to replay the request body without
|
|
`Request.GetBody`.
|
|
|
|
These are related reliability improvements, not a promise to eliminate all
|
|
service failures. This document records the target state and scope. The
|
|
completed implementation sequence is retained in Git history; current behavior
|
|
is documented in the canonical guides linked below.
|
|
|
|
## Pre-implementation Limitations
|
|
|
|
The [transcribe stage](../internal/stage-transcribe.md) writes successful
|
|
responses into the invocation's run directory. It installs canonical transcripts
|
|
and returns its per-file metadata only after every speaker succeeds. A failure
|
|
therefore leaves no durable per-speaker progress that ordinary resume can use,
|
|
even though some completed JSON files may remain on disk.
|
|
|
|
The [WhisperX adapter](../integrations/whisperx.md) already retries requests at
|
|
the application level, reopening audio for each attempt. Its multipart request
|
|
body is streamed through a pipe and has no replay callback. Transport-level
|
|
recovery within an attempt cannot reconstruct a consumed body. Providing that
|
|
capability addresses the replay limitation, not the underlying peer protocol
|
|
error; persistent failures can still require service or proxy investigation.
|
|
|
|
## Targeted End State
|
|
|
|
### Durable Per-Speaker Completion
|
|
|
|
Each successfully transcribed speaker file is validated, saved, and recorded
|
|
durably before that work is treated as reusable. The session manifest remains
|
|
the progress authority. Follow the existing analyze-stage checkpoint pattern
|
|
without turning speakers into pipeline stages or introducing a general task
|
|
engine.
|
|
|
|
Ordinary execution reconciles the current prepared audio set with recorded
|
|
speaker results. It reuses valid completed work and schedules failed,
|
|
unattempted, missing, or obsolete results. A result is not reusable merely
|
|
because a matching filename exists. Evidence binds it to the speaker/audio
|
|
identity, audio content, Narratio-visible recognition settings, and the saved
|
|
transcript's integrity.
|
|
|
|
The relevant recognition settings remain the WhisperX service URL and language.
|
|
Retry count/delay, concurrency, timeout, credentials, and diagnostic locations
|
|
remain operational settings rather than reasons to discard completed results.
|
|
Changes to private service models or configuration remain an operator reason
|
|
to force transcription; Narratio does not inspect the service's private state.
|
|
|
|
Changes to individual audio inputs require replacement of affected results,
|
|
not automatic retranscription of unrelated unchanged speakers. New speakers
|
|
need transcription; removed speakers do not remain in the current aggregate
|
|
output set. Records must describe the inputs actually used, not grant reuse to
|
|
old bytes under a changed identity.
|
|
|
|
### Aggregate Completion, Failure, and Force
|
|
|
|
The overall stage succeeds only when every currently required speaker has a
|
|
validated, durably recorded result. Incomplete per-speaker progress does not
|
|
make the aggregate stage successful or authorize downstream consumption of an
|
|
incomplete collection. Existing dependent-stage invalidation remains in effect.
|
|
|
|
Retain bounded concurrency and the current fail-fast behavior: a speaker
|
|
failure cancels outstanding work. Results already checkpointed remain reusable
|
|
after failure, cancellation, or process interruption. A later ordinary run or
|
|
resume retries the remaining work; it does not need a new CLI command.
|
|
|
|
Checkpoint writes must be serialized so concurrently finishing speakers cannot
|
|
overwrite each other's progress. A failed persistence operation must not
|
|
advertise an unrecorded result as complete. Previously committed progress remains
|
|
the recovery authority, and checkpoint errors must be surfaced rather than
|
|
reported as successful transcription.
|
|
|
|
`--force` on transcribe regenerates all currently required speakers instead of
|
|
reusing their prior completions. Successful replacements are checkpointed too,
|
|
so an ordinary retry after a failed forced invocation can retain the newly
|
|
completed work. Old results for speakers not successfully regenerated must not
|
|
satisfy that unfinished forced replacement on the next ordinary retry.
|
|
|
|
This deliberately relaxes the current all-speakers-before-canonical-installation
|
|
implementation: individual durable results may exist while the stage remains
|
|
failed or incomplete. The all-speakers-before-stage-success boundary remains.
|
|
|
|
### Replayable WhisperX Requests
|
|
|
|
Provide `Request.GetBody` so the HTTP transport can obtain a fresh body for the
|
|
same request. Replay reopens the audio and reproduces the multipart boundary,
|
|
filename, fields, and framing used by the original request. Independent bodies
|
|
must not share a consumed reader or mutable read position.
|
|
|
|
Preserve bounded-memory streaming rather than buffering entire audio files in
|
|
memory. Correctly close every original and replay body and its audio resources;
|
|
preserve source-read errors and cancellation, including partial uploads and
|
|
early server responses. Successful response handling must not accidentally wait
|
|
on, or report errors solely from, an abandoned body instead of the replay that
|
|
produced the response.
|
|
|
|
Keep the existing application-level retry count, delay, classification, and
|
|
per-attempt timeout semantics. Transport replays occur within an application
|
|
attempt and its context; they are not additional Narratio-configured attempts.
|
|
Do not add another custom retry loop, silently increase retries, or disable
|
|
HTTP/2 globally as a substitute for replay support.
|
|
|
|
The adapter still validates and installs only successful JSON responses.
|
|
Request replay does not provide exactly-once server execution: a lost response
|
|
can cause repeated transcription work, as can existing application retries.
|
|
No new service-side idempotency protocol is required by this feature.
|
|
|
|
## Required Change Areas
|
|
|
|
- **Manifest and persistence:** Add transcribe-owned per-speaker progress and
|
|
evidence, with safe checkpoint updates and preservation through runner failure
|
|
handling. Review manifest validation, copying, serialization, and existing
|
|
publish/restore handling wherever the new records pass through those paths.
|
|
- **Stage execution and reuse:** Reconcile speaker records before scheduling,
|
|
checkpoint validated completions, rebuild aggregate outputs from the current
|
|
set, and account for partial progress in ordinary and forced execution.
|
|
Review successful-stage reuse as well as failed-stage retries so stale audio
|
|
or missing/corrupt transcript evidence is not bypassed by an aggregate skip.
|
|
- **Artifact lifecycle:** Use existing path, installation, integrity, and manifest
|
|
mechanisms. Ensure checkpointed results do not depend on an ephemeral run
|
|
directory surviving cleanup. Do not introduce a second authoritative ledger
|
|
or infer completion by scanning abandoned runs.
|
|
- **HTTP adapter:** Make multipart bodies reproducible and correctly manage
|
|
every replay's lifetime, without changing the WhisperX HTTP schema or requiring
|
|
an upstream service change.
|
|
- **Tests:** Cover per-speaker recovery after a later failure, interruption and
|
|
checkpoint failure; changed/missing inputs or outputs; force followed by an
|
|
ordinary retry; concurrent successful completions; and aggregate downstream
|
|
gating. Extend existing fixtures rather than duplicating analyze's complete
|
|
test suite. Exercise actual body reconstruction after partial consumption,
|
|
multipart equivalence, cancellation/resource cleanup, and existing bounded
|
|
streaming and retry behavior—not just the presence of a callback. Default
|
|
tests remain offline and use local/fake HTTP boundaries, never paid services.
|
|
- **Documentation:** Update the canonical
|
|
[stage guide](../internal/stage-transcribe.md),
|
|
[manifest guide](../internal/manifest.md),
|
|
[WhisperX contract](../integrations/whisperx.md),
|
|
[operations guide](../operations.md), and
|
|
[troubleshooting guide](../troubleshooting.md) when implemented. Explain
|
|
partial completion, ordinary retry versus force, and the distinction between
|
|
request replay limitations and persistent peer protocol failures.
|
|
|
|
## Compatibility and Scope Boundaries
|
|
|
|
Existing speaker identities, raw transcript formats, and the fixed pipeline
|
|
order remain unchanged. Existing manifests remain readable, but must not be
|
|
treated as containing per-speaker evidence they never recorded. Recovery relies
|
|
on verified recorded evidence, not guessed associations between old transcripts
|
|
and current audio. No general legacy-migration framework is part of the feature.
|
|
|
|
Automatic adoption of unrecorded outputs from previously failed runs is outside
|
|
this feature. Such files may be recoverable after verifying their inputs, but
|
|
their presence alone does not prove completion. No recovery/import command,
|
|
speaker-selection CLI, historical invocation ledger, new service configuration
|
|
discovery, or broader orchestration refactor is included.
|
|
|
|
The feature does not diagnose the remote service/proxy's underlying HTTP/2
|
|
problem, guarantee recovery from persistent protocol errors, or change the
|
|
policy for continuing unrelated work after failure.
|
|
|
|
## Completion Criteria
|
|
|
|
For speakers A, B, C, and D: if A and B are checkpointed, C fails, and D has not
|
|
completed, ordinary resume retains A and B, retries C, and completes D. No
|
|
downstream stage receives an incomplete successful transcription collection.
|
|
Force replaces all four, while an ordinary retry can reuse checkpoints from
|
|
that forced replacement attempt.
|
|
|
|
The HTTP transport can reconstruct the upload after consuming the original
|
|
body, without whole-file memory buffering, lost source errors, or stranded
|
|
upload resources. A persistent peer error still fails normally after the
|
|
configured application attempts, without discarding checkpointed speakers.
|