narratio/docs/roadmap/implementation.md

# Implementation Plan: Speaker Checkpoints and Replayable Uploads

Status: Implementation in progress; Stages 1-4 complete.

## Scope and Instructions

Implement [the feature roadmap](transcribe-checkpoints-and-replay.md). This plan
replaces the completed Notarius-profile plan; that work remains in Git history.
Complete the numbered stages in order, one bounded implementation prompt per
stage for gpt-5.6-terra. Record completion and actual verification results here.
Intermediate stages are not a finished feature and must not be released alone.

Before working, read `AGENTS.md`, `docs/development.md`, all three policies in
`docs/policy/`, the roadmap, and the stage-specific preparation below. Use the
maintained Go 1.26 toolchain. Preserve operator choices, speaker filenames,
bounded concurrency, streaming, and existing credentials/settings. Do not add
new commands, dependencies, service requirements, arbitrary speaker limits,
strict transcript schemas, or a generic checkpoint framework. No release or
upstream service work is part of this plan.

## Settled Design Decisions

### Progress Representation and Recovery

Add optional `transcribe_state_version` (version 1) and `transcribe_speakers`
to `manifest.StageRecord`. The latter is a map keyed by the existing derived
speaker ID, containing completed results only. Absence means that work must be
done; a separate pending/running/failed state machine is unnecessary. Aggregate
stage status/error remains authoritative for invocation failure. A versioned
empty collection is meaningful: no earlier completions are reusable.

Use a typed `TranscribeSpeakerRecord` with fingerprint version/digest,
session-relative audio path, audio SHA-256 and size, one `ArtifactRecord` output
of kind `transcript_raw`, output size, and producer run ID. The output carries
its session-relative canonical path and SHA-256. No new transcript source IDs,
synthetic schema contracts, timestamps, or historical attempt collections are
needed. Existing aggregate diagnostics can retain attempt/status/duration data
for work performed in the current invocation; do not fabricate that information
for reused outputs.

Keep the session-manifest schema identifier unchanged for this optional additive
state; validate supported per-stage versions and retain strict decoding. New
Narratio reads old manifests. Old binaries may reject the added fields under
their existing strict decoding; document this downgrade limitation.

For pre-feature manifests with no speaker evidence, regenerate transcription
once when transcribe is next selected, even if the aggregate status succeeded.
Old records do not bind individual transcript results to the audio bytes they
used. Do not guess that association from current files, old `per_file` metadata,
or abandoned run directories. This is the conservative upgrade path, not a new
general migration system. No other stage or unselected range is forcibly run.

### Reuse and Persistence

Compute a versioned per-speaker fingerprint from speaker ID, session-relative
audio identity, actual audio SHA-256/size, and the existing transcribe semantic
configuration fingerprint (URL and language). Do not include invocation ID,
absolute workspace root, concurrency, retry settings, timeout, or credentials.
Stream input hashing with cancellation; do not trust stored input checksums
alone. Recheck input bytes before accepting a new result to catch changes
observed during transcription; fail that job without checkpointing mismatched
evidence. This is not a hostile-filesystem snapshot or concurrent-editor
transaction guarantee and requires no extra audio copies or file locks.

Validate reusable output identity, canonical location, JSON, checksum, and size.
Use existing confinement/file primitives. Missing or corrupt regular outputs
become work; unsafe paths/non-regular conflicts follow existing safety errors,
not recursive deletion. Missing required audio is an input error, not permission
to reuse an unverifiable transcript. Removed inputs are removed from the current
collection and aggregate outputs without deleting arbitrary files.

One stage-owned mutex serializes candidate collection construction, manifest
save, and installation of committed state. Save a detached candidate and only
then update live authority, following analyze's pattern. Workers must not mutate
the live manifest outside that serialized section. Keep expensive HTTP work
outside the mutex. Stop dispatch/acceptance on checkpoint failure, join workers
before return, and preserve the last successfully committed collection.

Before launching requests, durably checkpoint the reconciled collection. For
force, that collection is empty: this prevents later ordinary retries from
resurrecting pre-force outputs. If this initial save fails, do not start requests
or overwrite outputs. Force again clears the current collection again; ordinary
retry retains replacements already checkpointed after the previous force.

Save each validated result at its canonical raw-transcript location before
checkpointing it. Run-local files remain staging/diagnostics, not reuse
dependencies. Bytes left by an unsuccessful checkpoint have no completion
authority. A worker may checkpoint a completed result using the parent stage
context even if a sibling canceled the work context; parent cancellation or a
checkpoint failure may prevent that save. Guarantee retention of committed
work, not all responses that happened to finish during cancellation.

### Multipart Replay Mechanism

Replace the pipe/producer upload mechanism with a small pull-based multipart
body backed by the audio reader. Use `mime/multipart.Writer` to encode a bounded
metadata prefix and suffix around the file payload: capture the prefix after
`CreateFormFile`, then encode the language field and closing boundary as the
suffix. Generate this framing once per application attempt. Each body combines
fresh readers for prefix, freshly opened audio, and suffix using `io.MultiReader`.
No full-audio buffer or temporary multipart spool is needed.

The initial body and `Request.GetBody` use the same factory/framing. Each owns
an independent audio handle. Give bodies idempotent Close and context-triggered
closure (for example `context.AfterFunc`), without holding a mutex across a
blocking Read. Read/Close coordination must release a blocked source and retain
genuine source-read/close errors while not treating deliberate early closure as
source failure. The attempt owns cleanup of all created bodies, including
replays; reject new body creation after its context expires.

Remove the old producer `Wait` dependency. Track body-local source errors with
synchronization so a swallowed upload read error cannot become a successful
transcript, and a deliberately closed original body cannot poison a successful
replay. Keep the outer retry policy and per-attempt context. Do not add replay
counters to the user retry budget or depend on Go's exact internal retry count.
Preserve ordinary HTTP redirect policy; body replay can also enable Go's
body-preserving redirects. Do not introduce a new redirect restriction.

## Stage 1: Add Durable Speaker Completion State

Status: Complete.

Completion marker (2026-09-21): Added optional versioned transcribe-owned
per-speaker completion records, strict load/save validation, isolated collection
cloning, legacy aggregate-only compatibility, and lifecycle-preservation
coverage. Verified with `go test ./internal/manifest` and `git diff --check`.

Preparation: `internal/manifest/manifest.go`, `store.go`, `analyze_state.go`,
their tests, and `docs/internal/manifest.md`.

1. Implement the additive fields, record type, version constants, validation,
   and deep-copy helper described above in a focused transcribe state file.
2. Validate ownership by the transcribe stage, supported versions, required
   completed-record evidence, output kind, safe relative paths, SHA-256 values,
   sizes, and producer identity consistency. Use existing speaker IDs as-is;
   do not apply analysis-artifact key regexes or copy unrelated analysis limits.
3. Integrate validation with both load and save. Preserve speaker state through
   running/failed/stale transitions and aggregate result clearing. Generic stage
   invalidation does not discard checkpoints; reconciliation checks their data.
4. Test round-trip populated and versioned-empty collections, old manifests
   without fields, malformed evidence, deep-copy isolation, and lifecycle
   preservation. Avoid imposing JSON formatting or map order as contracts.

Verify: `go test ./internal/manifest`.

Done when durable storage can represent partial completion independently of
aggregate status, without activating new stage behavior yet.

## Stage 2: Implement Read-Only Speaker Reconciliation

Status: Complete.

Completion marker (2026-09-21): Extracted shared deterministic transcribe job
construction and added read-only, cancellation-aware per-speaker input and
output reconciliation. Covered unchanged reuse, semantic and operational
settings, changed/added/removed inputs, legacy and forced work, missing/corrupt
outputs, unsafe evidence, and missing audio without connecting runtime skip or
checkpoint behavior. Verified with
`go test ./internal/stage ./internal/manifest` and `git diff --check`.

Preparation: `internal/stage/transcribe.go`, `semantic_contracts_initial.go`,
`analyze_reconciliation.go`, `extract_resume.go`, their tests, and the artifact
path/integrity helpers they use.

1. Extract the existing audio discovery/job construction into a shared,
   transcribe-local reconciliation path. Preserve current input selection and
   speaker identity behavior. Build sorted jobs and classify completed results
   against the current input set and evidence defined above.
2. Add pure fingerprint construction and streamed input/output inspection.
   Resolve manifest input paths with existing session-relative/absolute reading
   helpers; do not change input precedence or assume the current working directory
   is the session root. Newly written checkpoint paths are always relative.
3. Return current records and work-to-do without writing files or manifest state.
   Force selects all work and an empty starting collection; old unversioned
   transcribe state supplies no reusable records. Do not connect aggregate skip
   validation until execution can produce the new evidence in Stage 3.
4. Test unchanged reuse; one changed audio among unchanged speakers; URL/language
   changes; operational-setting exclusions; added/removed speakers; missing and
   corrupt regular outputs; unsafe record paths; missing audio; and force.
   Prefer compact tables and real temporary files over mocked hashing.

Verify: `go test ./internal/stage ./internal/manifest`.

Done when one read-only decision path describes exactly which speakers need
work, without altering existing runtime behavior prematurely.

## Stage 3: Execute and Checkpoint Speakers Incrementally

Status: Complete.

Completion marker (2026-09-21): Connected reconciliation to bounded execution,
durably checkpointed the starting and per-speaker collections through detached
manifest candidates, preserved committed work across failures and cancellation,
rebuilt deterministic aggregate outputs from reusable evidence, and added
speaker-aware resume validation. Updated production-style fixtures and covered
serialized concurrent saves, force/retry replacement, input rechecks, save
failures, partial recovery, and aggregate reconstruction. Verified with
`go test ./internal/stage ./internal/app`, `go test -race ./internal/stage`, and
`git diff --check`.

Preparation: `internal/stage/transcribe.go`, `analyze_execution_state.go`,
`run_local.go`,
`internal/app/runner.go`, and transcribe/checkpoint test fixtures.

1. Wire reconciliation into Run, require the existing ManifestStore collaborator,
   and checkpoint the starting collection before launching HTTP work. Implement
   detached candidate saves and serialized live-state updates. Audit Store.Save's
   normalization side effects so a failed candidate save cannot mutate shared
   authority; use detached records/collections wherever it may write.
2. Retain bounded errgroup execution and fail-fast cancellation. Check the work
   context before beginning each queued request. After success, validate adapter
   output path/JSON, recheck audio evidence, materialize canonical bytes, compute
   output evidence, and checkpoint the new record. Check a checkpoint-failure
   latch under the serialization lock before accepting further completions.
3. Preserve committed successes on all error/cancellation exits, including
   starting-save, canonical-write, per-speaker-save, and aggregate-save failures.
   Return meaningful speaker/error context and let the runner retain its existing
   terminal failure behavior. Do not report partial aggregate outputs as success.
4. Assemble final outputs in deterministic speaker order from the reconciled
   completed records, including reused speakers, with original producer IDs and
   checksums. If all records are current but aggregate status is failed/running,
   finish without WhisperX calls and rebuild the aggregate success.
5. Implement `transcribeStage.ValidateResume` using the same read-only
   reconciliation, so a succeeded stage cannot skip changed audio or invalid
   transcripts. Retain the existing URL/language semantic check and runner
   invalidation behavior; no whole-config digest invalidation.
6. Adapt existing tests that require no canonical partial files: the new contract
   permits checkpointed files but forbids aggregate success. Update fixtures to
   provide real manifest stores and records rather than a production fallback
   that silently disables checkpointing.
7. Test A/B completion followed by C failure and D unattempted (concurrency 1),
   reload and retry only C/D; concurrently finishing speakers without lost saves;
   failure and cancellation after a checkpoint; save failure retaining earlier
   authority; and force failure followed by ordinary retry without old-result reuse.

Verify: `go test ./internal/stage ./internal/app` and
`go test -race ./internal/stage`.

Done when incremental progress and aggregate success work together safely under
ordinary, concurrent, canceled, and forced execution.

## Stage 4: Verify Runner, Merge, and Durable Lifecycle Integration

Status: Complete.

Completion marker (2026-09-22): Added real-persistence application coverage for
cross-invocation retry, interrupted running state, failed aggregate saves,
force/reset persistence, downstream invalidation, and one-time legacy upgrade.
Versioned merge input discovery now requires aggregate success and an exact
match between current speaker evidence and declared aggregate outputs, while
legacy/manual directory fallback remains available. Restore coverage preserves
session-relative speaker paths, leaves unpublished raw transcripts absent, and
canonical transcripts remain reusable after run-directory cleanup. Verified
with `go test ./internal/app ./internal/stage ./internal/manifest`,
`go test -race ./internal/app`, and `git diff --check`.

Preparation: `internal/app/runner.go`, `runner_runtime.go`,
`analyze_checkpoint_recovery_test.go`, `runner_terminal_failure_test.go`,
`restore_manifest_paths.go`, `internal/stage/merge.go`, and existing publish,
restore, and cleanup tests routed from `docs/internal/overview.md`.

1. Add application-level tests with real manifest persistence and fake WhisperX:
   failed stage across separate invocations; all speakers committed before a
   failed aggregate save; interrupted running state; force/reset persistence;
   and unchanged upstream/succeeded downstream invalidation behavior. Inject
   errors at durable boundaries, not every private method call.
2. Close `discoverRawTranscripts`' directory-scan fallback for versioned
   transcribe state: incomplete aggregate state is not usable, and completed
   state consumes only declared current aggregate outputs. Keep legacy/manual
   fallback behavior for manifests without the new state rather than banning
   previously allowed standalone workflows. Test that partial files and removed
   speaker files cannot leak into a versioned merge, including direct stage calls.
3. Audit manifest copies/projections, remote manifest publication, restored path
   handling, and cleanup. Preserve relative speaker paths on restore and verify
   they resolve under the destination session, not the producer machine. Reuse
   existing confinement validation, not a second path-policy layer.
4. Add focused round-trip coverage through existing publish/restore fixtures for
   speaker records. Restored outputs absent from the published selection are
   non-reusable locally; do not silently expand remote publication to every raw
   speaker or upload audio as a new default. Removing run directories must not
   remove canonical checkpointed transcripts or make reuse require run files.
5. Verify old succeeded manifests are readable and rerun once when transcribe is
   selected, then reuse normally. Keep old manifests usable for other commands
   and bounded ranges. Do not automatically import old unrecorded JSON files.

Verify: `go test ./internal/app ./internal/stage ./internal/manifest` and
`go test -race ./internal/app`.

Done when the new authority survives runner and storage lifecycles and cannot
expose incomplete transcripts downstream.

## Stage 5: Implement Replayable Streaming Multipart Bodies

Status: Pending.

Preparation: `internal/adapters/whisperx/http.go`, `http_test.go`,
`docs/integrations/whisperx.md`, and the installed Go net/http Request/GetBody
contract. Keep this change inside the WhisperX adapter.

1. Implement the pull-based framing/body factory described above, using standard
   multipart encoding rather than hand-escaped MIME headers. Add no new payload
   cap or upload-size setting. Preserve file-before-language wire semantics and
   the existing form names.
2. Install the initial body and GetBody with the same per-attempt boundary.
   Track and close all opened bodies even if a custom transport returns early;
   make cleanup/context callbacks safe against overlapping reads and closes.
   A GetBody failure must be returned, not converted into an empty upload.
3. Replace pipe-specific Wait/error paths in doTranscribeAttempt with source-error
   handling appropriate to pull bodies. Genuine source errors remain visible;
   intentional closure during transport retry is not itself a source error.
   Keep response bounds, JSON validation, output installation, retry
   classification/delay, cancellation, and application attempt reporting intact.
4. Remove obsolete producer-only helpers rather than retaining two upload paths.
   Adapt tests tied to producer goroutines to assert streaming and resource
   release, which are the actual contracts.
5. Add focused body tests: original/replayed bytes parse identically; replay
   after partial consumption starts from the beginning; simultaneous bodies use
   independent handles; source failures propagate; Close/cancel releases a
   blocked reader without deadlock. Use synthetic audio and gated readers.

Verify: `go test ./internal/adapters/whisperx` and
`go test -race ./internal/adapters/whisperx`.

Done when replay is available without whole-file buffering or producer-lifetime
ambiguity, and all existing HTTP behavior tests still pass.

## Stage 6: Verify Replay Through the HTTP Adapter Boundary

Status: Pending.

Preparation: Existing injectable HTTPClient/RoundTripper seam and local-server
tests in `internal/adapters/whisperx/http_test.go`; app WhisperX wiring tests.

1. Add a deterministic transport-boundary regression: consume part of the
   original request, close it as a transport would, call GetBody, consume and
   parse the replay using the request Content-Type, then return valid JSON.
   Verify exact audio/language, installed output, and one application attempt.
   This must fail before replay support; merely checking non-nil GetBody is
   insufficient. Do not assert Go's internal retry count or require a flaky
   handcrafted HTTP/2 peer in the default suite.
2. Cover replay-open/read failure, early response, parent cancellation during
   replay, and disposal of all body handles. Reuse Stage 5 body coverage rather
   than repeat every low-level case. Prove an intentionally abandoned original
   body does not turn a valid replay response into failure.
3. Retain local HTTP tests for application retries, non-retryable responses,
   response limits, invalid JSON, and bounded streaming; exercise that an
   exhausted HTTP failure still leaves earlier speaker checkpoints reusable
   using the existing stage/app seams.
4. Record the verification boundary honestly: controlled transport replay proves
   Narratio's body contract, not recovery from every real proxy PROTOCOL_ERROR.

Verify: `go test ./internal/adapters/whisperx ./internal/app ./internal/stage`
and race tests for changed packages.

Done when the replay defect is protected at the adapter boundary and the
application retry/checkpoint behavior remains intact.

## Stage 7: Document and Validate the Completed Feature

Status: Pending.

1. Update canonical transcribe and manifest internals for the collection,
   reconciliation, durable-reset force behavior, serialized saves, and aggregate
   gate. Remove obsolete all-speakers-before-canonical-write descriptions.
2. Update the WhisperX contract for replayable bounded streaming, application
   attempts versus transport replay, resource ownership, and the lack of an
   exactly-once server guarantee. Do not claim the remote protocol issue is fixed.
3. Update operations/troubleshooting with ordinary resume versus force, durable
   partial success, old-manifest one-time retranscription on selecting transcribe,
   downgrade limitations, and private-model-change guidance. No new CLI flags
   or promises of automatic historical-file recovery. Keep detailed wire and
   storage contracts in their canonical owners.
4. Review changes against the feature roadmap and the implementation-discipline
   guidance. Do not elevate selected storage encoding, internal ordering, test
   synchronization, or exact error wording into new product guarantees.
5. Run the full repository gates below; document results and update completion
   statuses only after they pass. Leave release preparation for a separate task.

```sh
go test ./...
go test -race ./...
go vet ./...
go build ./...
go test ./internal/doccheck
go test ./internal/config -run '^TestExamplesLoadAndValidate$'
git diff --check
```

## Open Questions

None requiring user input. The agreed roadmap supplies the product decisions;
this plan selects a completed-only manifest collection, conservative old-record
handling, and pull-based replay as implementation choices. No stronger remote
reliability guarantee, new operator restriction, or generic orchestration
surface is proposed.