Document shared checkpoint persistence
This commit is contained in:
@@ -193,6 +193,25 @@ establish a current immutable snapshot.
|
|||||||
- writes through a sibling temporary file and atomic replacement visibility;
|
- writes through a sibling temporary file and atomic replacement visibility;
|
||||||
- updates `updated_at` on save.
|
- updates `updated_at` on save.
|
||||||
|
|
||||||
|
### Checkpoint persistence boundary
|
||||||
|
|
||||||
|
Analyze and transcribe share one private checkpoint operation in
|
||||||
|
`internal/stage`. Their typed callers still decide when to checkpoint, validate
|
||||||
|
their collections, populate stage-owned fields, and serialize access. The
|
||||||
|
shared operation copies the manifest and its checkpoint-relevant mutable
|
||||||
|
collections, applies the typed update to the copied target record, and calls
|
||||||
|
the existing `manifest.Store` exactly once. Only after that save succeeds does
|
||||||
|
it promote the saved target record and manifest `updated_at` into live state;
|
||||||
|
it preserves an existing target record's identity and leaves unrelated live
|
||||||
|
stage records in place.
|
||||||
|
|
||||||
|
The copy boundary includes stage records, input and artifact slices, aggregate
|
||||||
|
output/log/configuration slices, and the typed analyze and transcribe
|
||||||
|
collections that callbacks or `LocalStore.Save` may modify. Metadata and other
|
||||||
|
pointer fields remain shared read-only because neither checkpoint callback nor
|
||||||
|
the store mutates them. This is a bounded execution helper, not a general
|
||||||
|
manifest clone or a transaction across transcript/artifact files.
|
||||||
|
|
||||||
Manifest replacement provides atomic visibility, not a crash- or power-loss
|
Manifest replacement provides atomic visibility, not a crash- or power-loss
|
||||||
durability guarantee. Callers reload after a returned save error before retrying.
|
durability guarantee. Callers reload after a returned save error before retrying.
|
||||||
A transcribe or analyze checkpoint error leaves the prior manifest collection
|
A transcribe or analyze checkpoint error leaves the prior manifest collection
|
||||||
|
|||||||
@@ -115,7 +115,11 @@ Supported source families:
|
|||||||
- before the first adapter call, checkpoints the reconciled starting collection
|
- before the first adapter call, checkpoints the reconciled starting collection
|
||||||
when it removed configuration records or cleared obsolete authority. After
|
when it removed configuration records or cleared obsolete authority. After
|
||||||
each successful artifact, checkpoints the complete collection before making
|
each successful artifact, checkpoints the complete collection before making
|
||||||
that artifact available to the next scheduled dependent.
|
that artifact available to the next scheduled dependent. Analyze supplies
|
||||||
|
its typed update to the shared
|
||||||
|
[checkpoint persistence boundary](manifest.md#checkpoint-persistence-boundary)
|
||||||
|
and refreshes its working and committed collections only after the save
|
||||||
|
succeeds.
|
||||||
- keeps adapter output in the invocation's run-local analyze directory until
|
- keeps adapter output in the invocation's run-local analyze directory until
|
||||||
it is a safe, non-empty, bounded regular file with a calculated checksum and
|
it is a safe, non-empty, bounded regular file with a calculated checksum and
|
||||||
complete output contract. Canonical replacement uses the shared atomic file
|
complete output contract. Canonical replacement uses the shared atomic file
|
||||||
|
|||||||
@@ -35,9 +35,10 @@ Generate raw per-speaker transcripts from prepared audio using WhisperX.
|
|||||||
- output path returned by adapter must match requested output path.
|
- output path returned by adapter must match requested output path.
|
||||||
- an empty adapter result path means the requested path; adapters cannot select
|
- an empty adapter result path means the requested path; adapters cannot select
|
||||||
an alternate destination.
|
an alternate destination.
|
||||||
- candidate collection construction, manifest save, and installation into live
|
- candidate collection construction, the shared
|
||||||
state are serialized. Expensive hashing and HTTP work remain outside that
|
[checkpoint persistence boundary](manifest.md#checkpoint-persistence-boundary),
|
||||||
critical section.
|
and installation into live state are serialized by transcribe's mutex.
|
||||||
|
Expensive hashing and HTTP work remain outside that critical section.
|
||||||
- a canonical file has no completion authority until its manifest checkpoint
|
- a canonical file has no completion authority until its manifest checkpoint
|
||||||
succeeds. A save failure preserves the last successfully committed
|
succeeds. A save failure preserves the last successfully committed
|
||||||
collection for ordinary retry.
|
collection for ordinary retry.
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Implementation Plan: Shared Stage Checkpoint Mechanics
|
# Implementation Plan: Shared Stage Checkpoint Mechanics
|
||||||
|
|
||||||
Status: Implementation in progress; Stages 1-3 complete.
|
Status: Complete.
|
||||||
|
|
||||||
## Scope and Authority
|
## Scope and Authority
|
||||||
|
|
||||||
@@ -273,7 +273,24 @@ copy implementation, with their different models and synchronization intact.
|
|||||||
|
|
||||||
## Stage 4: Document, Review Scope, and Validate
|
## Stage 4: Document, Review Scope, and Validate
|
||||||
|
|
||||||
Status: Pending.
|
Status: Complete.
|
||||||
|
|
||||||
|
Completion marker (2026-09-22): Documented the private shared checkpoint
|
||||||
|
persistence boundary and linked the stage guides back to it while retaining
|
||||||
|
their separate analyze and transcribe progress policies. The refactor removed
|
||||||
|
the two local candidate/save/promotion sequences and the transcription-only
|
||||||
|
bounded clone; the retained thin wrappers own typed validation, updates,
|
||||||
|
working-state refresh, and their existing serialization and failure behavior.
|
||||||
|
The collective test review retained focused helper integrity coverage plus
|
||||||
|
stage/application recovery tests, without duplicating the helper's isolation
|
||||||
|
matrix. A cumulative scope audit found no schema, validation-policy,
|
||||||
|
scheduling, force, concurrency, output-selection, stage-order, adapter-call,
|
||||||
|
configuration, or public-interface change.
|
||||||
|
|
||||||
|
Verified with `go test ./...`, `go test -race ./...`, `go vet ./...`,
|
||||||
|
`go build ./...`, `go test ./internal/doccheck`,
|
||||||
|
`go test ./internal/config -run '^TestExamplesLoadAndValidate$'`, and
|
||||||
|
`git diff --check`; all passed.
|
||||||
|
|
||||||
Preparation: `docs/internal/manifest.md`, `docs/internal/stage-transcribe.md`,
|
Preparation: `docs/internal/manifest.md`, `docs/internal/stage-transcribe.md`,
|
||||||
`docs/internal/stage-analyze.md`, documentation/testing policies, and the
|
`docs/internal/stage-analyze.md`, documentation/testing policies, and the
|
||||||
|
|||||||
Reference in New Issue
Block a user