eric/notarius
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Harden checkpoint reuse and combat validation

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-22 14:24:03 +00:00
parent 23c55f8925
commit 748e02db80
15 changed files with 520 additions and 252 deletions
Download Patch File Download Diff File Expand all files Collapse all files

28
docs/internal/pipeline.md
View File

@@ -193,10 +193,12 @@ the runner returns.
The pipeline-wide coordinator owns the ordered step loop, generated-reference
sets at each barrier, and deterministic merging of step outcomes. For one step,
the lane engine initializes checkpoint state in lane order, dispatches bounded
extract work, advances terminal lanes through serial merge and normalize work,
selects failures by stable pipeline scope, and merges lane-local outcomes back
in resolved order. Completion timing never becomes public ordering.
one run-local lane engine owns worker lifecycle, cancellation, dispatch,
continuation queues, and result collection. It initializes checkpoint state in
lane order, dispatches bounded extract work, advances terminal lanes through
serial merge and normalize work, selects failures by stable pipeline scope, and
merges lane-local outcomes back in resolved order. Completion timing never
becomes public ordering.
The runner:
@@ -329,14 +331,16 @@ current non-empty checkpoint identity to match, so the invocation identity
still binds the input, resolved topology and configuration, references, runtime
overrides, profiles, and component fingerprints.
The runner decodes that accepted normalized artifact with the prepared codec,
re-encodes it, and requires exact kind, schema identity and digest, media type,
canonical bytes, content digest, and producer provenance. A valid result becomes
a runner-owned cloned normalized output, restores only normalize-checkpoint
warnings, and records one `accepted_artifact_reused` normalize decision. It does
not invoke or record extract, merge, normalize, or their validators. Invalid or
unavailable accepted state records its decision and fails the producer step;
the dependent step never starts and the producer is not implicitly rerun.
The runner decodes and canonically re-encodes each reusable artifact once with
the prepared codec, requiring exact kind, schema identity and digest, media
type, canonical bytes, content digest, and producer provenance. A valid accepted
producer becomes a runner-owned cloned normalized output, restores only
normalize-checkpoint warnings, and records one `accepted_artifact_reused`
normalize decision. It does not invoke or record extract, merge, normalize, or
their validators. Invalid or unavailable accepted state records its decision
and fails the producer step; the dependent step never starts and the producer
is not implicitly rerun. If a later required lane fails during initialization,
already hydrated terminal lanes remain in the failed output in resolved order.
Generated references add downstream dependencies containing the producer's
artifact kind, complete schema identity, media type, canonical content digest,

16
docs/internal/state.md
View File

@@ -57,14 +57,14 @@ records the decision, and stops without executing the producer or consumer.
The loader assigns a typed category and reason code at each validation site;
diagnostic prose is not classified after the fact. The runner then applies
forced-execution policy, validates reusable artifact bytes through the prepared
codec, and records the final decision before enforcing a required-predecessor
failure. That failure names only the step, lane, and stable reason code. Decision
detail passes through one UTF-8-safe bounded sanitizer and contains only
allowlisted diagnostic context, never payloads, references, credentials,
environment values, or physical paths. Typed categories and codes remain intact
through pipeline events and become strings only in manifest and debug-summary
JSON. [Operations](../operations.md#resume-and-selective-recompute) is the
canonical operator-facing reason-code reference.
codec once, returns the canonical hydrated value to the stage, and records the
final decision before enforcing a required-predecessor failure. That failure
names only the step, lane, and stable reason code. Decision detail is selected
from code-owned descriptions by reason code and then UTF-8 normalized and
bounded; callers cannot supply arbitrary diagnostic prose. Typed categories and
codes remain intact through pipeline events and become strings only in manifest
and debug-summary JSON. [Operations](../operations.md#resume-and-selective-recompute)
is the canonical operator-facing reason-code reference.
`internal/core/fileio` provides confined atomic file writes used by state
collaborators. The chunk-plan store retains its stronger entry validation.