Reconcile ordered pipeline documentation

This commit is contained in:
2026-07-22 02:40:36 +00:00
parent f15fd4f9c1
commit 906d97b391
7 changed files with 85 additions and 73 deletions

View File

@@ -191,6 +191,13 @@ the runner returns.
## Execution Flow
The pipeline-wide coordinator owns the ordered step loop, generated-reference
sets at each barrier, and deterministic merging of step outcomes. For one step,
the lane engine initializes checkpoint state in lane order, dispatches bounded
extract work, advances terminal lanes through serial merge and normalize work,
selects failures by stable pipeline scope, and merges lane-local outcomes back
in resolved order. Completion timing never becomes public ordering.
The runner:
1. validates its prepared input;
@@ -306,8 +313,8 @@ implementations when collaborators are absent. Each checkpointed workflow
boundary records a running, succeeded, or failed transition. Reuse decisions
are consulted in workflow order and accepted payloads are cloned before
entering the normal handoff path. Typed extract, merge, and normalize
checkpoints store codec bytes with artifact kind, schema ID and version, exact
schema digest, and media type. Reuse compares that identity with the prepared
checkpoints store codec bytes with artifact kind, schema ID, name, version and
exact digest, and media type. Reuse compares that identity with the prepared
codec and decodes through the codec; missing identity, mismatches, corrupt
bytes, and decode failures become explicit reuse misses and execute the lane
normally. Dependency fingerprints and debug content digests use the same stable
@@ -339,7 +346,8 @@ hydration and execute normally. A missing, rejected, corrupt, incompatible, or
changed producer blocks its dependent while leaving independent work eligible
for reuse. The runner records bounded decision
categories: `reused`, `executed`, `forced_recompute`, and
`dependency_invalidated`.
`dependency_invalidated`. Operator meanings for the stable reason codes belong
to [Operations](../operations.md#resume-and-selective-recompute).
The CLI includes prepared-component fingerprints in the run-wide checkpoint
identity alongside resolved configuration, raw input, reference provenance,
@@ -350,9 +358,9 @@ fingerprint intentionally produces a cold cache miss. Existing checkpoint
schemas and paths remain unchanged.
The CLI's `--recompute-step` policy forces the selected step and all transitive
dependents, but requires reusable checkpoints for all selected producers that
precede it. It changes loader decisions only; it does not alter persistent
checkpoint identity.
dependents, but requires accepted normalized artifacts for every unselected
producer on which that closure depends. It changes execution policy only; it
does not alter persistent checkpoint identity.
Debug instrumentation wraps run, stage, attempt, validator, and structured LLM
boundaries. Every executed chunk, extract, merge, and normalize attempt writes
@@ -418,7 +426,9 @@ stage, resolved lane, and source chunk rather than completion time.
- `internal/framework/pipeline/references_test.go`: target resolution and
materialization.
- `internal/cli/run_contract_test.go`: production run transitions, retries,
rejections, warnings, debug hooks, and manifests.
rejections, warnings, CLI recomputation controls, debug hooks, and manifests.
- `internal/cli/recompute_execution_contract_test.go`: filesystem-backed
selective recomputation and accepted-producer recovery.
- `internal/cli/production_contract_test.go`: production composition and
configuration-resolution smoke coverage.
- `internal/cli/example_contract_test.go`: maintained example resolution and

View File

@@ -35,13 +35,13 @@ they do not describe a current public state surface.
Ordered-step lane checkpoints include the step identity in their storage scope.
When a later lane consumes a generated artifact, its dependency fingerprints
include the producer's artifact kind, complete schema identity, media type,
canonical content digest, and size. A producer checkpoint may be decoded through
the registered codec and handed off without rerunning it. Missing, rejected,
corrupt, incompatible, or changed producer state produces a bounded
`dependency_invalidated` decision for every transitive dependent lane; it does
not permit stale downstream reuse. The CLI's selective recomputation policy
records `forced_recompute` for the selected step and its dependents while
requiring compatible predecessor checkpoints.
canonical content digest, and size. Ordinary resume compares those fingerprints
when progressively loading consumer stage checkpoints, so changed producer
content produces `dependency_invalidated` rather than stale downstream reuse.
Selective recomputation instead requires each unselected producer's accepted
normalized artifact; invalid accepted state records its specific bounded reason
and stops before the dependent. The selected step and its transitive dependents
record `forced_recompute`.
Ordinary resume loads extract, merge, and normalize checkpoints progressively
and may execute later lane stages after an earlier cache miss. Selective
@@ -63,7 +63,8 @@ detail passes through one UTF-8-safe bounded sanitizer and contains only
allowlisted diagnostic context, never payloads, references, credentials,
environment values, or physical paths. Typed categories and codes remain intact
through pipeline events and become strings only in manifest and debug-summary
JSON.
JSON. [Operations](../operations.md#resume-and-selective-recompute) is the
canonical operator-facing reason-code reference.
`internal/core/fileio` provides confined atomic file writes used by state
collaborators. The chunk-plan store retains its stronger entry validation.
@@ -98,6 +99,8 @@ separately and never replace the command's primary error.
terminalization, and output/report boundaries.
- `internal/cli/state_hardening_test.go`: independent roots, reuse, failures,
permissions, cleanup, and redaction.
- `internal/cli/recompute_execution_contract_test.go`: selective recomputation,
filesystem recovery, deterministic decisions, and failed predecessor state.
- `internal/cli/production_contract_test.go`: production composition and
configuration validation at the CLI boundary.
- `internal/cli/example_contract_test.go`: maintained example ownership.