146 lines
8.1 KiB
Markdown
146 lines
8.1 KiB
Markdown
# Run State Internals
|
|
|
|
This document describes the implementation collaborators behind output, cache,
|
|
and debug state. User-visible fields belong in [Configuration](../config.md),
|
|
and physical layout, retention, recovery, reason codes, and cleanup belong in
|
|
[Operations](../operations.md).
|
|
|
|
## Composition
|
|
|
|
`internal/cli` is the only physical-path composition root. It resolves the
|
|
effective configuration, selects exact roots, allocates requested debug bundles,
|
|
constructs cache collaborators, writes logical output files, and reports paths.
|
|
Pipeline modules receive interfaces and request data, never output, cache, or
|
|
debug roots.
|
|
|
|
The CLI creates no chunk-plan store in bypass mode. It creates a checkpoint
|
|
recorder only when recording is enabled and a checkpoint loader only for a
|
|
resume invocation. It allocates debug state only after a safe run identity has
|
|
been generated and only when debug capture was requested. These choices keep
|
|
the three state families independently composable.
|
|
|
|
## Output And Cache
|
|
|
|
The pipeline runner returns logical output files. After validating every
|
|
logical name, the CLI exclusively creates the run directory beneath the
|
|
selected output root and performs confined, atomic file writes within it.
|
|
The runner supplies an accepted chunk map as an optional, defensively owned
|
|
output-request artifact. The JSON encoder alone decides whether its explicit
|
|
option writes the map and optional index descriptor; neither the map payload
|
|
nor its annotations are copied into the run manifest. The durable fields are
|
|
owned by the [Accepted Chunk Map contract](../integrations/chunk-map.md).
|
|
|
|
`internal/framework/chunkplan` owns source-addressed plan storage, validation,
|
|
and atomic publication. Its store is constructed only when the selected mode is
|
|
not `bypass`.
|
|
|
|
`internal/framework/checkpoint` owns checkpoint identity, manifests, payload
|
|
codecs, loader, and recorder. The CLI constructs a recorder whenever checkpoint
|
|
recording is enabled and constructs a loader only for a `--resume` invocation.
|
|
Identity incorporates explicit stable semantic fingerprints collected from
|
|
prepared modules and validators in addition to configuration, input,
|
|
references, runtime overrides, and LLM profiles.
|
|
The serialized
|
|
`workspace_schema_version` identifiers are frozen wire-compatibility fields;
|
|
they do not describe a current public state surface.
|
|
|
|
Ordered-step lane checkpoints include the step identity in their storage scope.
|
|
When a later lane consumes a generated artifact, its dependency fingerprints
|
|
include the producer's artifact kind, complete schema identity, media type,
|
|
canonical content digest, and size. Ordinary resume compares those fingerprints
|
|
when progressively loading consumer stage checkpoints, so changed producer
|
|
content produces `dependency_invalidated` rather than stale downstream reuse.
|
|
Selective recomputation instead requires each unselected producer's accepted
|
|
normalized artifact; invalid accepted state records its specific bounded reason
|
|
and stops before the dependent. The selected step and its transitive dependents
|
|
record `forced_recompute`.
|
|
|
|
Ordinary resume loads extract, merge, and normalize checkpoints progressively
|
|
and may execute later lane stages after an earlier cache miss. Selective
|
|
recomputation instead asks the loader for the required producer's accepted
|
|
normalize artifact. That lookup reuses the existing normalize files, requires
|
|
workspace schema v3 plus an exact non-empty invocation identity, and deliberately
|
|
does not require extract or merge checkpoint files or dependency fingerprints.
|
|
The runner performs canonical codec and producer-provenance validation before
|
|
cloning the artifact into normal step output. Success restores only stored
|
|
normalize warnings and emits one normalize decision; failure retains the files,
|
|
records the decision, and stops without executing the producer or consumer.
|
|
|
|
The loader assigns a typed category and reason code at each validation site;
|
|
diagnostic prose is not classified after the fact. The runner then applies
|
|
forced-execution policy, validates reusable artifact bytes through the prepared
|
|
codec once, returns the canonical hydrated value to the stage, and records the
|
|
final decision before enforcing a required-predecessor failure. That failure
|
|
names only the step, lane, and stable reason code. Decision detail is selected
|
|
from code-owned descriptions by reason code and then UTF-8 normalized and
|
|
bounded; callers cannot supply arbitrary diagnostic prose. Typed categories and
|
|
codes remain intact through pipeline events and become strings only in manifest
|
|
and debug-summary JSON.
|
|
[Operations](../operations.md#checkpoint-recording-resume-and-recompute) is the
|
|
canonical operator-facing reason-code reference.
|
|
|
|
`internal/core/fileio` provides confined atomic file writes used by state
|
|
collaborators. The chunk-plan store retains its stronger entry validation.
|
|
|
|
The CLI constructs selective-recomputation policy from resolved generated
|
|
artifact dependencies. It forces the selected step and transitive consumers,
|
|
while marking unforced producers as required reusable inputs. The runner owns
|
|
the actual hydration and rejection decisions; the [Operations guide](../operations.md#checkpoint-recording-resume-and-recompute)
|
|
owns the operator workflow and stable reason-code meanings.
|
|
|
|
## Debug Bundles
|
|
|
|
`internal/core/debugbundle` allocates an explicitly requested per-run bundle
|
|
with `summary/` and `trace/` roots. `SummaryWriter` persists redacted command,
|
|
resolution, run, warning, and failure artifacts. `internal/framework/debug`
|
|
implements the pipeline-facing trace recorder under the trace root.
|
|
|
|
The CLI allocates a bundle before pipeline resolution and treats requested
|
|
summary or trace persistence failures as command failures. The pipeline's debug
|
|
boundaries redact sensitive metadata and credential-shaped bytes while allowing
|
|
application-owned trace material. Debug data is never a checkpoint source or
|
|
cache input.
|
|
|
|
Generated reference bytes exist only in cloned operation requests and are not
|
|
written as paths into checkpoints, manifests, or debug summaries. Those state
|
|
surfaces retain canonical identities and bounded producer provenance so that a
|
|
resume decision can be explained without copying generated campaign content.
|
|
|
|
After allocation, one CLI-owned state value accumulates the known report paths,
|
|
pipeline outcome counts, and validation status. A single guarded terminalization
|
|
operation writes the success report, or makes one attempt each to write the
|
|
failure report and error log. Terminal persistence failures are reported
|
|
separately and never replace the command's primary error.
|
|
|
|
## Invariants To Preserve
|
|
|
|
- Modules receive state collaborators and request data, never physical roots.
|
|
- Output logical paths are validated before a run directory is allocated, and
|
|
files are atomically written within that directory.
|
|
- Chunk-plan publication occurs only for accepted plans; bypass does not
|
|
construct or touch a plan store.
|
|
- Checkpoint recording and checkpoint loading remain separate collaborators.
|
|
- Debug state is opt-in, is not cache input, and terminal reporting does not
|
|
obscure the command's primary failure.
|
|
|
|
## Tests To Inspect
|
|
|
|
- `internal/cli/run_contract_test.go`: command-owned state allocation,
|
|
terminalization, and output/report boundaries.
|
|
- `internal/cli/cache_contract_test.go`: cache-mode precedence, root selection,
|
|
and resume collaborator construction.
|
|
- `internal/cli/state_hardening_test.go`: independent roots, reuse, failures,
|
|
permissions, cleanup, and redaction.
|
|
- `internal/cli/recompute_policy_test.go`: forced dependents and required
|
|
reusable predecessors for selective recomputation.
|
|
- `internal/cli/recompute_execution_contract_test.go`: selective recomputation,
|
|
filesystem recovery, deterministic decisions, and failed predecessor state.
|
|
- `internal/cli/production_contract_test.go`: production composition and
|
|
configuration validation at the CLI boundary.
|
|
- `internal/cli/example_contract_test.go`: maintained example ownership.
|
|
- `internal/core/debugbundle/*_test.go`: bundle allocation and summary writes.
|
|
- `internal/framework/checkpoint/*_test.go`: checkpoint serialization and
|
|
reuse.
|
|
- `internal/framework/chunkplan/store_test.go`: plan envelope, confinement,
|
|
publication, and permissions.
|