From 6d18a3126ebd2fbbc3f91f72a2ea4114ad19b8ca Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Fri, 4 Sep 2026 13:21:27 +0000 Subject: [PATCH] Plan analyze artifact checkpointing --- docs/roadmap/implementation.md | 811 ++++++++++++++++----------------- 1 file changed, 395 insertions(+), 416 deletions(-) diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 99ce983..9910a27 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,483 +1,462 @@ -# Runner Lifecycle Refactoring Implementation Plan +# Workstream 6 Implementation Plan — Per-Artifact Analyze Checkpointing -## Status And Purpose +## Status -**Status: Completed** +Planning document for Workstream 6, **Per-artifact analyze checkpointing and +recovery**, in [`refactoring.md`](refactoring.md). Workstreams 1 through 5 are +complete and are prerequisites for this plan. All stages below are pending and +must be implemented in numerical order. -This is the decision-complete implementation plan for Workstream 5, **Runner -lifecycle and pre-execution decisions**, in -[`refactoring.md`](refactoring.md). Workstreams 1 through 4 are complete. -Workstream 5 is complete; Workstream 6 is the next pending workstream. This -plan's implementation scope remained limited to Workstream 5. +The purpose of this workstream is to make analyze a deterministic resumable +artifact queue backed only by the session manifest. If artifacts A and B +succeed, C fails, and D is not attempted, A and B must remain reusable, C must +be retried, and D must follow after C succeeds. The implementation must remove +the current end-of-stage analyze projection and rollback machinery rather than +building checkpointing alongside it as a second durable model. -The implementation must reduce the change blast and cognitive load of -`internal/app.executePlan` without changing the pipeline's observable behavior. -It must leave one explicit lifecycle owner, one session lock, one durable -session-manifest authority, a narrow shared planning/execution decision model, -and invocation-scoped test seams. It must not create a workflow engine, a -general dependency-injection framework, or new public extension points. +## Authoritative Scope And Decisions -The authoritative scope and target state are Workstream 5, the dependency -relationships, and the consolidated target state in -[`refactoring.md`](refactoring.md). The supporting audit decisions are H04, -H07, H14, H15, H17, H34, and H35 in -[`audit-decisions.md`](audit-decisions.md), with detailed analysis in -[`audit.md`](audit.md). +The authoritative product and architecture scope is Workstream 6, the +dependency relationships, compatibility intent, and consolidated target state +in [`refactoring.md`](refactoring.md). Supporting decisions are H04, H14, H15, +H36, H37, C02, and C03 in [`audit-decisions.md`](audit-decisions.md), with the +detailed rationale in H36 and H37 of [`audit.md`](audit.md) and Finding 3 of +[`contract-audit.md`](contract-audit.md). -## Scope And Fixed Decisions +The following decisions are fixed for this implementation: -- Keep the existing `internal/app` package boundary. Use private types, - functions, and focused files; do not create a runner package or expose - lifecycle internals. -- Keep the session manifest as the sole durable local progress ledger. Run IDs, - run directories, diagnostics, and `RunSummary` remain, but no second durable - invocation record may be introduced. -- Preserve the canonical flat stage order, bounded contiguous execution, - dependency-aware invalidation outside the selected range, stage-owned resume - validation, self-skip semantics, force semantics, and current analyze - projection behavior. -- Preserve the two prerequisite checks: a read-only check before persistent - mutation and a second check against the manifest while holding the session - lock. -- Keep lock acquisition and the deferred release visibly paired in the - top-level lifecycle function. A release failure remains the final invocation - error when no earlier error exists and is joined with an earlier error - otherwise. -- Preserve manifest ordering: establish identity before adapter construction; - persist `running` before invoking a stage; persist terminal stage state before - moving to the next stage; and persist terminal invocation failures where the - current runner does so. Do not make diagnostics a persistence authority. -- Keep actual `Stage.Run` results, result validation, persistence, logging, and - failure handling execution-only. Planning may model a successful or - configured self-skipped outcome on a cloned manifest, but must never pretend - that it observed a runtime result. -- The shared pure decision value must describe only whether to run or skip and - the pre-execution stale/invalidation work implied by recorded status, force, - and normalized resume validation. Small transition helpers may be shared - only where planning and execution perform the same manifest mutation. -- Preserve analyze's current single end-of-stage projection/save boundary and - rollback on save failure. Per-artifact checkpointing belongs to Workstream 6 - and is out of scope. -- Preserve post-publish cleanup and its durable obligation behavior until - Workstream 9. This work may move the call into finalization and make its - destructive function invocation-scoped, but must not turn cleanup into a - stage or change eligibility/retry semantics. -- Continue using `stage.Env` for injected stage collaborators. Add only small - private dependency values for nondeterministic or external construction - boundaries that cannot be represented by `stage.Env`, and pass those values - explicitly per command or invocation. -- Remove `executeStagesFn` and `runCommandFn`. Replace their tests with private - command helpers that accept a pipeline executor value for that call. Do not - migrate unrelated restore/config-loader globals in this workstream; their - owners should adopt the same narrow pattern when later work touches them. -- Replace the runner's mutable clock use with an invocation-scoped clock. A - normal immutable `nowUTC()` function may remain for unrelated command owners. - Route post-publish deletion through the invocation dependency value and - remove `removeRunScopedDirFn`. -- Preserve current CLI syntax, output, errors at their stable semantic level, - configuration behavior, workspace modes, secret loading/redaction, remote - lock revalidation, Linux/macOS/Windows support, and offline operation. -- Do not optimize for a particular private file layout or exact helper count. - The finished top-level runner must nonetheless read as a short, explicit - sequence of preparation, lock ownership, locked initialization, composition, - stage execution, and finalization rather than forwarding all work to another - monolith. -- Mark only Workstream 5 `Complete`, and only after all stages and full - validation succeed. Workstream 6 remains the next pending workstream. +- Keep analyze as one fixed pipeline stage in `internal/stage`; per-artifact + work does not become dynamic pipeline stages or a generic workflow engine. +- Keep the session manifest's `analyze_artifacts` collection as the only durable + per-artifact authority. Do not introduce an invocation collection, journal, + sidecar, database, or replacement run manifest. +- Use the existing `stage.Env.ManifestStore` and the canonical session-manifest + path for checkpoints. Production execution already supplies that store and + holds the session lock for the entire `Stage.Run` call. Do not add a callback + framework or expose application lifecycle types to `internal/stage`. +- Checkpoint a copy-on-write candidate manifest. Validate and save the candidate + before installing its analyze collection into the live in-memory manifest. + A failed current-record checkpoint must never make newly materialized bytes + authoritative through a later terminal save. +- Persist the reconciled starting collection before the first adapter call when + it changes durable authority. This removes obsolete records and stales + scheduled or invalid current records before replacement work begins. +- For every successful artifact: validate and materialize its canonical output, + construct and validate its current record, update the runtime catalog, stale + transitive dependents when output identity changed, checkpoint the complete + session collection, and only then attempt the next artifact. +- For an artifact failure: retain earlier successful checkpoints, replace the + active artifact with a bounded failed record, stale its current transitive + dependents, and expose that collection on the live manifest. The application + runner remains responsible for marking aggregate analyze failed, invalidating + publish/notify, and atomically persisting the terminal manifest before the + command returns the error. +- If persistence of a successful-artifact checkpoint fails, stop immediately. + Keep the last durable collection, mark the active artifact failed in the live + collection without promoting its incidental output, and let the runner's + terminal failure save make that retry state durable when storage permits. +- Preserve the existing deterministic work planner, reconciliation, + fingerprints, evidence validation, dependency order, and current-output + identity comparison. File presence alone never creates authority. +- Preserve artifact selection and force behavior. With no `--artifacts`, the + effective configured set is targeted. With `--artifacts`, only the named + targets and their required dependency closure participate. Force reruns + explicit targets; a valid current prerequisite is reused unless it is itself + explicit or becomes non-current through dependency work. Unrelated valid + records are retained. +- On aggregate success, return ordinary `StageResult.Outputs` rebuilt in + deterministic normalized artifact-key order from all current session analyze + records. Per-artifact records retain their own logs, generated configuration + references, bounded Scriptorium provenance, and evidence; aggregate logs and + metadata continue to describe the current invocation. +- On aggregate failure, aggregate outputs remain empty. Earlier current + per-artifact records remain available to a later analyze run, but publish and + notify are invalidated by the existing application-stage dependency relation. +- A crash or cancellation after a successful checkpoint may leave aggregate + analyze `running` or later normalized to `interrupted`; the next ordinary run + must reconcile the durable collection, reuse valid completed artifacts, and + continue with the first failed, stale, or missing artifact. +- Keep the manifest schema and analyze-state contract version unchanged. This + work changes persistence timing and removes an internal projection transport; + it does not introduce a new serialized representation. +- Do not enter Workstream 7 configuration decomposition, Workstream 8 publish + locks/decomposition, or Workstream 9 cleanup-stage work. ## Instructions For Every Stage Before each stage: 1. Read `docs/development.md`, every document under `docs/policy/`, this plan, - Workstream 5 plus the dependency and consolidated-target sections of - [`refactoring.md`](refactoring.md), and audit rows H04, H07, H14, H15, H17, - H34, and H35 in [`audit-decisions.md`](audit-decisions.md). Consult the - corresponding detailed findings in [`audit.md`](audit.md) when a boundary is - unclear. -2. Inspect the named production code and focused tests. Prefer the codebase - knowledge graph for Go symbol discovery and call tracing; use text search for - documentation, fixtures, serialized fields, and global-variable assignments. -3. Confirm the worktree state and preserve unrelated changes. Do not enter - Workstream 6 or Workstream 9 merely because analyze or cleanup code is - adjacent to runner lifecycle code. + and Workstream 6 in [`refactoring.md`](refactoring.md). Read the decision + entries and audit sections named above; do not re-audit unrelated findings. +2. Read [`../internal/stage-analyze.md`](../internal/stage-analyze.md) and + [`../internal/manifest.md`](../internal/manifest.md), then inspect the + production files and focused tests named by the stage. +3. Prefer the codebase knowledge graph for Go symbol discovery and call tracing. + Use text search for serialized fields, documentation, fixtures, and removal + of projection terminology. +4. Confirm the worktree state and preserve unrelated changes. Do not begin a + later workstream merely because adjacent code is visible. During every stage: -- Keep the repository compiling and focused tests passing. Do not leave both an - old and new runner path active except for a temporary private delegation that - the next numbered stage explicitly removes. -- Preserve package ownership and acyclic dependency direction. Private state - should aggregate already-related values, not become a service locator. -- Preserve error wrapping and joining where errors represent distinct failures. - Do not freeze incidental complete error strings in new tests. -- Keep tests deterministic, offline, behavior-focused, and lean under - `docs/policy/testing.md`. Prefer existing real temporary filesystems and - current adapter/storage fakes. Do not add mock choreography for private - lifecycle helpers. -- Update the canonical current-behavior document owned by a changed behavior in - the same stage. Pure file movement or private extraction requires no user - documentation change. -- Run `gofmt` on changed Go files, focused tests for the changed boundary, and - `git diff --check`. Mark a stage `Completed` only after its exit criteria pass. +- Keep the repository compiling and focused tests passing. A temporary private + delegation or projection bridge is permitted only where the next numbered + stage explicitly removes it; do not leave two durable authorities active. +- Preserve package ownership and acyclic dependency direction. Keep new types + private and cohesive; do not add public extension points. +- Keep all tests deterministic, offline, and independent of real Scriptorium or + paid APIs. Prefer the existing fake runner, real temporary filesystem, and a + small recording/failing manifest store only where checkpoint interaction is + itself the contract. +- Assert persisted manifest state, adapter calls, dependency order, and retry + behavior. Do not pin private helper choreography, complete error wording, or + incidental timestamps. +- Preserve bounded, non-secret errors and diagnostics. Never place prompt or + transcript content, variables containing secrets, or raw credentials in new + manifest fields or error assertions. +- Use `gofmt` on changed Go files, run the focused tests listed for the stage, + and run `git diff --check`. Mark a stage `Completed` only after its exit + criteria pass. -## Stage 1 — Unify Pre-Execution Decisions +## Stage 1 — Decompose The Single-Artifact Transaction -**Status: Completed** +**Status:** Pending ### Goal -Give planning and execution one pure representation of the decision made before -a stage attempt, while keeping planning predictions distinct from observed -runtime outcomes. +Make one artifact execution understandable and independently reviewable before +changing its persistence timing. This stage is behavior-preserving. ### Required Work -1. In `internal/app/run_control.go` or a focused private file, replace the - current action-only decision with an unexported value that records: - `run` versus `skip`; whether the selected successful record must first be - marked stale; the stage-owned reason for that stale transition; and the - application-owned invalidation reason, if any. Its constructor must be pure: - it receives the prior manifest status, `force`, and an already normalized - resume-validation result and does not mutate state, persist, log, read files, - invoke a stage, or obtain the current time. -2. Encode these cases explicitly: - - a non-forced succeeded record with no validator result or a resumable - result skips; - - a non-forced succeeded record with a non-resumable result runs after the - selected record is staled with the validator's reason and succeeded - dependents are invalidated with `staleReasonNotResumable`; - - a forced stage runs and invalidates succeeded dependents with - `staleReasonForcedReplacement` when the attempt begins; and - - absent, failed, stale, or skipped aggregate records run without inventing a - resume-stale transition. -3. Add a small helper that applies only the identical pre-execution manifest - changes described by the decision at a caller-supplied timestamp. Retain the - existing fixed invalidation relation and stable application-owned reasons. - The caller remains responsible for persistence and error context. -4. Use the decision constructor and application helper from both `Plan` and - `executePlan`. Each caller may perform stage-owned resume validation before - constructing the decision. Preserve analyze planning's read-only artifact - details when analyze is already a run candidate; those details are preview - information, not an execution outcome. -5. Extract only the exact shared attempt/result mutations already identical in - both paths: beginning an attempt (`running` plus forced invalidation), - applying a configured/runtime self-skip and its conditional invalidation, - and applying a predicted/actual success and its conditional invalidation. - Require callers to supply the actual or modeled outcome, outputs, semantic - fingerprint, prior outcome, and timestamp. Do not put `Stage.Run`, result - validation, logging, or manifest saves in these helpers. -6. Rewrite `modelPlannedStageRun` around these values or delete it if the - resulting focused helpers make it redundant. Keep all planning mutation on - the cloned manifest. -7. Add table-driven decision tests for every status/force/resume combination - and focused parity tests proving that plan and execution produce the same - run/skip choice and dependent-stale effects for ordinary skip, non-resumable - success, forced replacement, unchanged self-skip, changed self-skip, and a - successful result replacing a non-success record. Assert durable state and - decision semantics, not private helper call order or exact timestamps. +1. Refactor `executeAnalyzeArtifact` in `internal/stage/analyze.go` into one + short transaction owner plus private phases with explicit intermediate + values. Keep the existing `analyzeExecutionContext` and artifact plan, or + replace them with equally narrow private values; do not restore the former + broad parameter list. +2. Separate these concerns without changing their order or contracts: + - input identity resolution, omitted optional inputs, reused-current input + reporting, Scriptorium variables, timeout, canonical/run-local paths, and + request/diagnostic path preparation; + - optional render-debug invocation and validation; + - Scriptorium artifact invocation and adapter failure adaptation; + - authoritative output-path selection, bounded non-empty regular-file + validation, output-contract construction, canonical materialization, and + installed-checksum verification; and + - result/evidence construction, including provenance, logs, generated + configuration paths, metadata, output size, and reused artifacts. +3. Place the extracted code in focused analyze-owned files when that improves + navigation. Suggested responsibilities are preparation, adapter invocation, + and validated materialization/evidence; exact private filenames are not a + contract. +4. Keep Scriptorium request and response types inside the stage/adapter boundary. + Do not move Narratio artifact, manifest, path, or metadata policy into the + Scriptorium adapter. +5. Preserve the current run-local-before-canonical materialization boundary, + render-debug behavior, useful error context, workspace modes, centralized + size limit, and checksum comparison. Do not weaken validation or add a new + write path while moving code. +6. Retain existing behavior tests at their stable boundaries. Consolidate only + tests that become purely coupled to moved private helpers; do not add one + test per extracted function. ### Exit Criteria -- `go test ./internal/app` succeeds. -- `Plan` and `executePlan` no longer independently encode the same - pre-execution stale/force decision sequence. -- The decision constructor is pure, and actual stage outcomes and persistence - remain outside it. -- Planning remains read-only and execution behavior is unchanged. +- `executeAnalyzeArtifact` reads as a short ordered transaction over explicit + phases and no longer mixes all preparation, invocation, materialization, and + evidence construction in one large function. +- Existing analyze request, render, validation, materialization, provenance, + and failure behavior is unchanged. +- `go test ./internal/stage ./internal/adapters/scriptorium` succeeds. - `git diff --check` succeeds. -## Stage 2 — Establish The Invocation Runtime And Lock Lifecycle +## Stage 2 — Establish Session-Only Analyze Execution State -**Status: Completed** +**Status:** Pending ### Goal -Make invocation state cohesive and make the top-level transaction order obvious -without hiding session-lock ownership. +Replace `analyzeStage.Run`'s parallel local collections with one private +session-oriented execution state that is ready to checkpoint, while retaining +the existing end-of-stage persistence boundary temporarily. ### Required Work -1. Introduce an unexported `invocationRuntime` (or equivalently narrow private - type) under `internal/app`. It should carry only state already shared across - lifecycle phases: context, resolved configuration, bounded plan/stages, - normalized options, effective artifacts, run identity and canonical paths, - `stage.Env`, manifest store, artifact store, loaded session manifest, and - summary accumulators. Do not expose it or let stages depend on it. -2. Extract unlocked preparation that performs the current read-only bounded - prerequisite inspection, resolves effective artifacts, generates the run - ID, resolves invocation identity, initializes/clones `stage.Env` fields, and - supplies default manifest/artifact stores and logger. Preserve the rule that - injected `Env` values provide collaborators but never an alternate config, - selected-artifact set, or effective artifact set. -3. Extract layout preparation separately. Keep `EnsureLayoutFor` before lock - acquisition as today, verify its manifest path against the canonical - resolved path, and return enough state for the locked lifecycle. -4. In the short top-level executor, acquire the session lock after unlocked - preparation and visibly install one deferred release beside that acquisition. - Preserve exact release-error joining behavior. Do not put lock ownership in - `invocationRuntime`, a constructor, or a helper whose lifetime is ambiguous. -5. Extract locked initialization that checks the incomplete-restore marker, - loads and validates or creates the session manifest, rechecks bounded - prerequisites, applies identity and effective-configuration provenance, - saves the identity transition, ensures the canonical run directory, and - loads configured secrets. All steps after manifest availability that - currently record a terminal failure must continue to do so through the - session-only failure helper. -6. Keep pre-lock failures free of persistent mutation and keep all later - manifest transitions protected by the same held lock. Do not add rollback, - a second lock, or a second manifest authority. -7. Adapt existing bounded-run, restore-marker, identity, run-directory, - terminal-failure, and lock-release tests to exercise the assembled lifecycle. - Add coverage only if the two-check prerequisite ordering or lock-release - error behavior is not already credibly protected. +1. Introduce a private analyze execution state/accumulator that owns: + - the shared execution context and deterministic work plan; + - the complete reconciled session artifact collection; + - prior current records needed for output-identity comparison; + - the evaluated/scheduled key set used for dependent invalidation; + - the mutable runtime artifact catalog; and + - invocation-only aggregate logs, generated configurations, generated + metadata, reused-artifact metadata, and its deduplication set. +2. Keep setup, reconciliation, work planning, adapter-availability validation, + deterministic iteration, and final `StageResult` assembly visible in + `analyzeStage.Run`. Move per-item state mutation and diagnostic collection + behind narrow methods on the private state; do not turn the accumulator into + a service locator. +3. Rename `analyzeWorkPlan.ProjectedRecords`, `projectedAnalyzeRecords`, and + related private terminology to describe the reconciled starting session + collection. It is the candidate durable state, not an invocation projection. + Preserve the planner's read-only behavior and existing classification. +4. Make the accumulator install its complete collection into the passed session + manifest on success and its failed/staled collection on error. During this + stage, continue returning the existing `AnalyzeStateProjection` as a + temporary compatibility bridge for the application runner; both views must + be derived from the same collection, and no additional save is introduced. +5. Add optional `ProducerRunID` provenance to `artifacts.Ref`. Update the + generic application result mapper to use that value when it is non-empty and + otherwise retain the current behavior of assigning the active run ID. This + is data needed by ordinary stage-result transport, not an analyze-specific + branch; cover both explicit and fallback behavior in the mapper tests. +6. Add a private helper that deterministically converts all current analyze + records into ordinary `artifacts.Ref` outputs. Preserve source ID, relative + path, contract, checksum, and producer run ID. Include valid current records + retained outside a partial selection; do not include stale, missing, or + failed records. Continue using the projection path for aggregate outputs + until Stage 3 switches ownership. +7. Rewrite stage tests to inspect the manifest collection mutated by the real + analyze stage rather than manually installing returned projections. Retain a + minimal assertion that the temporary projection matches that collection so + the transition cannot silently diverge. ### Exit Criteria -- `go test ./internal/app ./internal/artifacts ./internal/manifest` succeeds. -- The top-level executor visibly performs unlocked preparation, acquires and - defers release of exactly one session lock, initializes locked state, and - proceeds to execution. -- Invalid preflight requests create no persistent invocation state; the locked - prerequisite recheck still prevents races with a prior invocation. -- The private runtime has cohesive fields and no public or cross-package API. +- `analyzeStage.Run` coordinates one private session collection rather than + parallel record and diagnostic variables. +- Planning/reconciliation remains read-only, execution order and force/selection + behavior are unchanged, and stage tests primarily assert manifest state. +- Ordinary `StageResult.Outputs` can preserve a reused artifact's original + producer run identity without an analyze-specific application path. +- The existing application projection path still functions and no checkpoint + timing has changed yet. +- `go test ./internal/stage ./internal/app` succeeds. - `git diff --check` succeeds. -## Stage 3 — Isolate Collaborator Composition +## Stage 3 — Cut Analyze Over To Durable Per-Artifact Checkpoints -**Status: Completed** +**Status:** Pending ### Goal -Make production collaborator construction one bounded lifecycle phase that is -selective, testable per invocation, and independent of stage transitions. +Make each validated artifact completion durable before later artifact work and +stop using the end-of-stage projection transport for real analyze execution. ### Required Work -1. Define a small unexported invocation dependency value with explicit defaults - for the nondeterministic and external construction boundaries used by this - runner: UTC time, run-ID generation, default adapter/object-store creation, - effective remote-lock loading/revalidation support, and post-publish scoped - removal. Include a field only where tests need control over a real external, - destructive, clock, or randomness boundary; continue injecting completed - stage collaborators through `stage.Env`. -2. Add a private executor variant that accepts that dependency value and have - the ordinary executor supply production defaults. Normalize missing fields - once at the entry boundary so lifecycle methods never fall back to mutable - package state. Do not export the dependency type or store it globally. -3. Route run-ID creation and every runner timestamp through the invocation - dependencies. Convert `nowUTC` to an ordinary function for unrelated - command owners if it no longer needs to be mutable; do not broaden this stage - into a restore lifecycle refactor. -4. Extract collaborator composition from the runner into one operation over the - invocation runtime. Preserve current selection gates exactly: WhisperX only - for `transcribe`; Seriatim only for `merge`, `normalize`, `trim`, or `render`; - Audita only for `polish`; Notarius only when selected extraction requires it; - Scriptorium only for `trim` or `analyze`; object storage only when selected - stages/effective artifacts require it; remote lock loading only when needed; - and the no-op notifier only for `notify`. -5. Preserve injected collaborators, secret-loaded environment behavior, - effective lock merging, and the publish-time revalidation closure. A - composition failure after identity initialization remains a persisted - terminal session failure with its existing useful context. -6. Route post-publish cleanup's scoped deletion through the invocation - dependency value and delete `removeRunScopedDirFn`. Preserve cleanup - obligation creation, per-target persistence, retry, and failure behavior. -7. Consolidate selective-composition tests into table-driven or focused - behavioral cases. Test that an unselected external boundary is not - constructed only where avoiding that construction is an operational - requirement; do not assert private helper ordering. +1. Add a private analyze checkpoint operation in `internal/stage`. It must: + - require a non-nil session manifest, analyze stage record, and + `Env.ManifestStore` when analyze has state to persist; + - derive the canonical manifest path through `artifacts` helpers from the + resolved workspace, campaign, and session identity; + - validate the full candidate collection with + `manifest.ValidateAnalyzeArtifactCollection`; + - shallow-copy the manifest, clone the top-level stage map, copy the analyze + stage record, and deep-clone the candidate analyze collection so the save + cannot mutate live authority before success; + - preserve the aggregate analyze status (normally `running`) and every + unrelated manifest field; + - save the candidate through `Env.ManifestStore`; and + - only after success, install the candidate analyze version/collection and + the store-updated manifest timestamp into the live manifest. + Do not add a general manifest transaction framework or JSON round-trip clone. +2. Before invoking the first scheduled artifact, checkpoint the reconciled + starting collection when it differs from durable state. This must remove + records for removed configuration and stale any selected/forced/non-current + authority before replacement work begins. If this save fails, invoke no + adapter. +3. For each scheduled artifact, preserve this order: + - compute its execution fingerprint; + - execute the decomposed artifact transaction; + - construct and validate its current manifest record; + - prove the generated record can be added to the runtime catalog for use by + later dependents; + - stale unscheduled transitive dependents only when prior and replacement + output identities differ; + - checkpoint the complete updated session collection; and + - only then expose the result to a later artifact. +4. If a successful artifact's checkpoint fails, stop immediately and do not + call the next artifact. Build a failed record for the active artifact from + the last committed collection, with no output authority; stale its current + transitive dependents; and install that failure candidate in memory so the + runner's terminal save can persist retry state. Return the checkpoint + failure with artifact identity and manifest context. +5. Have real analyze execution return ordinary `StageResult.Outputs` built from + every current session record plus its existing aggregate diagnostics and + metadata. Stop returning `AnalyzeStateProjection` from `analyzeStage.Run`. + The legacy projection type and application handlers may remain only as dead + transitional code until Stage 4; add no new uses. +6. In `executeSelectedStage`, make a terminal save failure after any stage's + apparent success mark that stage failed and invalidate succeeded transitive + dependents before terminal persistence. This behavior currently occurs only + on the analyze-projection branch; it must remain correct when real analyze no + longer returns a projection and should be correct generically. +7. Update direct analyze fixtures so they supply a real temporary + `manifest.LocalStore`, complete campaign/run identity, an aggregate analyze + record in the state appropriate to the test, and an initially persisted + manifest. Keep unit-only planner/reconciliation tests free of persistence. +8. Add focused checkpoint tests at the stage/application boundary: + - the manifest store observes A current before B is invoked and A+B current + before C is invoked; + - a failed B checkpoint prevents C from being invoked; + - the failed B checkpoint never persists B as current merely because its + canonical file exists; and + - final aggregate success exposes outputs for all and only current records + in deterministic normalized artifact-key order, preserving each record's + producer run identity. ### Exit Criteria -- `go test ./internal/app` succeeds, including composition, remote-lock, - cleanup, and bounded-stage tests. -- Collaborator construction is absent from the top-level runner and stage loop, - but remains an application-owned production-composition concern. -- Runner time, randomness, external construction, and destructive cleanup are - controllable per invocation without mutable runner globals. -- Selected-stage composition and terminal-failure behavior remain unchanged. +- Every successful scheduled artifact is durable before the next adapter call. +- A checkpoint failure stops later work and cannot promote incidental output. +- Real analyze execution no longer relies on an end-of-stage state projection. +- Aggregate success/failure persistence and dependent invalidation remain + application-owned and correct. +- `go test ./internal/stage ./internal/app ./internal/manifest` succeeds. - `git diff --check` succeeds. -## Stage 4 — Extract One-Stage Execution +## Stage 4 — Complete Failure Recovery And Remove Projection Machinery -**Status: Completed** +**Status:** Pending ### Goal -Give one private operation complete ownership of a single stage attempt while -preserving every durable transition and failure boundary. +Complete the A/B/C/D recovery contract and delete the superseded projection, +validation, application, and rollback paths. ### Required Work -1. Extract an operation on the invocation runtime that executes exactly one - selected stage and returns a small execution-owned outcome sufficient to - update the in-memory summary or stop the invocation. It may coordinate the - shared decision/transition helpers but must not select the next stage or - release the session lock. -2. Within that operation, preserve this order: - - compute the stage semantic fingerprint; - - run stage-owned resume validation when the recorded success is a skip - candidate; - - build and apply the pure pre-execution decision and persist any - non-resumable stale/invalidation transition; - - return a skip outcome without invoking the stage when still resumable; - - capture prior aggregate/analyze state, begin the attempt, persist - `running`, and then invoke `Stage.Run`; - - validate the stage result and any analyze projection; - - apply failure, self-skip, or success transitions and required dependent - invalidation; and - - persist the terminal state before returning. -3. Preserve failure semantics for semantic fingerprinting, resume validation, - invalidation, the pre-run save, stage execution, result validation, - self-skip/success saves, and terminal failure persistence. A failed stage or - persistence operation stops later stages. Distinct invalidation/save errors - remain joined with the originating failure. -4. Preserve current analyze behavior exactly: a valid partial failure - projection may be applied before recording failure; a successful projection - supplies aggregate outputs; and failure to save a successful projection - restores prior analyze authority before recording aggregate failure. - Introduce no callback/store dependency into `internal/stage`. -5. Keep manifest save calls explicit in the one-stage owner. Do not hide them in - generic state-machine hooks or teach `manifest.Store` orchestration policy. -6. Reduce the outer stage loop to call the one-stage operation, append its - executed/skipped classification, and stop on error. Preserve deterministic - `StageNames`, `Executed`, and `Skipped` order and current logging semantics. -7. Reconcile runner tests around observable manifest state, invocation summary, - stage calls, and errors. Preserve coverage for ordinary skip, force, - non-resumable state, self-skip, result validation, cancellation, downstream - invalidation outside a bounded range, save failures, analyze rollback, and - stopping after failure. Remove redundant tests that exist only to pin the old - monolithic control flow. - -### Exit Criteria - -- `go test ./internal/app ./internal/stage ./internal/manifest` succeeds. -- The stage loop contains no duplicated transition policy and one-stage - execution has one clear persistence owner. -- A later stage is never attempted after a stage or required manifest save - fails, and no failed/interrupted attempt is presented as successful. -- Analyze still uses its pre-Workstream-6 aggregate persistence boundary. -- `git diff --check` succeeds. - -## Stage 5 — Isolate Finalization And Simplify The Lifecycle Owner - -**Status: Completed** - -### Goal - -Complete the runner decomposition so the lifecycle and transaction boundaries -are easier to read than the former monolith. - -### Required Work - -1. Extract finalization that performs the current post-publish cleanup only - when publish actually executed, using the runtime's session manifest, - manifest path, environment, executed-stage list, and invocation-scoped - deletion dependency. Preserve the special cleanup-failure persistence and - the fact that a committed remote publish remains authoritative. -2. Build `RunSummary` from the runtime's resolved identity and ordered - accumulators only after all selected stages and required cleanup complete. - Continue returning no success summary on invocation failure. -3. Make the top-level executor an explicit sequence: unlocked preparation; - layout preparation; visible lock acquisition/deferred release; locked state - initialization; collaborator composition; ordered one-stage execution; and - finalization/summary. It must retain meaningful error context at each phase - and must not delegate that entire sequence to a new monolithic helper. -4. Place private lifecycle concerns in focused files according to ownership—for - example runtime/preparation, composition, stage execution, and finalization— - while keeping exact shared transition logic in `run_control.go`. Remove dead - helpers, stale comments, duplicate state, and imports left by extraction. -5. Review the resulting runtime for service-locator growth. Remove fields used - by only one local operation unless retaining them materially clarifies the - lifecycle. Keep collaborators in `stage.Env` and persistence/path policy in - their existing owners. -6. Run focused assembled-workflow, cleanup, bounded-run, resume, manifest-save, - and release-error tests. Add no tests for private file placement, helper - count, or line count. - -### Exit Criteria - -- `go test ./internal/app ./internal/stage ./internal/manifest ./internal/artifacts` - succeeds. -- `executePlan` is a short, readable lifecycle owner with one visibly scoped - lock and no nested stage state machine or collaborator-construction block. -- Cleanup remains a post-publish obligation with unchanged behavior, awaiting - Workstream 9. -- The refactor has removed obsolete code rather than retaining parallel old and - new lifecycle paths. -- `git diff --check` succeeds. - -## Stage 6 — Remove Global Runner Seams And Close Workstream 5 - -**Status: Completed** - -### Goal - -Prove the target architecture through invocation-scoped command tests, accurate -current documentation, and complete repository validation. - -### Required Work - -1. Introduce the smallest private command-level helper needed to pass a - pipeline-executor function for one call through `Execute`, `Run`, - `RegenerateArtifacts`, and the single-stage command path. Public command - functions continue to use the production executor. The alias must still be - exactly equivalent to `run --force --from extract --through analyze` with - caller options forwarded under the existing rules. -2. Delete `executeStagesFn` and `runCommandFn`. Rewrite affected CLI, profile, - remote-session, restore-workflow, assembled-workflow, bounded-run, alias, and - artifact-selection tests to call the private command helper with an explicit - executor or to exercise the real assembled runner. Do not serialize tests - around package-global save/replace/restore choreography. -3. Confirm that `removeRunScopedDirFn` is gone and that runner tests do not - replace `nowUTC`. Inventory remaining mutable function variables in - `internal/app`; leave unrelated configuration-loading and restore seams for - their owning future work, but ensure no runner, run-command, alias, or - post-publish-cleanup test mutates process-global behavior. -4. Review test value under `docs/policy/testing.md`. Consolidate duplicate - command-wiring and lifecycle cases exposed by the new narrow seams while - retaining consequential lock, persistence, invalidation, recovery, - cancellation, external-boundary, and representative assembled-workflow +1. On any artifact-specific failure—including fingerprinting, preparation, + render, adapter execution, result validation, materialization, evidence + construction, catalog admission, or checkpoint persistence—construct one + bounded failed record for the active key, clear its output authority, and + stale current transitive dependents. Preserve unrelated current records and + all earlier successfully checkpointed records. +2. Install the failed/staled collection on the live manifest before returning + the artifact error. The runner must then use its ordinary stage-error path to + mark aggregate analyze failed, clear aggregate outputs, invalidate publish + and notify, and persist the terminal session manifest. Join a terminal save + failure with the originating artifact/checkpoint error; never replace the + useful cause. +3. Ensure unattempted artifacts behave according to existing authority: + previously absent records remain absent/missing on reconciliation; unrelated + valid current records remain current; and a prior current record that + depends directly or transitively on the failed artifact becomes stale with + no output authority. +4. Remove `AnalyzeStateProjection` from `stage.StageResult`, delete + `internal/app/analyze_projection.go`, and remove all validation, cloning, + application, capture, rollback, and special output reconstruction that exists + solely for that projection. Remove fake projection stages and projection-only + tests; replace their meaningful risks with real analyze checkpoint/recovery coverage. -5. Update current internal documentation where ownership requires it: - `docs/internal/overview.md` should summarize the explicit runner lifecycle; - `docs/internal/manifest.md` should describe the shared pre-execution decision - and single persistence owner without documenting private helper names; and - `docs/internal/adapters.md` should describe selective application-owned - composition. Update architecture policy only if implementation reveals a - normative boundary not already stated. Do not duplicate CLI/configuration - contracts or edit historical release documents and completed audit findings. -6. Search current code and documentation for stale claims about the monolithic - runner, independent plan/run decisions, or mutable runner seams. Remove only - inaccurate current-behavior text; roadmap and audit history remain valid. -7. Update Workstream 5 in `refactoring.md` from `Pending` to `Complete`, and - change all six stage statuses plus this plan's status to completed only after - the target state and all checks are proven. Leave Workstream 6 `Pending`. -8. Run the complete repository validation from `docs/development.md`: - `go test ./...`, `go test -race ./...`, `go vet ./...`, `go build ./...`, - `go test ./internal/doccheck`, and - `go test ./internal/config -run '^TestExamplesLoadAndValidate$'`. Also run - `git diff --check` and confirm the implementation commit leaves no unintended - worktree changes. +5. Simplify `executeSelectedStage` to its generic result/error handling while + preserving analyze's live manifest collection across aggregate lifecycle + clearing. Do not add a type assertion, stage-name branch, callback, or new + result union to replace the projection special case. +6. Add a durable assembled-run regression for the approved scenario: + - configure deterministic A, B, C, and D work in dependency order; + - make A and B succeed, C fail, and verify D is not attempted; + - reload the manifest and verify A/B current, C failed, aggregate analyze + failed, and the correct missing/stale state for D; + - run again without force, verify only C and then D execute, and verify the + final aggregate and all selected records are current; and + - verify valid unrelated records and upstream transcript stages were never + invalidated or rerun. +7. Add focused variants for failure of the first and last artifact, cancellation + after at least one checkpoint, changed-output invalidation of an unselected + transitive dependent, and byte/contract-identical replacement preserving an + unselected dependent. Consolidate with existing incremental/failure tests so + each durable risk has one clear owner. ### Exit Criteria -- All repository-wide validation succeeds on Go 1.26. -- Planning and execution share one pure pre-execution decision model while - actual results and persistence remain execution-owned. -- The runner reads as an explicit lifecycle with one session lock and one - session-manifest authority. -- Runner, run-command, alias, and cleanup tests use invocation-scoped - dependencies rather than mutable package-global function substitution. -- Current internal documentation accurately describes the implemented design - at the appropriate level. -- Workstream 5 and all six stages are marked `Complete`; Workstream 6 is the - next pending workstream. -- `git status --short` is empty after the implementation commit. +- A/B remain durable after C fails; an ordinary later run retries C and then D + without invoking A/B. +- Artifact failure and checkpoint failure leave no false current authority, and + publish/notify cannot remain successfully current after aggregate failure. +- No analyze projection type, field, helper, application branch, rollback path, + test fixture, or documentation claim remains. +- `go test ./internal/stage ./internal/app ./internal/manifest ./internal/artifacts` + succeeds. +- `git diff --check` succeeds. + +## Stage 5 — Verify Force Semantics, Align Documentation, And Close Workstream 6 + +**Status:** Pending + +### Goal + +Verify the complete operational contract, remove superseded test/documentation +language, and leave Workstream 6 closed with repository-wide evidence. + +### Required Work + +1. Review existing force/selection/reconciliation tests and retain a lean set + that proves: + - an ordinary run reuses every valid current selected artifact; + - unqualified force reruns the full effective configured target set; + - selected force reruns the explicit targets, includes their dependency + closure, reuses valid current prerequisites, and leaves unrelated valid + artifacts untouched; + - stale, failed, missing, or dependency-invalid prerequisites execute in + deterministic topological order before their dependents; and + - an incidental canonical file without current manifest evidence is never + reused. +2. Replace projection-oriented test names, helpers, and assertions with + checkpoint/recovery language. Remove redundant tests whose only purpose was + to validate the retired transport or rollback choreography. Do not reduce + coverage of output validation, dependency invalidation, cancellation, + persistence failure, or assembled resume behavior. +3. Update the canonical current-behavior owners: + - `docs/internal/stage-analyze.md` for checkpoint order, resume behavior, + failure semantics, and the absence of projections; + - `docs/internal/manifest.md` for per-artifact checkpoint authority, + aggregate status, interruption recovery, and persistence failure; + - `docs/policy/architecture.md` only where its stage/persistence description + currently implies a single end-of-stage analyze transaction; and + - `docs/operations.md` and `docs/troubleshooting.md` only where operators + need to understand reuse after partial analyze failure or recover from a + manifest checkpoint error. + Do not duplicate CLI or configuration contracts, since syntax and fields do + not change. +4. Search production, tests, and current-behavior documentation for + `AnalyzeStateProjection`, `AnalyzeState`, `analyze projection`, + `projected records`, and `end-of-stage` language. Remove or revise every + reference tied to the retired mechanism; preserve unrelated planning uses of + the ordinary English word “projection” only when accurate. +5. Run the full repository validation from `docs/development.md`: + + ```sh + go test ./... + go test -race ./... + go vet ./... + go build ./... + go test ./internal/doccheck + go test ./internal/config -run '^TestExamplesLoadAndValidate$' + ``` + +6. Run `go test -race -shuffle=on -count=3 ./internal/stage ./internal/app` to + stress the changed recovery and persistence boundaries. Run the supported + release cross-build checks if an implementation change affects build-tagged + code; otherwise the repository build is sufficient for this workstream. +7. After all checks pass, mark Stages 1–5 in this document `Completed` and mark + only Workstream 6 in [`refactoring.md`](refactoring.md) `Complete`. + Workstream 7 must remain the next pending workstream. + +### Exit Criteria + +- Current documentation describes analyze as a session-manifest-backed, + per-artifact checkpointed queue and contains no retired projection behavior. +- The force, partial-selection, dependency, interruption, and persistence-error + cases are protected through stable behavioral boundaries. +- All repository validation and focused shuffled race checks pass. +- Workstream 6 is marked complete, Workstream 7 remains pending, and the + worktree contains no accidental generated or temporary files. +- `git diff --check` succeeds. ## Open Questions -None. The approved audits, current policies, and Workstream 5 decisions resolve -the behavior, architecture, testability, and sequencing choices required for -implementation. +None. The audits, approved Workstream 6 scope, existing selection contract, and +the fixed decisions above resolve the product and implementation choices needed +for the coding agent to execute this plan.