eric/narratio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 27 Wiki Activity

Plan runner lifecycle refactor

Browse Source
This commit is contained in:
Eric Rakestraw
2026-09-04 00:26:04 +00:00
parent ec072dc163
commit 9331373245
1 changed files with 411 additions and 326 deletions
Download Patch File Download Diff File Expand all files Collapse all files

737
docs/roadmap/implementation.md
View File

@@ -1,396 +1,481 @@
# Workstream 4 Implementation Plan # Runner Lifecycle Refactoring Implementation Plan
## Purpose And Status ## Status And Purpose
This is the executable plan for [`refactoring.md`](refactoring.md), Workstream This is the decision-complete implementation plan for Workstream 5, **Runner
4: **One durable local manifest authority**. Finding 2 of lifecycle and pre-execution decisions**, in
[`contract-audit.md`](contract-audit.md) and rows H14, H15, H17, C02, and C03 [`refactoring.md`](refactoring.md). Workstreams 1 through 4 are complete.
of [`audit-decisions.md`](audit-decisions.md) own the approved decisions. Workstream 5 is the next pending workstream and is the only feature scope of
this plan.
**Status: Completed** The implementation must reduce the change blast and cognitive load of
`internal/app.executePlan` without changing the pipeline's observable behavior.
It must leave one explicit lifecycle owner, one session lock, one durable
session-manifest authority, a narrow shared planning/execution decision model,
and invocation-scoped test seams. It must not create a workflow engine, a
general dependency-injection framework, or new public extension points.
All five stages are completed. Workstreams 1 through 4 are complete. The authoritative scope and target state are Workstream 5, the dependency
Workstreams 5 and later remain out of scope: do not decompose the runner into a relationships, and the consolidated target state in
new lifecycle type, redesign pre-execution decisions, add per-artifact analyze [`refactoring.md`](refactoring.md). The supporting audit decisions are H04,
checkpoints, split publish into separate transactions, add cleanup as a stage, H07, H14, H15, H17, H34, and H35 in
or change remote lock policy while following this plan. [`audit-decisions.md`](audit-decisions.md), with detailed analysis in
[`audit.md`](audit.md).
## Settled Implementation Decisions ## Scope And Fixed Decisions
- The session manifest is the only durable local progress ledger. It alone - Keep the existing `internal/app` package boundary. Use private types,
records stage status, current artifacts, semantic evidence, diagnostics, functions, and focused files; do not create a runner package or expose
failures, resume authority, restore authority, and publish authority. lifecycle internals.
- Delete the structured invocation run manifest completely: its schema and - Keep the session manifest as the sole durable local progress ledger. Run IDs,
types, store interface and methods, path helper, lifecycle mutations, run directories, diagnostics, and `RunSummary` remain, but no second durable
terminal recovery, analyze projection, publish dependency, fixtures, tests, invocation record may be introduced.
and current documentation all leave the repository. - Preserve the canonical flat stage order, bounded contiguous execution,
- Do not replace it with another JSON summary, journal, database, or event log. dependency-aware invalidation outside the selected range, stage-owned resume
Historical per-invocation state is not an operational requirement. validation, self-skip semantics, force semantics, and current analyze
- Preserve run IDs. Every invocation still receives one before session state is projection behavior.
initialized; it remains the identity used by the session manifest, - Preserve the two prerequisite checks: a read-only check before persistent
run-local directory, adapter provenance, logs, generated configuration, mutation and a second check against the manifest while holding the session
receipts, immutable publish prefix, command summary, and diagnostics. lock.
- Explicitly create the canonical run directory once per invocation after the - Keep lock acquisition and the deferred release visibly paired in the
session lock and identity checks. Removing the initial run-manifest save must top-level lifecycle function. A release failure remains the final invocation
not make diagnostic writers responsible for accidentally creating the run error when no earlier error exists and is joined with an earlier error
root. Directory-creation failure remains an invocation initialization error. otherwise.
- `RunSummary` remains the immediate in-memory command result and retains - Preserve manifest ordering: establish identity before adapter construction;
session ID, run ID, session manifest path, selected stage names, executed persist `running` before invoking a stage; persist terminal stage state before
stages, and skipped stages. Remove only `RunManifestPath`; do not add a moving to the next stage; and persist terminal invocation failures where the
replacement persistent summary. current runner does so. Do not make diagnostics a persistence authority.
- A run-manifest write can no longer fail an otherwise valid pipeline action. - Keep actual `Stage.Run` results, result validation, persistence, logging, and
Session-manifest transition and terminal-save failures remain correctness failure handling execution-only. Planning may model a successful or
failures and continue to join with the originating error where applicable. configured self-skipped outcome on a cloned manifest, but must never pretend
- Publish no longer opens or validates `runs/<run_id>/manifest.json`. When that it observed a runtime result.
`upload_run` is enabled, it derives the current run root canonically and uses - The shared pure decision value must describe only whether to run or skip and
the session manifest's recorded output, log, and generated-configuration the pre-execution stale/invalidation work implied by recorded status, force,
references as an allowlist. It includes only safe regular files confined to and normalized resume validation. Small transition helpers may be shared
the current run root and ignores records from earlier runs. only where planning and execution perform the same manifest mutation.
- New remote commits do not upload or classify a run manifest. Keep - Preserve analyze's current single end-of-stage projection/save boundary and
`RemoteArtifactTypeRunManifest` as a read-only wire value so already committed rollback on save failure. Per-artifact checkpointing belongs to Workstream 6
and verified remote snapshots remain readable; it must have no local loader, and is out of scope.
path, special publish branch, or newly emitted declaration. This preserves a - Preserve post-publish cleanup and its durable obligation behavior until
durable remote protocol without retaining the local dual-authority model. Workstream 9. This work may move the call into finalization and make its
- Preserve `RemoteArtifactTypeRunFile` for declared run-local diagnostics. destructive function invocation-scoped, but must not turn cleanup into a
Publishing no current-run diagnostics is valid and must not require a stage or change eligibility/retry semantics.
placeholder file. - Continue using `stage.Env` for injected stage collaborators. Add only small
- Remove the invocation subset from `AnalyzeStateProjection` and from analyze private dependency values for nondeterministic or external construction
execution. During this workstream, analyze still returns and persists one boundaries that cannot be represented by `stage.Env`, and pass those values
validated reconciled session collection at the existing end-of-stage explicitly per command or invocation.
boundary. Do not implement artifact-by-artifact persistence yet. - Remove `executeStagesFn` and `runCommandFn`. Replace their tests with private
- Preserve the current safe behavior when the end-of-stage analyze manifest command helpers that accept a pipeline executor value for that call. Do not
save fails: incidental output bytes do not establish authority, prior migrate unrelated restore/config-loader globals in this workstream; their
in-memory analyze authority is restored, and the aggregate stage fails. owners should adopt the same narrow pattern when later work touches them.
Workstream 6 will replace this with the approved checkpoint model. - Replace the runner's mutable clock use with an invocation-scoped clock. A
- Preserve current stage ordering, force/invalidation rules, self-skip behavior, normal immutable `nowUTC()` function may remain for unrelated command owners.
aggregate transition timing, cleanup obligations, remote commit ordering, Route post-publish deletion through the invocation dependency value and
and command output except where they directly expose the removed run remove `removeRunScopedDirFn`.
manifest. - Preserve current CLI syntax, output, errors at their stable semantic level,
- Delete tests whose only contract is historical run-manifest persistence. configuration behavior, workspace modes, secret loading/redaction, remote
Rewrite mixed tests around session state, command summaries, run-local lock revalidation, Linux/macOS/Windows support, and offline operation.
diagnostics, or remote commit contents, and consolidate redundant mechanism - Do not optimize for a particular private file layout or exact helper count.
assertions rather than recreating a second audit record in tests. The finished top-level runner must nonetheless read as a short, explicit
- Historical release notes and completed audit documents remain unchanged. sequence of preparation, lock ownership, locked initialization, composition,
Current architecture, CLI, operations, troubleshooting, and internal stage execution, and finalization rather than forwarding all work to another
documentation must describe session-only authority. monolith.
- Mark only Workstream 4 `Complete`, and only after all stages and final - Mark only Workstream 5 `Complete`, and only after all stages and full
validation succeed. validation succeed. Workstream 6 remains the next pending workstream.
## Instructions For Every Stage ## Instructions For Every Stage
Before each stage: Before each stage:
1. Read `docs/development.md`, every file under `docs/policy/`, this plan, 1. Read `docs/development.md`, every document under `docs/policy/`, this plan,
Workstream 4 plus the dependency and target sections of Workstream 5 plus the dependency and consolidated-target sections of
[`refactoring.md`](refactoring.md), Finding 2 and the Workstream-4-relevant [`refactoring.md`](refactoring.md), and audit rows H04, H07, H14, H15, H17,
boundary in Finding 3 of [`contract-audit.md`](contract-audit.md), and rows H34, and H35 in [`audit-decisions.md`](audit-decisions.md). Consult the
H14, H15, H17, C02, and C03 of corresponding detailed findings in [`audit.md`](audit.md) when a boundary is
[`audit-decisions.md`](audit-decisions.md). unclear.
2. Inspect the named implementation and focused tests. Prefer the codebase 2. Inspect the named production code and focused tests. Prefer the codebase
graph for Go discovery and text search for schemas, paths, fixtures, knowledge graph for Go symbol discovery and call tracing; use text search for
documentation, and serialized field names. documentation, fixtures, serialized fields, and global-variable assignments.
3. Confirm the worktree state and preserve unrelated changes. Do not enter a 3. Confirm the worktree state and preserve unrelated changes. Do not enter
later workstream merely because adjacent lifecycle, analyze, publish, or Workstream 6 or Workstream 9 merely because analyze or cleanup code is
cleanup code would benefit from broader refactoring. adjacent to runner lifecycle code.
During every stage: During every stage:
- Keep the repository compiling and focused tests passing. A temporary - Keep the repository compiling and focused tests passing. Do not leave both an
invocation projection or unused run-manifest domain type may remain only old and new runner path active except for a temporary private delegation that
when the next numbered stage explicitly removes it; do not introduce a new the next numbered stage explicitly removes.
compatibility facade. - Preserve package ownership and acyclic dependency direction. Private state
- Preserve package ownership and acyclic dependency direction. Keep this a should aggregate already-related values, not become a service locator.
deletion-oriented refactor inside the existing `app`, `stage`, `manifest`, - Preserve error wrapping and joining where errors represent distinct failures.
and `artifacts` owners. Do not freeze incidental complete error strings in new tests.
- Preserve the session lock, session-manifest atomic replacement, group - Keep tests deterministic, offline, behavior-focused, and lean under
workspace modes, bounded I/O, no-follow publish sources, remote verification, `docs/policy/testing.md`. Prefer existing real temporary filesystems and
secret redaction, and Linux/macOS/Windows support. current adapter/storage fakes. Do not add mock choreography for private
- Update the canonical current-behavior document owned by a change in the same lifecycle helpers.
stage. Do not edit historical release notes or completed audit findings. - Update the canonical current-behavior document owned by a changed behavior in
- Keep tests offline, deterministic, and behavior-focused. Remove tests with the same stage. Pure file movement or private extraction requires no user
the retired contract and retain consequential session-state, data-integrity, documentation change.
recovery, and remote-commit coverage at the narrowest stable boundary. - Run `gofmt` on changed Go files, focused tests for the changed boundary, and
- Run `gofmt` on changed Go files, focused tests, and `git diff --check`. Mark a `git diff --check`. Mark a stage `Completed` only after its exit criteria pass.
stage `Completed` only after its exit criteria pass.
## Stage 1 — Decouple Publish From The Run Manifest ## Stage 1 — Unify Pre-Execution Decisions
**Status: Completed** **Status: Pending**
### Goal ### Goal
Make publish consume only the session manifest and current run directory so the Give planning and execution one pure representation of the decision made before
runner can stop producing a structured run manifest without losing useful a stage attempt, while keeping planning predictions distinct from observed
diagnostic uploads. runtime outcomes.
### Required Work ### Required Work
1. In `internal/stage/publish.go`, replace 1. In `internal/app/run_control.go` or a focused private file, replace the
`resolvePublishRunManifestPath` with canonical current-run-root resolution current action-only decision with an unexported value that records:
based on resolved campaign, session ID, workspace root, and `m.RunID` using `run` versus `skip`; whether the selected successful record must first be
`artifacts.SessionRunRootForCampaign`. Require that exact root to be a real marked stale; the stage-owned reason for that stale transition; and the
directory and retain existing symlink/non-directory rejection. application-owned invalidation reason, if any. Its constructor must be pure:
2. Remove opening, decoding, rewinding, and passing a run-manifest file. Change it receives the prior manifest status, `force`, and an already normalized
current-run collection to inspect `manifest.Manifest` stage records and add resume-validation result and does not mutate state, persist, log, read files,
only declared `ArchivePath`, fallback `LocalPath`, `Logs`, and invoke a stage, or obtain the current time.
`GeneratedConfigs` that resolve within the selected current run root. 2. Encode these cases explicitly:
Records referring to canonical session files or an older run must not be - a non-forced succeeded record with no validator result or a resumable
uploaded as current-run diagnostics. result skips;
3. Make an empty current-run diagnostic set valid. Refactor the source-opening - a non-forced succeeded record with a non-resumable result runs after the
helper so it opens each allowlisted current-run file directly, with the selected record is staled with the validator's reason and succeeded
existing regular-file/no-follow and optional checksum verification. It must dependents are invalidated with `staleReasonNotResumable`;
not require a distinguished manifest file. - a forced stage runs and invalidates succeeded dependents with
4. Remove the `manifest.json` synthetic run-file entry and the special branch `staleReasonForcedReplacement` when the attempt begins; and
that emits `RemoteArtifactTypeRunManifest`. Continue emitting ordinary - absent, failed, stale, or skipped aggregate records run without inventing a
current-run diagnostics as `RemoteArtifactTypeRunFile` with deterministic resume-stale transition.
`run:<relative-path>` source identities and destinations. 3. Add a small helper that applies only the identical pre-execution manifest
5. Preserve session-manifest commit generation, previous-cache uploads, changes described by the decision at a caller-supplied timestamp. Retain the
configured published outputs, immutable upload verification, lock existing fixed invalidation relation and stable application-owned reasons.
revalidation, commit upload, and pointer-last ordering unchanged. The caller remains responsible for persistence and error context.
6. Rewrite focused publish tests and helpers to create a run root and populate 4. Use the decision constructor and application helper from both `Plan` and
session-manifest diagnostic references instead of writing a run manifest. `executePlan`. Each caller may perform stage-owned resume validation before
Cover current-run logs/generated configuration/declared archive outputs, constructing the decision. Preserve analyze planning's read-only artifact
exclusion of prior-run and session-canonical paths, empty diagnostics, details when analyze is already a run candidate; those details are preview
symlink rejection, deterministic ordering, and absence of newly emitted information, not an execution outcome.
`run_manifest` declarations. Retain existing upload integrity and pointer 5. Extract only the exact shared attempt/result mutations already identical in
tests without duplicating them. both paths: beginning an attempt (`running` plus forced invalidation),
7. Update `docs/internal/stage-publish.md` and the run-upload portion of applying a configured/runtime self-skip and its conditional invalidation,
`docs/operations.md`: the session manifest is the diagnostic allowlist, no and applying a predicted/actual success and its conditional invalidation.
run manifest is uploaded, and `upload_run` may legitimately upload zero Require callers to supply the actual or modeled outcome, outputs, semantic
run-local diagnostic files. fingerprint, prior outcome, and timestamp. Do not put `Stage.Run`, result
validation, logging, or manifest saves in these helpers.
6. Rewrite `modelPlannedStageRun` around these values or delete it if the
resulting focused helpers make it redundant. Keep all planning mutation on
the cloned manifest.
7. Add table-driven decision tests for every status/force/resume combination
and focused parity tests proving that plan and execution produce the same
run/skip choice and dependent-stale effects for ordinary skip, non-resumable
success, forced replacement, unchanged self-skip, changed self-skip, and a
successful result replacing a non-success record. Assert durable state and
decision semantics, not private helper call order or exact timestamps.
### Exit Criteria ### Exit Criteria
- `go test ./internal/stage ./internal/artifacts` succeeds. - `go test ./internal/app` succeeds.
- Publish succeeds when the current run directory exists but contains no run - `Plan` and `executePlan` no longer independently encode the same
manifest and no diagnostics. pre-execution stale/force decision sequence.
- Only session-manifest-declared files confined to the current run root become - The decision constructor is pure, and actual stage outcomes and persistence
`run_file` transfers; older-run and incidental files do not. remain outside it.
- New commits contain no `run_manifest` artifact, while remote decoding still - Planning remains read-only and execution behavior is unchanged.
accepts that historical wire value.
- `git diff --check` succeeds. - `git diff --check` succeeds.
## Stage 2 — Make Runner Persistence Session-Only ## Stage 2 — Establish The Invocation Runtime And Lock Lifecycle
**Status: Completed** **Status: Pending**
### Goal ### Goal
Remove the second persistence transaction from invocation execution while Make invocation state cohesive and make the top-level transaction order obvious
preserving the existing session-ledger lifecycle and immediate command result. without hiding session-lock ownership.
### Required Work ### Required Work
1. Remove `RunOptions.RunManifestStore` and `RunSummary.RunManifestPath`. Keep 1. Introduce an unexported `invocationRuntime` (or equivalently narrow private
all remaining summary fields and run ID generation. Update direct callers type) under `internal/app`. It should carry only state already shared across
and CLI rendering only if they refer to the removed path. lifecycle phases: context, resolved configuration, bounded plan/stages,
2. In `executePlan`, delete run-manifest path resolution, construction, normalized options, effective artifacts, run identity and canonical paths,
identity/config projection, initial save, per-stage action/running/skip/ `stage.Env`, manifest store, artifact store, loaded session manifest, and
failure/success saves, final success save, and every error path created only summary accumulators. Do not expose it or let stages depend on it.
by those operations. 2. Extract unlocked preparation that performs the current read-only bounded
3. After acquiring the session lock, validating/restoring session state, and prerequisite inspection, resolves effective artifacts, generates the run
resolving invocation identity, explicitly ensure the canonical ID, resolves invocation identity, initializes/clones `stage.Env` fields, and
`SessionRunRootForCampaign` directory with the ordinary workspace directory supplies default manifest/artifact stores and logger. Preserve the rule that
contract. Do not create `manifest.json` or another placeholder there. injected `Env` values provide collaborators but never an alternate config,
4. Replace `persistTerminalFailure` with a narrowly named session-only helper. selected-artifact set, or effective artifact set.
It must record the terminal failure in the session manifest when available, 3. Extract layout preparation separately. Keep `EnsureLayoutFor` before lock
atomically attempt the session save, preserve the originating error, and acquisition as today, verify its manifest path against the canonical
join a session-save error without invoking any diagnostic store. Keep the resolved path, and return enough state for the locked lifecycle.
distinct post-publish-cleanup failure behavior until its later workstream. 4. In the short top-level executor, acquire the session lock after unlocked
5. Simplify all runner failure paths to the new helper. Preserve the existing preparation and visibly install one deferred release beside that acquisition.
order of session identity save, resume validation, running transition, Preserve exact release-error joining behavior. Do not put lock ownership in
force invalidation, stage execution, success/failure/self-skip transition, `invocationRuntime`, a constructor, or a helper whose lifetime is ambiguous.
session save, logging, and post-publish cleanup. 5. Extract locked initialization that checks the incomplete-restore marker,
6. Make `applyAnalyzeProjection` session-only at its application boundary so loads and validates or creates the session manifest, rechecks bounded
the runner no longer needs a run-manifest value. The invocation subset may prerequisites, applies identity and effective-configuration provenance,
remain temporarily inside validation until Stage 3 deletes it. saves the identity transition, ensures the canonical run directory, and
7. Delete run-only helpers such as `invocationIdentity.applyToRunManifest`, loads configured secrets. All steps after manifest availability that
`requestedStageNames`, `applyStageResultToRunManifest`, currently record a terminal failure must continue to do so through the
`applyEffectiveConfigProvenanceToRun`, and session-only failure helper.
`setRunStageSemanticConfig` once unused. 6. Keep pre-lock failures free of persistent mutation and keep all later
8. Rewrite runner, terminal-failure, semantic-resume, assembled-workflow, and manifest transitions protected by the same held lock. Do not add rollback,
lifecycle tests that currently inject a `RunStore` or load a second lock, or a second manifest authority.
`RunSummary.RunManifestPath`. Assert the durable outcome through the session 7. Adapt existing bounded-run, restore-marker, identity, run-directory,
manifest and the immediate invocation outcome through `RunSummary`. terminal-failure, and lock-release tests to exercise the assembled lifecycle.
Preserve coverage for cancellation, collaborator/preflight failures, Add coverage only if the two-check prerequisite ordering or lock-release
session-save failure joining, successful and failed stage transitions, error behavior is not already credibly protected.
self-skip, force invalidation, semantic evidence, distinct run IDs, and
run-local diagnostic paths. Delete tests whose sole purpose was run-record ### Exit Criteria
creation, terminalization, or historical inspection.
9. Update `docs/policy/architecture.md`, `docs/cli.md`, and the runner portions - `go test ./internal/app ./internal/artifacts ./internal/manifest` succeeds.
of `docs/internal/manifest.md`: one atomic session-manifest update is the - The top-level executor visibly performs unlocked preparation, acquires and
only durable progress transaction; a run ID and run directory remain, but defers release of exactly one session lock, initializes locked state, and
there is no invocation manifest or persistent command summary. proceeds to execution.
- Invalid preflight requests create no persistent invocation state; the locked
prerequisite recheck still prevents races with a prior invocation.
- The private runtime has cohesive fields and no public or cross-package API.
- `git diff --check` succeeds.
## Stage 3 — Isolate Collaborator Composition
**Status: Pending**
### Goal
Make production collaborator construction one bounded lifecycle phase that is
selective, testable per invocation, and independent of stage transitions.
### Required Work
1. Define a small unexported invocation dependency value with explicit defaults
for the nondeterministic and external construction boundaries used by this
runner: UTC time, run-ID generation, default adapter/object-store creation,
effective remote-lock loading/revalidation support, and post-publish scoped
removal. Include a field only where tests need control over a real external,
destructive, clock, or randomness boundary; continue injecting completed
stage collaborators through `stage.Env`.
2. Add a private executor variant that accepts that dependency value and have
the ordinary executor supply production defaults. Normalize missing fields
once at the entry boundary so lifecycle methods never fall back to mutable
package state. Do not export the dependency type or store it globally.
3. Route run-ID creation and every runner timestamp through the invocation
dependencies. Convert `nowUTC` to an ordinary function for unrelated
command owners if it no longer needs to be mutable; do not broaden this stage
into a restore lifecycle refactor.
4. Extract collaborator composition from the runner into one operation over the
invocation runtime. Preserve current selection gates exactly: WhisperX only
for `transcribe`; Seriatim only for `merge`, `normalize`, `trim`, or `render`;
Audita only for `polish`; Notarius only when selected extraction requires it;
Scriptorium only for `trim` or `analyze`; object storage only when selected
stages/effective artifacts require it; remote lock loading only when needed;
and the no-op notifier only for `notify`.
5. Preserve injected collaborators, secret-loaded environment behavior,
effective lock merging, and the publish-time revalidation closure. A
composition failure after identity initialization remains a persisted
terminal session failure with its existing useful context.
6. Route post-publish cleanup's scoped deletion through the invocation
dependency value and delete `removeRunScopedDirFn`. Preserve cleanup
obligation creation, per-target persistence, retry, and failure behavior.
7. Consolidate selective-composition tests into table-driven or focused
behavioral cases. Test that an unselected external boundary is not
constructed only where avoiding that construction is an operational
requirement; do not assert private helper ordering.
### Exit Criteria
- `go test ./internal/app` succeeds, including composition, remote-lock,
cleanup, and bounded-stage tests.
- Collaborator construction is absent from the top-level runner and stage loop,
but remains an application-owned production-composition concern.
- Runner time, randomness, external construction, and destructive cleanup are
controllable per invocation without mutable runner globals.
- Selected-stage composition and terminal-failure behavior remain unchanged.
- `git diff --check` succeeds.
## Stage 4 — Extract One-Stage Execution
**Status: Pending**
### Goal
Give one private operation complete ownership of a single stage attempt while
preserving every durable transition and failure boundary.
### Required Work
1. Extract an operation on the invocation runtime that executes exactly one
selected stage and returns a small execution-owned outcome sufficient to
update the in-memory summary or stop the invocation. It may coordinate the
shared decision/transition helpers but must not select the next stage or
release the session lock.
2. Within that operation, preserve this order:
- compute the stage semantic fingerprint;
- run stage-owned resume validation when the recorded success is a skip
candidate;
- build and apply the pure pre-execution decision and persist any
non-resumable stale/invalidation transition;
- return a skip outcome without invoking the stage when still resumable;
- capture prior aggregate/analyze state, begin the attempt, persist
`running`, and then invoke `Stage.Run`;
- validate the stage result and any analyze projection;
- apply failure, self-skip, or success transitions and required dependent
invalidation; and
- persist the terminal state before returning.
3. Preserve failure semantics for semantic fingerprinting, resume validation,
invalidation, the pre-run save, stage execution, result validation,
self-skip/success saves, and terminal failure persistence. A failed stage or
persistence operation stops later stages. Distinct invalidation/save errors
remain joined with the originating failure.
4. Preserve current analyze behavior exactly: a valid partial failure
projection may be applied before recording failure; a successful projection
supplies aggregate outputs; and failure to save a successful projection
restores prior analyze authority before recording aggregate failure.
Introduce no callback/store dependency into `internal/stage`.
5. Keep manifest save calls explicit in the one-stage owner. Do not hide them in
generic state-machine hooks or teach `manifest.Store` orchestration policy.
6. Reduce the outer stage loop to call the one-stage operation, append its
executed/skipped classification, and stop on error. Preserve deterministic
`StageNames`, `Executed`, and `Skipped` order and current logging semantics.
7. Reconcile runner tests around observable manifest state, invocation summary,
stage calls, and errors. Preserve coverage for ordinary skip, force,
non-resumable state, self-skip, result validation, cancellation, downstream
invalidation outside a bounded range, save failures, analyze rollback, and
stopping after failure. Remove redundant tests that exist only to pin the old
monolithic control flow.
### Exit Criteria ### Exit Criteria
- `go test ./internal/app ./internal/stage ./internal/manifest` succeeds. - `go test ./internal/app ./internal/stage ./internal/manifest` succeeds.
- `executePlan` creates no `runs/<run_id>/manifest.json` and exposes no run - The stage loop contains no duplicated transition policy and one-stage
manifest store seam, while run IDs, run directories, diagnostics, session execution has one clear persistence owner.
transitions, and summaries remain available. - A later stage is never attempted after a stage or required manifest save
- A hypothetical diagnostic-record write cannot fail a stage because no such fails, and no failed/interrupted attempt is presented as successful.
persistence operation exists; session-manifest failures remain visible and - Analyze still uses its pre-Workstream-6 aggregate persistence boundary.
authoritative.
- Runner behavior tests obtain durable state only from the session manifest.
- `git diff --check` succeeds. - `git diff --check` succeeds.
## Stage 3 — Remove Invocation-Only Analyze Projection ## Stage 5 — Isolate Finalization And Simplify The Lifecycle Owner
**Status: Completed** **Status: Pending**
### Goal ### Goal
Give analyze one in-memory projection—the reconciled session collection—without Complete the runner decomposition so the lifecycle and transaction boundaries
prematurely implementing Workstream 6 checkpoint semantics. are easier to read than the former monolith.
### Required Work ### Required Work
1. Remove `Invocation` from `stage.AnalyzeStateProjection` and update its 1. Extract finalization that performs the current post-publish cleanup only
comment to describe only the reconciled session authority proposed for the when publish actually executed, using the runtime's session manifest,
next session-manifest save. Retain the pointer wrapper so “no analyze update” manifest path, environment, executed-stage list, and invocation-scoped
remains distinguishable from a valid empty collection. deletion dependency. Preserve the special cleanup-failure persistence and
2. In `internal/stage/analyze.go`, delete the separate invocation-record map and the fact that a committed remote publish remains authoritative.
every write, clone, or failure projection associated only with it. Successful 2. Build `RunSummary` from the runtime's resolved identity and ordered
and failed analyze results continue to return the complete reconciled accumulators only after all selected stages and required cleanup complete.
session collection under the existing rules. Continue returning no success summary on invocation failure.
3. Simplify `internal/app/analyze_projection.go`: validate and clone only the 3. Make the top-level executor an explicit sequence: unlocked preparation;
session collection, remove invocation/session subset comparison, apply only layout preparation; visible lock acquisition/deferred release; locked state
the session collection, and derive aggregate analyze outputs only from that initialization; collaborator composition; ordered one-stage execution; and
collection. Retain rejection of projections from non-analyze stages, finalization/summary. It must retain meaningful error context at each phase
contradictory ordinary outputs, malformed records, and illegal skipped or and must not delegate that entire sequence to a new monolithic helper.
error result shapes. 4. Place private lifecycle concerns in focused files according to ownership—for
4. Preserve `captureAnalyzeState` and `restoreAnalyzeState` for the current example runtime/preparation, composition, stage execution, and finalization—
single-save boundary. Do not add manifest-store access to the analyze stage, while keeping exact shared transition logic in `run_control.go`. Remove dead
persist inside its artifact loop, or change retry/force behavior in this helpers, stale comments, duplicate state, and imports left by extraction.
stage. 5. Review the resulting runtime for service-locator growth. Remove fields used
5. Rewrite `analyze_projection_test.go`, analyze incremental tests, and mixed by only one local operation unless retaining them materially clarifies the
runner tests around one reconciled session collection. Delete assertions lifecycle. Keep collaborators in `stage.Env` and persistence/path policy in
about per-invocation artifact subsets. Retain behavior coverage for partial their existing owners.
failure state, unrelated current records, invalid projections, persistence 6. Run focused assembled-workflow, cleanup, bounded-run, resume, manifest-save,
failure rollback, and aggregate output derivation. and release-error tests. Add no tests for private file placement, helper
6. Update `docs/internal/stage-analyze.md` and the analyze-state section of count, or line count.
`docs/internal/manifest.md` to describe one session projection and clearly
leave per-artifact checkpointing as future roadmap work rather than current
behavior.
### Exit Criteria ### Exit Criteria
- `go test ./internal/stage ./internal/app ./internal/manifest` succeeds. - `go test ./internal/app ./internal/stage ./internal/manifest ./internal/artifacts`
- No production analyze type, result, validator, or test distinguishes session
state from an invocation artifact subset.
- Analyze still performs one end-of-stage session save, retains valid unrelated
records, persists its permitted failure projection, and restores prior
in-memory authority if that save fails.
- No Workstream 6 checkpoint callback or persistence loop has been introduced.
- `git diff --check` succeeds.
## Stage 4 — Delete The Run-Manifest Domain Model
**Status: Completed**
### Goal
Remove the now-unreferenced structured schema, persistence API, canonical file
path, and mechanism tests from their owning packages.
### Required Work
1. Delete `internal/manifest/run_manifest.go`, `RunManifestSchemaVersion`,
`RunManifest`, `RunStageRecord`, run status/action enums, constructors,
transition methods, and all run-only validation/normalization helpers.
2. Remove `manifest.RunStore` and `LocalStore.CreateRun`, `LoadRun`,
`LoadRunReader`, and `SaveRun`. Simplify imports and shared validation only
where the deleted run code made them unnecessary; do not weaken session
manifest validation.
3. Delete `artifacts.SessionRunManifestPathForCampaign` while retaining
`SessionRunsDirForCampaign` and `SessionRunRootForCampaign` as canonical
run-diagnostic and invocation-work paths.
4. Delete `run_manifest_test.go` and run-manifest-only store/path/schema tests.
Remove run-specific portions of otherwise valuable manifest, artifact,
semantic fingerprint, restore, operator, and publish fixtures. Do not replace
serialization round-trip tests for a format that no longer exists.
5. Keep the remote `run_manifest` artifact enum accepted only by remote commit
decoding/validation. Add a concise comment at its declaration explaining
that Narratio no longer emits or interprets it and retains the value solely
so immutable existing commits remain readable. No production symbol in
`manifest`, `app`, or `stage` may refer to it.
6. Update package comments or focused internal documentation affected by the
domain deletion in the same change.
### Exit Criteria
- `go test ./internal/manifest ./internal/artifacts ./internal/app ./internal/stage`
succeeds. succeeds.
- Except for the explicitly retained `RemoteArtifactTypeRunManifest` wire - `executePlan` is a short, readable lifecycle owner with one visibly scoped
value and its remote-decoder compatibility test, searches for `RunManifest`, lock and no nested stage state machine or collaborator-construction block.
`RunStore`, `CreateRun`, `LoadRun`, `SaveRun`, `RunManifestPath`, - Cleanup remains a post-publish obligation with unchanged behavior, awaiting
`run_manifest.go`, and the local `runs/<run_id>/manifest.json` contract find Workstream 9.
no current production or test implementation. - The refactor has removed obsolete code rather than retaining parallel old and
- Session manifest schema validation and round trips remain fully covered. new lifecycle paths.
- Existing remote commit documents containing the historical `run_manifest`
artifact type still decode and validate, but new publish plans cannot emit
it.
- `git diff --check` succeeds. - `git diff --check` succeeds.
## Stage 5 — Reconcile Current Documentation And Close Workstream 4 ## Stage 6 — Remove Global Runner Seams And Close Workstream 5
**Status: Completed** **Status: Pending**
### Goal ### Goal
Prove the session-only target state across the repository and leave one accurate Prove the target architecture through invocation-scoped command tests, accurate
implementation narrative for the next workstream. current documentation, and complete repository validation.
### Required Work ### Required Work
1. Search current production code, tests, fixtures, examples, scripts, and 1. Introduce the smallest private command-level helper needed to pass a
nonhistorical documentation for run-manifest types, paths, JSON fields, pipeline-executor function for one call through `Execute`, `Run`,
stores, publish classifications, invocation projections, and dual-save or `RegenerateArtifacts`, and the single-stage command path. Public command
terminal-synchronization language. Remove stale surfaces or explain the functions continue to use the production executor. The alias must still be
retained run ID/run directory without implying a structured record. exactly equivalent to `run --force --from extract --through analyze` with
2. Reconcile canonical owners, especially `docs/policy/architecture.md`, caller options forwarded under the existing rules.
`docs/cli.md`, `docs/operations.md`, `docs/troubleshooting.md`, 2. Delete `executeStagesFn` and `runCommandFn`. Rewrite affected CLI, profile,
`docs/internal/overview.md`, `docs/internal/manifest.md`, remote-session, restore-workflow, assembled-workflow, bounded-run, alias, and
`docs/internal/stage-analyze.md`, `docs/internal/stage-extract.md`, and artifact-selection tests to call the private command helper with an explicit
`docs/internal/stage-publish.md`. Keep schema definitions in the manifest executor or to exercise the real assembled runner. Do not serialize tests
owner, filesystem/remote behavior in operations, and implementation details around package-global save/replace/restore choreography.
in internal documents rather than duplicating them. 3. Confirm that `removeRunScopedDirFn` is gone and that runner tests do not
3. Do not edit immutable historical release notes or completed audit documents. replace `nowUTC`. Inventory remaining mutable function variables in
Roadmap references may retain the phrase “run manifest” when describing the `internal/app`; leave unrelated configuration-loading and restore seams for
problem and approved removal. Current instructions must not claim the file their owning future work, but ensure no runner, run-command, alias, or
is produced or required. post-publish-cleanup test mutates process-global behavior.
4. Review the affected test suite under `docs/policy/testing.md`. Consolidate 4. Review test value under `docs/policy/testing.md`. Consolidate duplicate
leftover tests that merely encode the deleted write choreography, while command-wiring and lifecycle cases exposed by the new narrow seams while
retaining meaningful session durability, transition, recovery, current-run retaining consequential lock, persistence, invalidation, recovery,
diagnostics, remote publication, and representative assembled-workflow cancellation, external-boundary, and representative assembled-workflow
coverage. coverage.
5. Update Workstream 4 in `refactoring.md` from `Pending` to `Complete` and 5. Update current internal documentation where ownership requires it:
update every stage plus the purpose/status section of this plan to `docs/internal/overview.md` should summarize the explicit runner lifecycle;
`Completed` only after the target state and all checks are proven. Do not `docs/internal/manifest.md` should describe the shared pre-execution decision
change later workstream statuses. and single persistence owner without documenting private helper names; and
6. Run the complete repository validation from `docs/development.md`: `docs/internal/adapters.md` should describe selective application-owned
composition. Update architecture policy only if implementation reveals a
normative boundary not already stated. Do not duplicate CLI/configuration
contracts or edit historical release documents and completed audit findings.
6. Search current code and documentation for stale claims about the monolithic
runner, independent plan/run decisions, or mutable runner seams. Remove only
inaccurate current-behavior text; roadmap and audit history remain valid.
7. Update Workstream 5 in `refactoring.md` from `Pending` to `Complete`, and
change all six stage statuses plus this plan's status to completed only after
the target state and all checks are proven. Leave Workstream 6 `Pending`.
8. Run the complete repository validation from `docs/development.md`:
`go test ./...`, `go test -race ./...`, `go vet ./...`, `go build ./...`, `go test ./...`, `go test -race ./...`, `go vet ./...`, `go build ./...`,
`go test ./internal/doccheck`, and `go test ./internal/doccheck`, and
`go test ./internal/config -run '^TestExamplesLoadAndValidate$'`. Run `go test ./internal/config -run '^TestExamplesLoadAndValidate$'`. Also run
`git diff --check` and confirm the worktree contains only intended changes. `git diff --check` and confirm the implementation commit leaves no unintended
worktree changes.
### Exit Criteria ### Exit Criteria
- The session manifest is the only structured durable local progress record; - All repository-wide validation succeeds on Go 1.26.
no current code or documentation requires a run manifest. - Planning and execution share one pure pre-execution decision model while
- Run IDs, canonical run directories, bounded diagnostics, generated actual results and persistence remain execution-owned.
configuration, receipts, immediate summaries, immutable run prefixes, and - The runner reads as an explicit lifecycle with one session lock and one
remote publication remain functional without a second progress transaction. session-manifest authority.
- Publish uses session-manifest evidence, emits no new run-manifest artifact, - Runner, run-command, alias, and cleanup tests use invocation-scoped
and still reads verified historical remote commits without interpreting their dependencies rather than mutable package-global function substitution.
old diagnostic record. - Current internal documentation accurately describes the implemented design
- All repository-wide validation succeeds on the maintained Go toolchain. at the appropriate level.
- Workstream 4 and all five stages are marked `Complete`; Workstream 5 remains - Workstream 5 and all six stages are marked `Complete`; Workstream 6 is the
the next pending workstream. next pending workstream.
- `git status --short` is empty after the implementation commit. - `git status --short` is empty after the implementation commit.
## Open Questions ## Open Questions
None. The approved audits, current architecture, and decisions above are None. The approved audits, current policies, and Workstream 5 decisions resolve
sufficient to implement this workstream without further product or policy the behavior, architecture, testability, and sequencing choices required for
choices. implementation.