narratio/docs/roadmap/implementation.md

# Workstream 4 Implementation Plan

## Purpose And Status

This is the executable plan for [`refactoring.md`](refactoring.md), Workstream
4: **One durable local manifest authority**. Finding 2 of
[`contract-audit.md`](contract-audit.md) and rows H14, H15, H17, C02, and C03
of [`audit-decisions.md`](audit-decisions.md) own the approved decisions.

All five stages are pending and must be implemented in order. Each stage is
bounded for one `gpt-5.6-terra` prompt. Workstreams 1 through 3 are complete.
Workstreams 5 and later are out of scope: do not decompose the runner into a
new lifecycle type, redesign pre-execution decisions, add per-artifact analyze
checkpoints, split publish into separate transactions, add cleanup as a stage,
or change remote lock policy while following this plan.

## Settled Implementation Decisions

- The session manifest is the only durable local progress ledger. It alone
  records stage status, current artifacts, semantic evidence, diagnostics,
  failures, resume authority, restore authority, and publish authority.
- Delete the structured invocation run manifest completely: its schema and
  types, store interface and methods, path helper, lifecycle mutations,
  terminal recovery, analyze projection, publish dependency, fixtures, tests,
  and current documentation all leave the repository.
- Do not replace it with another JSON summary, journal, database, or event log.
  Historical per-invocation state is not an operational requirement.
- Preserve run IDs. Every invocation still receives one before session state is
  initialized; it remains the identity used by the session manifest,
  run-local directory, adapter provenance, logs, generated configuration,
  receipts, immutable publish prefix, command summary, and diagnostics.
- Explicitly create the canonical run directory once per invocation after the
  session lock and identity checks. Removing the initial run-manifest save must
  not make diagnostic writers responsible for accidentally creating the run
  root. Directory-creation failure remains an invocation initialization error.
- `RunSummary` remains the immediate in-memory command result and retains
  session ID, run ID, session manifest path, selected stage names, executed
  stages, and skipped stages. Remove only `RunManifestPath`; do not add a
  replacement persistent summary.
- A run-manifest write can no longer fail an otherwise valid pipeline action.
  Session-manifest transition and terminal-save failures remain correctness
  failures and continue to join with the originating error where applicable.
- Publish no longer opens or validates `runs/<run_id>/manifest.json`. When
  `upload_run` is enabled, it derives the current run root canonically and uses
  the session manifest's recorded output, log, and generated-configuration
  references as an allowlist. It includes only safe regular files confined to
  the current run root and ignores records from earlier runs.
- New remote commits do not upload or classify a run manifest. Keep
  `RemoteArtifactTypeRunManifest` as a read-only wire value so already committed
  and verified remote snapshots remain readable; it must have no local loader,
  path, special publish branch, or newly emitted declaration. This preserves a
  durable remote protocol without retaining the local dual-authority model.
- Preserve `RemoteArtifactTypeRunFile` for declared run-local diagnostics.
  Publishing no current-run diagnostics is valid and must not require a
  placeholder file.
- Remove the invocation subset from `AnalyzeStateProjection` and from analyze
  execution. During this workstream, analyze still returns and persists one
  validated reconciled session collection at the existing end-of-stage
  boundary. Do not implement artifact-by-artifact persistence yet.
- Preserve the current safe behavior when the end-of-stage analyze manifest
  save fails: incidental output bytes do not establish authority, prior
  in-memory analyze authority is restored, and the aggregate stage fails.
  Workstream 6 will replace this with the approved checkpoint model.
- Preserve current stage ordering, force/invalidation rules, self-skip behavior,
  aggregate transition timing, cleanup obligations, remote commit ordering,
  and command output except where they directly expose the removed run
  manifest.
- Delete tests whose only contract is historical run-manifest persistence.
  Rewrite mixed tests around session state, command summaries, run-local
  diagnostics, or remote commit contents, and consolidate redundant mechanism
  assertions rather than recreating a second audit record in tests.
- Historical release notes and completed audit documents remain unchanged.
  Current architecture, CLI, operations, troubleshooting, and internal
  documentation must describe session-only authority.
- Mark only Workstream 4 `Complete`, and only after all stages and final
  validation succeed.

## Instructions For Every Stage

Before each stage:

1. Read `docs/development.md`, every file under `docs/policy/`, this plan,
   Workstream 4 plus the dependency and target sections of
   [`refactoring.md`](refactoring.md), Finding 2 and the Workstream-4-relevant
   boundary in Finding 3 of [`contract-audit.md`](contract-audit.md), and rows
   H14, H15, H17, C02, and C03 of
   [`audit-decisions.md`](audit-decisions.md).
2. Inspect the named implementation and focused tests. Prefer the codebase
   graph for Go discovery and text search for schemas, paths, fixtures,
   documentation, and serialized field names.
3. Confirm the worktree state and preserve unrelated changes. Do not enter a
   later workstream merely because adjacent lifecycle, analyze, publish, or
   cleanup code would benefit from broader refactoring.

During every stage:

- Keep the repository compiling and focused tests passing. A temporary
  invocation projection or unused run-manifest domain type may remain only
  when the next numbered stage explicitly removes it; do not introduce a new
  compatibility facade.
- Preserve package ownership and acyclic dependency direction. Keep this a
  deletion-oriented refactor inside the existing `app`, `stage`, `manifest`,
  and `artifacts` owners.
- Preserve the session lock, session-manifest atomic replacement, group
  workspace modes, bounded I/O, no-follow publish sources, remote verification,
  secret redaction, and Linux/macOS/Windows support.
- Update the canonical current-behavior document owned by a change in the same
  stage. Do not edit historical release notes or completed audit findings.
- Keep tests offline, deterministic, and behavior-focused. Remove tests with
  the retired contract and retain consequential session-state, data-integrity,
  recovery, and remote-commit coverage at the narrowest stable boundary.
- Run `gofmt` on changed Go files, focused tests, and `git diff --check`. Mark a
  stage `Completed` only after its exit criteria pass.

## Stage 1 — Decouple Publish From The Run Manifest

**Status: Completed**

### Goal

Make publish consume only the session manifest and current run directory so the
runner can stop producing a structured run manifest without losing useful
diagnostic uploads.

### Required Work

1. In `internal/stage/publish.go`, replace
   `resolvePublishRunManifestPath` with canonical current-run-root resolution
   based on resolved campaign, session ID, workspace root, and `m.RunID` using
   `artifacts.SessionRunRootForCampaign`. Require that exact root to be a real
   directory and retain existing symlink/non-directory rejection.
2. Remove opening, decoding, rewinding, and passing a run-manifest file. Change
   current-run collection to inspect `manifest.Manifest` stage records and add
   only declared `ArchivePath`, fallback `LocalPath`, `Logs`, and
   `GeneratedConfigs` that resolve within the selected current run root.
   Records referring to canonical session files or an older run must not be
   uploaded as current-run diagnostics.
3. Make an empty current-run diagnostic set valid. Refactor the source-opening
   helper so it opens each allowlisted current-run file directly, with the
   existing regular-file/no-follow and optional checksum verification. It must
   not require a distinguished manifest file.
4. Remove the `manifest.json` synthetic run-file entry and the special branch
   that emits `RemoteArtifactTypeRunManifest`. Continue emitting ordinary
   current-run diagnostics as `RemoteArtifactTypeRunFile` with deterministic
   `run:<relative-path>` source identities and destinations.
5. Preserve session-manifest commit generation, previous-cache uploads,
   configured published outputs, immutable upload verification, lock
   revalidation, commit upload, and pointer-last ordering unchanged.
6. Rewrite focused publish tests and helpers to create a run root and populate
   session-manifest diagnostic references instead of writing a run manifest.
   Cover current-run logs/generated configuration/declared archive outputs,
   exclusion of prior-run and session-canonical paths, empty diagnostics,
   symlink rejection, deterministic ordering, and absence of newly emitted
   `run_manifest` declarations. Retain existing upload integrity and pointer
   tests without duplicating them.
7. Update `docs/internal/stage-publish.md` and the run-upload portion of
   `docs/operations.md`: the session manifest is the diagnostic allowlist, no
   run manifest is uploaded, and `upload_run` may legitimately upload zero
   run-local diagnostic files.

### Exit Criteria

- `go test ./internal/stage ./internal/artifacts` succeeds.
- Publish succeeds when the current run directory exists but contains no run
  manifest and no diagnostics.
- Only session-manifest-declared files confined to the current run root become
  `run_file` transfers; older-run and incidental files do not.
- New commits contain no `run_manifest` artifact, while remote decoding still
  accepts that historical wire value.
- `git diff --check` succeeds.

## Stage 2 — Make Runner Persistence Session-Only

**Status: Completed**

### Goal

Remove the second persistence transaction from invocation execution while
preserving the existing session-ledger lifecycle and immediate command result.

### Required Work

1. Remove `RunOptions.RunManifestStore` and `RunSummary.RunManifestPath`. Keep
   all remaining summary fields and run ID generation. Update direct callers
   and CLI rendering only if they refer to the removed path.
2. In `executePlan`, delete run-manifest path resolution, construction,
   identity/config projection, initial save, per-stage action/running/skip/
   failure/success saves, final success save, and every error path created only
   by those operations.
3. After acquiring the session lock, validating/restoring session state, and
   resolving invocation identity, explicitly ensure the canonical
   `SessionRunRootForCampaign` directory with the ordinary workspace directory
   contract. Do not create `manifest.json` or another placeholder there.
4. Replace `persistTerminalFailure` with a narrowly named session-only helper.
   It must record the terminal failure in the session manifest when available,
   atomically attempt the session save, preserve the originating error, and
   join a session-save error without invoking any diagnostic store. Keep the
   distinct post-publish-cleanup failure behavior until its later workstream.
5. Simplify all runner failure paths to the new helper. Preserve the existing
   order of session identity save, resume validation, running transition,
   force invalidation, stage execution, success/failure/self-skip transition,
   session save, logging, and post-publish cleanup.
6. Make `applyAnalyzeProjection` session-only at its application boundary so
   the runner no longer needs a run-manifest value. The invocation subset may
   remain temporarily inside validation until Stage 3 deletes it.
7. Delete run-only helpers such as `invocationIdentity.applyToRunManifest`,
   `requestedStageNames`, `applyStageResultToRunManifest`,
   `applyEffectiveConfigProvenanceToRun`, and
   `setRunStageSemanticConfig` once unused.
8. Rewrite runner, terminal-failure, semantic-resume, assembled-workflow, and
   lifecycle tests that currently inject a `RunStore` or load
   `RunSummary.RunManifestPath`. Assert the durable outcome through the session
   manifest and the immediate invocation outcome through `RunSummary`.
   Preserve coverage for cancellation, collaborator/preflight failures,
   session-save failure joining, successful and failed stage transitions,
   self-skip, force invalidation, semantic evidence, distinct run IDs, and
   run-local diagnostic paths. Delete tests whose sole purpose was run-record
   creation, terminalization, or historical inspection.
9. Update `docs/policy/architecture.md`, `docs/cli.md`, and the runner portions
   of `docs/internal/manifest.md`: one atomic session-manifest update is the
   only durable progress transaction; a run ID and run directory remain, but
   there is no invocation manifest or persistent command summary.

### Exit Criteria

- `go test ./internal/app ./internal/stage ./internal/manifest` succeeds.
- `executePlan` creates no `runs/<run_id>/manifest.json` and exposes no run
  manifest store seam, while run IDs, run directories, diagnostics, session
  transitions, and summaries remain available.
- A hypothetical diagnostic-record write cannot fail a stage because no such
  persistence operation exists; session-manifest failures remain visible and
  authoritative.
- Runner behavior tests obtain durable state only from the session manifest.
- `git diff --check` succeeds.

## Stage 3 — Remove Invocation-Only Analyze Projection

**Status: Pending**

### Goal

Give analyze one in-memory projection—the reconciled session collection—without
prematurely implementing Workstream 6 checkpoint semantics.

### Required Work

1. Remove `Invocation` from `stage.AnalyzeStateProjection` and update its
   comment to describe only the reconciled session authority proposed for the
   next session-manifest save. Retain the pointer wrapper so “no analyze update”
   remains distinguishable from a valid empty collection.
2. In `internal/stage/analyze.go`, delete the separate invocation-record map and
   every write, clone, or failure projection associated only with it. Successful
   and failed analyze results continue to return the complete reconciled
   session collection under the existing rules.
3. Simplify `internal/app/analyze_projection.go`: validate and clone only the
   session collection, remove invocation/session subset comparison, apply only
   the session collection, and derive aggregate analyze outputs only from that
   collection. Retain rejection of projections from non-analyze stages,
   contradictory ordinary outputs, malformed records, and illegal skipped or
   error result shapes.
4. Preserve `captureAnalyzeState` and `restoreAnalyzeState` for the current
   single-save boundary. Do not add manifest-store access to the analyze stage,
   persist inside its artifact loop, or change retry/force behavior in this
   stage.
5. Rewrite `analyze_projection_test.go`, analyze incremental tests, and mixed
   runner tests around one reconciled session collection. Delete assertions
   about per-invocation artifact subsets. Retain behavior coverage for partial
   failure state, unrelated current records, invalid projections, persistence
   failure rollback, and aggregate output derivation.
6. Update `docs/internal/stage-analyze.md` and the analyze-state section of
   `docs/internal/manifest.md` to describe one session projection and clearly
   leave per-artifact checkpointing as future roadmap work rather than current
   behavior.

### Exit Criteria

- `go test ./internal/stage ./internal/app ./internal/manifest` succeeds.
- No production analyze type, result, validator, or test distinguishes session
  state from an invocation artifact subset.
- Analyze still performs one end-of-stage session save, retains valid unrelated
  records, persists its permitted failure projection, and restores prior
  in-memory authority if that save fails.
- No Workstream 6 checkpoint callback or persistence loop has been introduced.
- `git diff --check` succeeds.

## Stage 4 — Delete The Run-Manifest Domain Model

**Status: Pending**

### Goal

Remove the now-unreferenced structured schema, persistence API, canonical file
path, and mechanism tests from their owning packages.

### Required Work

1. Delete `internal/manifest/run_manifest.go`, `RunManifestSchemaVersion`,
   `RunManifest`, `RunStageRecord`, run status/action enums, constructors,
   transition methods, and all run-only validation/normalization helpers.
2. Remove `manifest.RunStore` and `LocalStore.CreateRun`, `LoadRun`,
   `LoadRunReader`, and `SaveRun`. Simplify imports and shared validation only
   where the deleted run code made them unnecessary; do not weaken session
   manifest validation.
3. Delete `artifacts.SessionRunManifestPathForCampaign` while retaining
   `SessionRunsDirForCampaign` and `SessionRunRootForCampaign` as canonical
   run-diagnostic and invocation-work paths.
4. Delete `run_manifest_test.go` and run-manifest-only store/path/schema tests.
   Remove run-specific portions of otherwise valuable manifest, artifact,
   semantic fingerprint, restore, operator, and publish fixtures. Do not replace
   serialization round-trip tests for a format that no longer exists.
5. Keep the remote `run_manifest` artifact enum accepted only by remote commit
   decoding/validation. Add a concise comment at its declaration explaining
   that Narratio no longer emits or interprets it and retains the value solely
   so immutable existing commits remain readable. No production symbol in
   `manifest`, `app`, or `stage` may refer to it.
6. Update package comments or focused internal documentation affected by the
   domain deletion in the same change.

### Exit Criteria

- `go test ./internal/manifest ./internal/artifacts ./internal/app ./internal/stage`
  succeeds.
- Except for the explicitly retained `RemoteArtifactTypeRunManifest` wire
  value and its remote-decoder compatibility test, searches for `RunManifest`,
  `RunStore`, `CreateRun`, `LoadRun`, `SaveRun`, `RunManifestPath`,
  `run_manifest.go`, and the local `runs/<run_id>/manifest.json` contract find
  no current production or test implementation.
- Session manifest schema validation and round trips remain fully covered.
- Existing remote commit documents containing the historical `run_manifest`
  artifact type still decode and validate, but new publish plans cannot emit
  it.
- `git diff --check` succeeds.

## Stage 5 — Reconcile Current Documentation And Close Workstream 4

**Status: Pending**

### Goal

Prove the session-only target state across the repository and leave one accurate
implementation narrative for the next workstream.

### Required Work

1. Search current production code, tests, fixtures, examples, scripts, and
   nonhistorical documentation for run-manifest types, paths, JSON fields,
   stores, publish classifications, invocation projections, and dual-save or
   terminal-synchronization language. Remove stale surfaces or explain the
   retained run ID/run directory without implying a structured record.
2. Reconcile canonical owners, especially `docs/policy/architecture.md`,
   `docs/cli.md`, `docs/operations.md`, `docs/troubleshooting.md`,
   `docs/internal/overview.md`, `docs/internal/manifest.md`,
   `docs/internal/stage-analyze.md`, `docs/internal/stage-extract.md`, and
   `docs/internal/stage-publish.md`. Keep schema definitions in the manifest
   owner, filesystem/remote behavior in operations, and implementation details
   in internal documents rather than duplicating them.
3. Do not edit immutable historical release notes or completed audit documents.
   Roadmap references may retain the phrase “run manifest” when describing the
   problem and approved removal. Current instructions must not claim the file
   is produced or required.
4. Review the affected test suite under `docs/policy/testing.md`. Consolidate
   leftover tests that merely encode the deleted write choreography, while
   retaining meaningful session durability, transition, recovery, current-run
   diagnostics, remote publication, and representative assembled-workflow
   coverage.
5. Update Workstream 4 in `refactoring.md` from `Pending` to `Complete` and
   update every stage plus the purpose/status section of this plan to
   `Completed` only after the target state and all checks are proven. Do not
   change later workstream statuses.
6. Run the complete repository validation from `docs/development.md`:
   `go test ./...`, `go test -race ./...`, `go vet ./...`, `go build ./...`,
   `go test ./internal/doccheck`, and
   `go test ./internal/config -run '^TestExamplesLoadAndValidate$'`. Run
   `git diff --check` and confirm the worktree contains only intended changes.

### Exit Criteria

- The session manifest is the only structured durable local progress record;
  no current code or documentation requires a run manifest.
- Run IDs, canonical run directories, bounded diagnostics, generated
  configuration, receipts, immediate summaries, immutable run prefixes, and
  remote publication remain functional without a second progress transaction.
- Publish uses session-manifest evidence, emits no new run-manifest artifact,
  and still reads verified historical remote commits without interpreting their
  old diagnostic record.
- All repository-wide validation succeeds on the maintained Go toolchain.
- Workstream 4 and all five stages are marked `Complete`; Workstream 5 remains
  the next pending workstream.
- `git status --short` is empty after the implementation commit.

## Open Questions

None. The approved audits, current architecture, and decisions above are
sufficient to implement this workstream without further product or policy
choices.