narratio/docs/roadmap/implementation.md

Workstream 3 Implementation Plan

Purpose And Status

This is the executable plan for refactoring.md, Workstream 3: Canonical contracts and compatibility purge. Finding 1 of contract-audit.md and rows C01/H43 of audit-decisions.md own the approved decisions.

All seven stages are pending and must be implemented in order. Each is bounded for one gpt-5.6-terra prompt. Workstreams 1 and 2 are complete. Workstreams 4 and later are out of scope: do not remove the run manifest, redesign runner lifecycle, change analyze checkpoint timing, replace locks, decompose publish, or add cleanup while following this plan.

Settled Implementation Decisions

• Delete deprecated representations rather than renaming, wrapping, or preserving them behind an internal compatibility facade. Historical releases and completed audits remain unchanged.
• Give both currently persisted local JSON manifest types an explicit required schema_version: narratio.session-manifest.v1 for Manifest and narratio.run-manifest.v1 for RunManifest, with discoverable constants in package manifest. Constructors set it; save and load validate it; neither path fills in a missing or unsupported value. Workstream 4 may later delete the versioned run-manifest model.
• Loaded current state may not acquire required schema, stage identity, stage records, or analyze authority through normalization. Representation- preserving normalization may remain.
• Every newly created analyze stage record in either manifest uses AnalyzeStateContractVersion. An empty collection is valid current state. Version zero, an unknown version, or aggregate-only authority is rejected.
• The only party input is a strict narratio.party.v1 document selected by required, campaign-owned campaign.inputs.party_file. Remove unversioned parties, campaign.inputs.players_file, session.inputs.players_file, and session.inputs.party_file; strict YAML decoding rejects those keys.
• Players remain the deterministic narratio.players.v1 projection generated from the party. Keep that prepared input, semantic identity, Notarius reference role, and operator visibility; it is never separately configured.
• Simplify party runtime data to a canonical party and source provenance. Remove PartyMode, both mode constants, the compatibility-only PartyDocument shell, and party_legacy.go. ParseParty returns the canonical representation directly.
• The only remote current selector is versioned current/commit.json and its verified immutable commit. Pointer absence returns a typed current-commit-pointer-missing error with publish/republish guidance. Do not inspect current/run_id.txt or current/manifest.json.
• Successful current-state loads always contain a validated non-nil pointer, commit, and committed session manifest. Keep CurrentPointerKey and the immutable manifest key; remove legacy key names and duplicated fields.
• Restore uses only objects declared by the selected commit and never lists a session prefix to reconstruct a snapshot. Preserve committed verification, manifest-last installation, conflicts, and root-scoped safety.
• Previous-session hydration resolves each requested source exactly once from the commit's unique published_output declaration. Derive its relative destination from that declaration's immutable destination key. Do not consult current config or previous-manifest outputs for alternate paths and do not issue candidate Exists calls.
• A built-in artifact is authoritative only when its succeeded producer stage has exactly one output matching registered source and kind, with safe path, producer run ID, and checksum. Verify checksum and content. Remove canonical- path and filename fallback; disk bytes alone never establish availability.
• Analyze authority comes only from the current versioned AnalyzeArtifacts collection. Preserve fingerprints, dependencies, selection, force behavior, and existing session/run projections until their later workstreams; remove only aggregate-only and incidental-file compatibility here.
• Extraction availability continues to require current input and receipt identity, canonical immutable paths, a complete configured source set, checksums, contracts, and Notarius provenance. The manifest schema boundary rejects older records; add no extraction migration reader.
• Unsupported state errors identify the missing/received and supported version and provide concise migration guidance. Tests assert error categories and small stable fragments, not full prose.
• Current docs must direct operators to replace legacy party config, move aside and regenerate unsupported local state, restore a current committed snapshot when authoritative, or republish healthy local state. Do not rewrite historical documents.
• Mark only Workstream 3 Complete, and only after all stages and final validation succeed.

Instructions For Every Stage

Before each stage:

1. Read docs/development.md, every file under docs/policy/, this plan, Workstream 3 plus dependency/target sections of refactoring.md, Finding 1 of contract-audit.md, and rows C01/H43 of audit-decisions.md.
2. Inspect named implementation and focused tests. Prefer the codebase graph for Go discovery and text search for fields, fixtures, examples, and docs.
3. Confirm worktree state and preserve unrelated changes. Do not enter a later workstream because adjacent code would also benefit.

During every stage:

• Keep the repository building and focused tests passing. Temporary current-only migration structure is allowed only when a later stage explicitly removes it; add no compatibility shim or automatic migration reader.
• Preserve package ownership, strict deterministic decoding, group workspace behavior, bounded I/O, secret redaction, remote verification, restore safety, and Linux/macOS/Windows support.
• Keep tests offline and behavior-focused. Delete tests only with their retired contract. Update canonical current-behavior docs with the code change.
• Run gofmt on changed Go, focused tests, and git diff --check. Mark a stage Completed only after all exit criteria pass.

Stage 1 — Establish Explicit Current Manifest Schemas

Status: Completed

Goal

Create an unambiguous local-state boundary before removing downstream legacy branches.

Required Work

1. Add the settled schema constants and required schema_version fields in internal/manifest/manifest.go and run_manifest.go; set them in New and NewRun.
2. Require exact versions in LocalStore.Save, LoadReader, SaveRun, and LoadRunReader. Missing/unknown versions return a clear migration error. Saving a hand-built zero-value manifest fails rather than backfilling.
3. Require a non-nil stage map; reject nil stage entries and embedded stage names that are empty or differ from their map keys. Remove load normalization that synthesizes these required markers, retaining only harmless ordering or cloning.
4. Make session and run ensureStage initialize the analyze contract version and an empty collection for analyze. Make validateAnalyzeStageState require that version for every present analyze stage regardless of lifecycle status. Delete version-zero acceptance.
5. Update current test construction to use constructors or explicit markers. Cover missing/supported/unknown versions, nil/mismatched stages, newly created analyze state, empty current collections, and aggregate-only rejection.
6. Update docs/internal/manifest.md, docs/internal/artifacts.md, docs/operations.md, and docs/troubleshooting.md with schema and local recovery guidance. Leave party/remote removal for their stages.

Exit Criteria

• go test ./internal/manifest ./internal/artifacts ./internal/stage ./internal/app succeeds.
• Both manifest kinds round-trip only with exact supported versions.
• A present analyze stage cannot round-trip without current per-artifact versioning; an empty current collection can.
• Load does not synthesize required stage identity or authority.
• git diff --check succeeds.

Stage 2 — Reject Unversioned Party Documents

Status: Completed

Goal

End interpretation of unversioned party files before removing the configuration fields and mode shell that supported them.

Required Work

1. Make ParseParty require exactly PartySchemaVersion in the first and only YAML document. Missing/unknown versions identify the supported schema; retain strict malformed, duplicate, ambiguity, and trailing-document checks.
2. Delete classifyLegacyPartyDocument, resolveLegacyParty, and party_legacy.go. For this stage only, the resolved mode shell may remain so all packages compile, but production cannot construct legacy mode.
3. Simplify resolvePartyInput to return parsed canonical data and source directly, with no classification branch.
4. Replace legacy parser/resolution tests with required-v1, missing/unknown version, multi-document, strict-field, alias, identity/order, and deterministic players-projection tests.
5. Update docs/integrations/party.md, the party section of docs/config.md, docs/internal/configuration.md, and troubleshooting. Do not yet claim removed config keys are unknown; Stage 3 changes structs.

Exit Criteria

• go test ./internal/config ./internal/stage ./internal/app succeeds.
• No production code classifies or resolves an unversioned party, and party_legacy.go is absent.
• Unsupported party schema fails before family expansion or preparation.
• Canonical parsing and derived players output remain deterministic.
• git diff --check succeeds.

Stage 3 — Remove Legacy Party And Players Configuration

Status: Completed

Goal

Make one campaign party the only configured roster authority and players an explicitly derived value.

Required Work

1. Remove PlayersFile from CampaignInputsConfig and both PlayersFile and PartyFile from SessionInputsConfig. Strict decode must reject all three retired YAML paths as unknown unsupported fields.
2. Keep campaign PartyFile in resolved stable inputs. Represent generated players as derived data, not a selectable ResolvedInputFile; introduce a clearly named derived value if prepared-input consumers need one.
3. Remove PartyMode, both constants, PartyDocument, IsCanonical, and all mode branches. Make ResolvedParty hold non-nil CanonicalParty plus source provenance, and make ParseParty return *CanonicalParty.
4. Simplify mergeCampaignSession, campaign-party loaders, resolved validation, family expansion, effective digest/source reporting, and config commands. Party selection always comes from campaign config.
5. Delete internal/stage/prepare_legacy.go. Simplify prepare/resume, extraction references, and semantic contracts to copy canonical party and render players. Preserve prepared paths, checksums, input kinds, Notarius selectors, and meaningful derived_from_party provenance.
6. Make operator/source rendering show campaign party and derived players, with no legacy_player role or separate player path.
7. Rewrite affected config, stage, app, template, profile, and command tests. Add strict rejection for all retired keys and an integrated config/prepare test proving one party produces both prepared documents.
8. Remove legacy roster text/fixtures from docs/config.md, docs/cli.md, docs/internal/configuration.md, docs/internal/stage-prepare.md, docs/internal/stage-extract.md, and examples/.

Exit Criteria

• go test ./internal/config ./internal/stage ./internal/app succeeds.
• No production YAML tag accepts players_file or session party_file.
• No production PartyMode, PartyDocument, or legacy prepare symbol remains.
• One campaign v1 party supplies both prepared party and derived players.
• Current docs/examples contain no legacy roster workflow.
• git diff --check succeeds.

Stage 4 — Make Remote Current State Commit-Only

Status: Completed

Goal

Make the versioned commit pointer the sole remote authority for status, restore, and reuse.

Required Work

1. Replace legacy missing errors with typed CurrentCommitPointerMissingError keyed to S3CurrentCommitPointerKey. LoadCurrentState returns it with publish/republish guidance when absent and otherwise loads only committed state.
2. Delete current_state_legacy.go, legacy limits, ResolveCurrentStateKeys, and retired key helpers once callers are gone. Retain current pointer/commit/session-manifest bounds.
3. Require non-nil pointer, commit, and manifest in CurrentState; keep CurrentPointerKey and committed manifest key; remove CurrentRunIDKey and legacy-oriented duplication. Update identity errors.
4. Mirror this in app.RemoteCurrentState, discovery, status, lock inspection, restore, and helpers. Status may report missing pointer as not published; malformed or unsupported current data remains an error.
5. Delete buildLegacyRestoreActions, legacy key mapping, prefix listing, and checksum inference. Require committed restore planning while preserving audio scope, verification, conflicts, reports, manifest-last order, and installation safety.
6. Replace legacy fixtures with complete pointer/commit/manifest fixtures. Cover missing pointer, bad versions/identity, undeclared/missing objects, checksum/size/generation mismatch, and proof that restore neither lists nor reads retired keys.
7. Remove legacy remote language from docs/operations.md, docs/troubleshooting.md, docs/internal/manifest.md, docs/internal/artifacts.md, docs/internal/command-restore.md, and docs/internal/stage-publish.md; document republishing.

Exit Criteria

• go test ./internal/artifacts ./internal/app ./internal/stage succeeds.
• Production never reads current/manifest.json or current/run_id.txt.
• Every successful current load has a verified pointer, commit, and manifest.
• Restore derives all remote scope from the commit without prefix listing.
• git diff --check succeeds.

Stage 5 — Resolve Previous Artifacts From Commit Declarations

Status: Pending

Goal

Use one deterministic source-to-object lookup per previous requirement without path candidates or remote probing.

Required Work

1. Rewrite previouscache.Resolve around guaranteed committed state. Require the unique session-manifest declaration and always copy its checksum, size, generation, and verified content into the plan.
2. For each sorted requirement, derive its canonical source ID and select its unique published_output. Validate and strip the selected run's immutable published-output prefix from DestinationKey to derive the local relative path; never use current config or previous-manifest outputs for it.
3. Preserve required/optional behavior, deterministic ordering, safe local paths, and checksum/size/generation metadata.
4. Delete legacy candidate/basename matching, manifest/config fallbacks, candidate Exists calls, probing ambiguity errors, and unused derivation helpers. Delete the test-only BuildPlan alias; call Resolve.
5. Test exact declarations, no candidate Exists or list calls, required/optional outcomes, unsafe destinations, duplicate-source rejection, and verified metadata through prepare/restore integration.
6. Update docs/internal/stage-prepare.md, docs/internal/command-restore.md, docs/internal/artifacts.md, docs/operations.md, and troubleshooting.

Exit Criteria

• go test ./internal/previouscache ./internal/stage ./internal/app succeeds.
• Resolution contains no candidate, basename/config fallback, ambiguity probe, or compatibility alias.
• Every hydration traces to one verified commit declaration.
• Optional absence skips and required absence fails without probing.
• git diff --check succeeds.

Stage 6 — Make Local Artifact Evidence Manifest-Only

Status: Pending

Goal

Remove aggregate and incidental paths so transcript, extraction, and analyze consumers require current manifest evidence.

Required Work

1. Rewrite ResolveSessionArtifact and its private candidate logic to require a succeeded registered producer and exactly one output matching both Kind and SourceID. Require safe path, producer run ID, and valid checksum; verify checksum and registered content before returning. Reject ambiguous/malformed evidence.
2. Delete fallback.canonical_path, canonical-path probing, filename fallback, and old-candidate deduplication. Missing evidence remains typed unavailable even when expected bytes exist.
3. Update render, extract input identity, transcript resolution, and analyze built-in resolution to use this path while retaining stage context, content checks, and semantic fingerprints.
4. Remove analyzeReconciliationLegacy and the version-zero reconciliation branch. Unsupported in-memory state is non-resumable/corrupt, not legacy state eligible for recovery.
5. Audit catalog hydration, analyze inputs, publish discovery, operator inspection, and projections: configured analyze output authority comes only from a current AnalyzeArtifacts record and its output evidence, never aggregate outputs or configured paths. Do not implement Workstream 6 checkpointing.
6. Retain only strict extraction proof in InspectExtractionEvidence: succeeded stage, matching input, canonical bundle, receipt, exact lane set plus index, verified JSON/checksums, contracts, and Notarius provenance. Delete any acceptance of missing contract/provenance; do not redesign the adapter.
7. Test that incidental files, aggregate-only analyze outputs, missing/failed/stale/unselected records, mismatched checksums, and incomplete extraction evidence remain unavailable. Retain positive coverage for all built-in transcript sources, configured extraction lanes, and analyze outputs.
8. Update docs/internal/artifacts.md, docs/internal/stage-analyze.md, docs/internal/stage-extract.md, docs/internal/stage-publish.md, docs/internal/manifest.md, docs/config.md, operations, and troubleshooting without legacy/fallback language.

Exit Criteria

• go test ./internal/artifacts ./internal/manifest ./internal/stage ./internal/app succeeds.
• Disk files cannot establish transcript or analyze availability without current evidence; aggregate analyze outputs cannot substitute.
• Extraction lacking receipt, contract, provenance, complete sources, or verified bytes cannot hydrate the catalog.
• Positive current transcript/extraction/analyze/publish/operator flows work.
• git diff --check succeeds.

Stage 7 — Complete The Purge And Validate The Repository

Status: Pending

Goal

Remove residual compatibility material, validate the current-only architecture, and close Workstream 3 without entering later work.

Required Work

1. Search active code, tests, fixtures, examples, and current docs for retired remote keys/readers, previous candidates, unversioned party modes, players_file, session party overrides, aggregate analyze authority, incidental artifact fallback, missing manifest schemas, and extraction evidence without contract/provenance.
2. Delete unused constants, fields, types, errors, helpers, aliases, files, fixtures, and imports belonging to those paths. Do not remove unrelated adapter defaults, optional-input behavior, or later-workstream seams merely because they use “fallback.”
3. Review canonical docs under the documentation policy for consistent current behavior and actionable migration. Leave docs/releases/, audits, and decision history unchanged.
4. Run go test ./..., go test -race ./..., go vet ./..., documented docs/example validation, and supported Linux/macOS/Windows release builds. Cross-builds prove compilation only.
5. Review tests against testing policy: retain rejection and positive tests at every authority boundary without recreating deleted algorithms in tests.
6. Mark only Workstream 3 Complete in refactoring.md and all stages here Completed. Accurately record unavailable native-platform evidence.

Exit Criteria

• Full tests, race tests, vet, docs/examples, and release cross-builds succeed.
• No runtime branch accepts or infers a retired Workstream 3 representation.
• Each current manifest, party, remote snapshot, previous input, transcript, extraction artifact, and analyze artifact has one documented authority.
• Only Workstream 3 is newly complete and git diff --check succeeds.

Open Questions

None. The approved audits, roadmap, policies, and current implementation make this plan decision-complete.