diff --git a/docs/roadmap/audit.md b/docs/roadmap/audit.md new file mode 100644 index 0000000..a4aff04 --- /dev/null +++ b/docs/roadmap/audit.md @@ -0,0 +1,104 @@ +# Implementation Plan Summary + +This is a concise, plain-language summary of +[`implementation.md`](implementation.md). It describes the intended real-world +outcome of each stage without reproducing its implementation details. + +1. **Stage 1 — Collaborative workspace permissions:** Treat ordinary Narratio and + Notarius data as shareable and make managed workspaces group-writable, while + retaining private handling for API keys. +2. **Stage 2 — Safe identifiers:** Reject campaign, session, run, artifact, and + source identifiers that could escape their intended filesystem namespace, and + use fuzz tests to cover platform-specific path tricks. +3. **Stage 3 — Durable file replacement:** Consolidate duplicated atomic-write + functions into one shared mechanism that fully persists a replacement before + reporting success. +4. **Stage 4 — Confined writes and downloads:** Rewrite destination mutations so + symlinks or concurrent directory replacement cannot redirect writes, + promotions, or downloads outside the intended root. +5. **Stage 5 — Safe deletion and crash-recoverable locks:** Confine recursive + cleanup to its authorized root and replace stale lock-file existence checks + with operating-system locks released automatically after process death. +6. **Stage 6 — Protected API-key reads:** Read API keys only from private, + bounded, regular files without following symlinks or exposing key material in + errors. +7. **Stage 7 — Bounded external results:** Prevent external adapters from causing + memory or disk exhaustion by validating regular result files and enforcing + generous, clearly reported per-adapter size limits. +8. **Stage 8 — Complete subprocess termination:** Ensure cancellation, timeout, + or a safety-limit failure terminates and reaps an external command's entire + process tree rather than only its parent process. +9. **Stage 9 — Safe subprocess diagnostics:** Redact known credentials from + stdout/stderr, cap persisted diagnostics, and terminate runaway producers when + those caps are reached. +10. **Stage 10 — Safe publish inputs:** Ensure publishing reads and uploads only + verified regular files declared within the selected run, even during + filesystem races. +11. **Stage 11 — Consistent run identity:** Resolve one campaign/session/run + identity for an invocation, reject conflicting authorities, and prevent stale + identity fields from leaking into a later run. +12. **Stage 12 — Reliable failure recording:** Consolidate terminal-failure + persistence so handled errors reliably update authoritative session state and + preserve any secondary persistence failures. +13. **Stage 13 — Immutable remote-state model:** Define a versioned immutable + remote snapshot selected by a small pointer, while isolating old-format read + compatibility so it can be removed after migration. +14. **Stage 14 — Transactional publication:** Upload and verify a complete + immutable snapshot before one final pointer change makes it current, using an + exact source-to-destination mapping instead of basename guesses. +15. **Stage 15 — Safe remote locking and pagination:** Use provider-enforced + conditional writes so publishers cannot overwrite another owner's lock, and + fail instead of looping when object-store pagination stops making progress. +16. **Stage 16 — Retryable cleanup:** Persist post-publication cleanup as a + durable obligation so interrupted or failed deletion is retried and never + mistaken for completed cleanup. +17. **Stage 17 — Snapshot-consistent restore:** Make restore and status use one + selected immutable snapshot throughout the operation, and prevent `--force` + from overwriting unsafe directory or non-file conflicts. +18. **Stage 18 — Race-safe, portable restore:** Serialize restore against runner + reuse, leave durable evidence of incomplete restores, and replace unsafe + producer-machine absolute paths with validated local references. +19. **Stage 19 — Correct audio-cache reuse:** Reuse downloaded audio only when its + local bytes and recorded metadata match the selected remote object version. +20. **Stage 20 — One previous-session resolver:** Give restore, prepare, run, and + dry-run one consistent view of required and optional previous-session inputs, + while avoiding ambiguous matches and duplicate downloads. +21. **Stage 21 — Strict configuration:** Reject multiple YAML documents, invalid + durations, implicit storage backends, and unmet previous-session expectations, + while separating configuration tests by responsibility. +22. **Stage 22 — Truthful product settings and temp-file ownership:** Remove + configuration fields that do nothing, reject unsupported notification + settings, and guarantee cleanup of remote-configuration temporary files. +23. **Stage 23 — Streaming WhisperX transport:** Stream uploads instead of + buffering entire audio files, reject unsupported endpoint schemes, and make + retries, cancellation, fake-server recording, and race tests reliable. +24. **Stage 24 — Correct prepare/transcribe transitions:** Prevent stale previous + data, cancelled or partial transcription work, and duplicate source identity + from being recorded as successful current output. +25. **Stage 25 — Authoritative output paths:** Require adapters to honor the + stage-requested output destination and consolidate duplicate singleton + transcript resolution without confusing it with multi-source discovery. +26. **Stage 26 — Shared extraction evidence:** Consolidate duplicated + extraction-bundle validation into one typed proof while allowing resume and + catalog consumers to apply their distinct policies. +27. **Stage 27 — Transcript-aware extraction reuse:** Include the direct + transcript's identity in extraction freshness checks so changed input cannot + reuse stale structured artifacts. +28. **Stage 28 — One effective artifact selection:** Resolve configured and + explicitly selected artifacts once, then use that same typed set for + prerequisites, extraction catalogs, analyze inputs, and execution planning. +29. **Stage 29 — Predictable analyze planning:** Represent optional and required + analyze inputs explicitly, produce deterministic dependency errors, and give + operators correct remediation commands. +30. **Stage 30 — Contract cleanup:** Remove dead or misleading interfaces and + helpers, move static Audita configuration to its proper owner, and correct + stale contract comments. +31. **Stage 31 — Enforced automated validation:** Require tests, race checks, vet, + builds, and example validation for changes and releases, while consolidating + redundant broad tests without losing focused coverage. +32. **Stage 32 — Documentation and closure:** Reconcile normative documentation + with the completed behavior and verify that every planned remediation has one + completed, traceable implementation stage. + +Every stage's purpose was readily determinable from the implementation plan; no +stage required an uncertainty note. diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md new file mode 100644 index 0000000..f5e0273 --- /dev/null +++ b/docs/roadmap/implementation.md @@ -0,0 +1,1145 @@ +# Audit Remediation Implementation Plan + +## Purpose and status + +This document is the executable roadmap for remediating the confirmed findings in +[`audit-findings.md`](audit-findings.md). It is written for a `gpt-5.6-terra` +coding agent that will implement exactly one stage per prompt, in order. + +The audit is an immutable requirements and evidence ledger. Do not edit +`audit-findings.md`, reinterpret accepted findings, or copy its detailed analysis +into this document. The line ranges below are stable references to the evidence +and acceptance criteria that the implementing agent must read for each stage. + +This plan owns remediation progress; a separate `audit.md` tracker is not needed. +All stages are pending when this plan is created. + +| Stage | Summary | Primary findings | Status | +| ---: | --- | --- | --- | +| 1 | Align data classification and group workspace modes | RSK-004 | Pending | +| 2 | Enforce safe identifiers and fuzz path/source contracts | COR-002, TST-013 | Pending | +| 3 | Consolidate crash-durable atomic file replacement | RSK-002, DUP-001, DUP-005 | Pending | +| 4 | Add confined destination and download/install capabilities | COR-003, DUP-003, TST-003 | Pending | +| 5 | Confine recursive cleanup and replace sentinel locks | RSK-003 | Pending | +| 6 | Harden API-key file acquisition | RSK-010 | Pending | +| 7 | Bound and verify external result acquisition | RSK-013, TST-007 | Pending | +| 8 | Terminate owned subprocess trees | RSK-011 | Pending | +| 9 | Redact and cap subprocess diagnostics | RSK-012 | Pending | +| 10 | Confine publish archive reads | COR-005 | Pending | +| 11 | Make manifest and run identity singular | COR-001, TST-006 | Pending | +| 12 | Centralize handled terminal-failure persistence | RSK-001, TST-002, SIM-001, COM-001 | Pending | +| 13 | Introduce the immutable remote-commit model and legacy boundary | ARC-003 | Pending | +| 14 | Publish through immutable commits and canonical mappings | COR-004, COR-011, DUP-002, TST-004 | Pending | +| 15 | Make remote locks generation-safe and harden pagination | RSK-005, RSK-014 | Pending | +| 16 | Persist retryable post-commit cleanup state | COR-006, COR-007 | Pending | +| 17 | Bind restore/status to a committed snapshot and reject conflicts | COR-008, COR-009, TST-005 | Pending | +| 18 | Serialize restore transitions and make restored paths portable | RSK-006, RSK-008 | Pending | +| 19 | Bind audio cache reuse to remote object identity | RSK-007 | Pending | +| 20 | Unify previous-source readiness and eliminate duplicate transfers | COR-010, EFF-001 | Pending | +| 21 | Tighten configuration parsing, values, and expectations | COR-012–COR-015, TST-011, TST-014 | Pending | +| 22 | Make product configuration truthful and own remote temp files | COR-024, RSK-009, ARC-004 | Pending | +| 23 | Stream WhisperX uploads and make the adapter race-safe | COR-016, EFF-002, TST-001 | Pending | +| 24 | Correct prepare/transcribe transition semantics | COR-017–COR-019, TST-008 | Pending | +| 25 | Enforce output-path authority and shared singleton resolution | ARC-006, DUP-006 | Pending | +| 26 | Centralize typed extraction-bundle evidence | DUP-007 | Pending | +| 27 | Bind extraction reuse to direct transcript identity | COR-020, TST-009 | Pending | +| 28 | Establish one effective artifact set and catalog bootstrap | COR-022, ARC-007, DUP-008 | Pending | +| 29 | Make analyze resolution typed, optional, actionable, and deterministic | COR-021, COR-023, RSK-015, SIM-003, TST-010 | Pending | +| 30 | Remove misleading contracts and align Audita ownership | ARC-001, ARC-005, SIM-004, COM-003, COM-006 | Pending | +| 31 | Enforce CI validation and streamline the assembled test suite | TST-012, TST-015 | Pending | +| 32 | Reconcile lifecycle/analyze documentation and close the remediation | COM-002, COM-005 | Pending | + +## Governing decisions + +The following decisions are settled requirements, not questions for the +implementing agent: + +1. Ordinary Narratio and Notarius campaign/session data is deliberately + shareable, not private or sensitive. Group-owned, group-writable workspaces are + an operational requirement. API keys are the only sensitive data handled by + these applications and must remain private. +2. On POSIX systems, Narratio-managed ordinary workspace directories and files + should converge on setgid `02775` and `0664` respectively, inheriting the + workspace's existing group. API-key directories and files must be `0700` and + `0600`. Do not add ownership-changing behavior or assume Narratio may `chown`. + Windows behavior must preserve the same collaboration/security intent using + the platform's available guarantees and explicit operational guidance. +3. Remote current-state publication uses an immutable run-scoped commit manifest + selected by one final pointer update. Readers may accept coherent legacy state, + but all compatibility code must be isolated behind a clearly named boundary, + covered by dedicated tests, and carry an in-code removal comment. New writers + must never produce the legacy format. +4. Explicit notification no-op is supported. Configured notification backends or + recipients are rejected until a real provider exists. +5. External-output limits are runaway safeguards, not normal operating limits. + Each limit must be a named, centrally discoverable package constant and at + least an order of magnitude above expected ordinary output. Every limit error + must identify the enforcing adapter/contract, the configured byte limit, and + what exceeded it. Narratio-owned stdout/stderr capture is streaming and capped; + reaching its cap terminates the owned process tree. Hard whole-disk protection + is an operations concern implemented with filesystem/service/container/volume + quotas, not output-root polling. +6. Retain every accepted-risk disposition in the audit. In particular, do not + turn abrupt process-death detection into a distributed liveness system, do not + add rollback of arbitrary external side effects, and do not add generic + speculative abstractions rejected by the audit. +7. CI must run ordinary tests, vet, build, and example checks on every change and + release path; the complete race suite runs on every change once TST-001 is + repaired; shuffled tests run on a schedule. Add native macOS/Windows jobs only + when authoritative Woodpecker runner labels and usable runners are available. + Do not invent labels. Cross-compilation is useful but does not count as native + filesystem/process evidence. + +## Instructions for every stage + +For each implementation prompt, the coding agent must: + +1. Read `docs/development.md`, then the exact audit ranges and canonical documents + named by that stage. Follow the task-specific reading guide in + `docs/development.md`. Do not read all 4,000+ audit lines unless a stage + explicitly requires them. +2. Inspect the current tree before editing. Earlier stages may have changed names + and ownership boundaries. Use the repository knowledge graph first for code + discovery and call tracing, then use text search for documentation, string + literals, configuration, and evidence the graph cannot supply. +3. Confirm the worktree state and preserve unrelated user changes. Do not rewrite + or amend earlier commits unless explicitly instructed. +4. Implement the stage completely, including production code, focused regression + tests, platform-specific implementations where applicable, current-behavior + documentation, and removal of code made obsolete by the stage. Do not leave a + second competing path or defer required tests to a later stage unless this plan + explicitly says so. +5. Prefer typed, owner-specific contracts and existing package ownership. Do not + create a generic abstraction merely because two call sites look similar. Keep + policy at the owner and mechanism in `internal/fileops` or the relevant + adapter/package. +6. Treat path/race/security checks as production behavior. Validation followed by + an unconfined pathname operation is not a completed fix. Unsupported safe + behavior must fail closed with an actionable error. +7. Run focused tests while developing, then at minimum `go test ./...` for every + stage. For production-code changes also run `go vet ./...`, `go build ./...`, + and the repository's documentation/example validation command. Run targeted + `-race` tests for concurrency changes and platform build/test checks named by + the stage. Do not require live cloud or third-party services. +8. Compare the final diff with the stage's finding list and exit criteria. Update + only that stage's row in the table from `Pending` to `Completed`, adding the + implementing commit hash if the workflow supplies one. Do not mark a stage + complete while a required check is failing or a required behavior is missing. +9. Leave `docs/roadmap/audit-findings.md` unchanged. If implementation evidence + reveals that a confirmed finding is factually impossible to satisfy as written, + stop and report the contradiction instead of silently changing its meaning. + +## Stage 1 — Align data classification and group workspace modes + +**Read first:** `audit-findings.md` lines 2660–2693 (RSK-004), 4171–4224 +(resolved decisions and accepted risks); `docs/policy/architecture.md`; +`docs/operations.md`; and the workspace/artifact/file-operation package docs named +by `docs/development.md`. + +**Goal:** Make the architecture and implementation consistently describe and +support shareable, group-owned workspaces while keeping API keys outside that +ordinary-data policy. + +**Work:** + +- Revise the architecture policy's current-behavior contract: campaign/session, + manifests, transcripts, generated configs, logs, and Notarius artifacts are + intentionally shareable; API-key material is sensitive. +- Identify every Narratio-owned ordinary workspace creation/promotion path. Give + the owning packages named, discoverable mode constants and converge final POSIX + directories/files to setgid `02775`/`0664`, without changing ownership. +- Preserve the parent workspace group through nested creation and atomic + replacement. Account for restrictive caller umasks explicitly; do not rely on a + favorable umask to add group write or setgid bits. +- Keep temporary-file exposure no broader than the intended final object and do + not weaken the separate API-key rules implemented in Stage 6. +- Document Windows ACL/ownership expectations and any mode-bit limitations; do + not claim POSIX mode semantics on Windows. + +**Tests and exit criteria:** Add focused tests under restrictive umasks for nested +directories, final files, promoted Notarius trees, and repeated replacement. +Verify that group-write/setgid behavior is correct on POSIX and platform behavior +is explicit elsewhere. The architecture, operations, and security statements must +agree. No test should assert that ordinary campaign output is private. + +## Stage 2 — Enforce safe identifiers and fuzz path/source contracts + +**Read first:** `audit-findings.md` lines 1770–1810 (COR-002) and 3837–3852 +(TST-013); configuration, manifest, workspace, artifact, and source-grammar docs. + +**Depends on:** Stage 1. + +**Goal:** Prevent externally influenced identity tokens from becoming absolute, +traversing, platform-ambiguous, or control-bearing filesystem segments. + +**Work:** + +- Define one strict portable segment grammar for campaign IDs, session IDs, run + IDs, artifact identifiers, and other opaque filesystem identity tokens: ASCII + letters, digits, `.`, `_`, and `-` only; reject empty, `.`/`..`, separators, + drive/volume syntax, absolute forms, control characters, and all other bytes. +- Enforce the contract at configuration/model construction and again at the + artifact/path boundary. Return actionable errors; never sanitize or silently + rewrite identity. +- Audit manifest/source parsing for equivalent tokens. Unsafe legacy values must + fail with migration guidance, not be accepted because an older file contains + them. +- Preserve intentional multi-segment paths only in typed path contracts that are + separately confined; do not force path-valued fields through the opaque-ID API. + +**Tests and exit criteria:** Add table and fuzz tests for Unix/Windows separators, +drive forms, Unicode lookalikes, controls, empty/dot segments, and valid boundary +values. Include source-token parsers as required by TST-013. Prove all path builders +reject unsafe identifiers before filesystem access. + +## Stage 3 — Consolidate crash-durable atomic file replacement + +**Read first:** `audit-findings.md` lines 2590–2624 (RSK-002), 3407–3427 +(DUP-001), and 3493–3514 (DUP-005); `internal/fileops` and manifest persistence +docs. + +**Depends on:** Stages 1–2. + +**Goal:** Provide one owner-neutral byte-file installation mechanism with explicit +durability semantics, while leaving serialization and policy with callers. + +**Work:** + +- Implement or complete a narrowly scoped `internal/fileops` byte-write/replace + primitive: sibling temporary file, complete write, file sync, atomic rename or + documented platform replacement, then parent-directory sync where supported. +- Accept the caller-owned final mode; make cleanup and error composition explicit. + Preserve cancellation checkpoints in callers before durable commit. +- Migrate duplicate local manifest, artifact, and adapter byte-write helpers to the + primitive. Keep manifest encoding/versioning out of `fileops`. +- Define and test the platform contract for directory sync and replacement. A + platform limitation must be explicit, not silently treated as equivalent. + +**Tests and exit criteria:** Exercise short writes, write/sync/rename/directory-sync +failures through injected seams, repeated replacement, mode preservation, and +temporary cleanup. Existing manifest and adapter behavior must remain compatible. +No duplicate sibling-temp byte writer should remain. + +## Stage 4 — Add confined destination and download/install capabilities + +**Read first:** `audit-findings.md` lines 1811–1852 (COR-003), 3448–3471 +(DUP-003), and 3596–3617 (TST-003); filesystem threat model and `internal/fileops` +docs. + +**Depends on:** Stages 2–3. + +**Goal:** Make destination mutation resistant to ancestor and leaf symlink races, +and share the safe remote-download installation mechanism without sharing caller +policy. + +**Work:** + +- Add root-relative, handle-confined creation, replacement, and promotion + capabilities that verify/traverse every ancestor without following symlinks. + Use platform-specific implementations where necessary and fail closed if the + required guarantee is unavailable. +- Replace validate-then-use pathname mutations in artifact creation, manifest + installation, Notarius tree promotion, restore/download installation, and + related destinations. +- Implement one destination-confined sibling-temp download/install mechanism. + Callers continue to own object identity, verification, force/conflict policy, + and final mode. +- Do not implement recursive deletion here; Stage 5 owns that distinct capability. + +**Tests and exit criteria:** Add adversarial tests that replace intermediate +directories and leaves with symlinks between logical phases, plus regular success, +collision, cleanup, and platform tests. Prove no external target is modified. +TST-003 is primarily owned by this stage and may receive additional cleanup cases +in Stage 5. + +## Stage 5 — Confine recursive cleanup and replace sentinel locks + +**Read first:** `audit-findings.md` lines 1811–1852 (COR-003), 2625–2659 +(RSK-003), and 3596–3617 (TST-003); local locking and workspace lifecycle docs. + +**Depends on:** Stage 4. + +**Goal:** Make deletion root-confined and make local crash recovery rely on +OS-released locks rather than stale file existence. + +**Work:** + +- Add a root-relative, no-follow cleanup/delete capability and migrate recursive + cleanup paths. Never pass an unchecked or re-resolved absolute path to broad + recursive deletion. +- Replace existence-based sentinel locking with a held OS lock whose ownership is + released by the kernel on process death. Retain useful owner/run metadata + without using metadata-file existence as the lock decision. +- Implement explicit Unix and Windows lock backends, wait/contention behavior, + cancellation, and release-error reporting. Do not auto-delete another process's + active lock. +- Ensure lock files and cleanup roots follow the group workspace policy where they + are ordinary runtime data. + +**Tests and exit criteria:** Cover contention across independent processes, +process-death recovery, cancellation, release failures, symlink swaps, malicious +cleanup inputs, and idempotent cleanup. Run native tests where available and at +least cross-build all platform-specific files. + +## Stage 6 — Harden API-key file acquisition + +**Read first:** `audit-findings.md` lines 2870–2900 (RSK-010); architecture, +configuration, and integration docs governing API keys. + +**Depends on:** Stages 1, 2, and 4. + +**Goal:** Make the sole sensitive-data path private, bounded, regular-file-only, +and resistant to path races. + +**Work:** + +- Resolve API-key directories and files through a root-confined, no-follow read + path. Reject symlinks, non-regular files, unsafe relative names, replacement + races, and oversized content. +- Define the small API-key byte limit as a named, discoverable constant in the + owning package. Errors must name the API-key loader and the limit. +- Enforce `0700` directory and `0600` file privacy for Narratio-created secret + storage on POSIX, and reject unsafe existing modes where Narratio reads key + files. Document Windows ownership/ACL requirements without pretending mode bits + are authoritative there. +- Preserve documented environment/file precedence without persisting or logging + secret values. + +**Tests and exit criteria:** Cover mode rejection, symlink/ancestor swaps, +non-regular files, over-limit reads, valid newline trimming, precedence, and error +redaction. No API-key content may appear in failures or fixtures intended for logs. + +## Stage 7 — Bound and verify external result acquisition + +**Read first:** `audit-findings.md` lines 2963–3006 (RSK-013), 3691–3718 +(TST-007), and 3744–3774 (TST-009); adapter contracts and integration docs. + +**Depends on:** Stages 4 and 6. + +**Goal:** Prevent a runaway adapter or producer from forcing unbounded memory or +post-process disk consumption while leaving abundant headroom for real sessions. + +**Work:** + +- Factor the non-secret portions of the Stage 6 read mechanism into an + owner-neutral bounded, regular-file, no-follow acquisition primitive if that + does not already exist. Keep each contract's limit in its owning package. +- Inventory every externally produced file Narratio reads after a process exits: + adapter JSON, Audita output, Scriptorium output, WhisperX result, Notarius lane + result/catalog/bundle evidence, and any equivalent stage handoff. Migrate all of + them to bounded verified acquisition. +- Set generous documented constants at least ten times larger than credible + ordinary output. Do not embed numeric byte limits in call sites or hide them in + a generic helper. +- Every over-limit error must state the responsible adapter or producer contract, + the configured limit, and the file/result category. Never return partial parsed + output. +- Document that hard disk exhaustion is prevented operationally with filesystem, + service, container, or volume quotas; do not add polling of output-root size. + +**Tests and exit criteria:** Add exact-limit and limit-plus-one tests, growth/race +cases, non-regular and symlink cases, malformed/truncated results, and clear error +assertions for each adapter family. TST-007 is primarily owned here; later adapter +stages add their specialized liveness cases. + +## Stage 8 — Terminate owned subprocess trees + +**Read first:** `audit-findings.md` lines 2901–2931 (RSK-011); adapter process and +platform-support docs. + +**Depends on:** Stage 7. + +**Goal:** Ensure cancellation, timeout, and enforced output limits stop and reap +the complete subprocess tree Narratio owns. + +**Work:** + +- Introduce a narrow platform-specific process-runner capability that starts an + owned process in a Unix process group or Windows job object and can terminate + and reap the tree. +- Migrate Notarius, Audita, Scriptorium, WhisperX, and other owned external command + execution to it without broadening environment inheritance. +- Define graceful-then-forceful cancellation behavior, bounded waits, error + composition, and already-exited handling. Ensure descendants cannot retain + pipes indefinitely after the leader exits. + +**Tests and exit criteria:** Use helper processes that spawn descendants and hold +stdout/stderr open. Prove cancellation/timeout kills descendants and reaps the +leader. Add native Unix/Windows tests where runners exist and cross-build all +platform code; record no unsupported fallback as safe. + +## Stage 9 — Redact and cap subprocess diagnostics + +**Read first:** `audit-findings.md` lines 2932–2962 (RSK-012) and 2963–3006 +(RSK-013); logging, adapter, and operations docs. + +**Depends on:** Stages 6–8. + +**Goal:** Make persisted stdout/stderr and diagnostic tails bounded and unable to +disclose known credentials. + +**Work:** + +- Define an explicit set of sensitive environment-variable names and pass their + values only to a streaming redaction layer; never persist the values or include + them in error text. +- Minimize inherited environments for external processes while preserving the + documented variables needed for execution. +- Stream stdout and stderr through redaction into capped diagnostic files using + named, centrally discoverable limits in the owning process/adapter package. + Redaction must work when a secret spans write chunks. Diagnostic tails must be + derived from already-redacted content. +- If either capture limit is reached, terminate and reap the process tree through + Stage 8 and return an unmistakable error naming stdout/stderr, the responsible + adapter, and the byte limit. Do not report successful partial execution. + +**Tests and exit criteria:** Cover exact cap, cap-plus-one, split-token secrets, +overlapping secret values, empty values, tails, cancellation, descendant cleanup, +and on-disk inspection. Assert that neither raw nor transformed error paths expose +test credentials. + +## Stage 10 — Confine publish archive reads + +**Read first:** `audit-findings.md` lines 1891–1924 (COR-005); publish, artifact, +manifest, and storage docs. + +**Depends on:** Stages 2, 4, and 7. + +**Goal:** Ensure publish uploads exactly the verified regular files inside the +selected run root, despite symlink or replacement races. + +**Work:** + +- Enumerate publish candidates through the artifact/run manifest rather than an + untrusted tree walk. +- Open each candidate root-relatively without following ancestors or the leaf; + verify regular-file type and any declared identity immediately on the opened + object, then stream from that verified handle. +- Reject undeclared files, directories, devices, sockets, symlinks, and identity + changes. Preserve deterministic ordering and actionable source-relative errors. +- Keep object-store destination policy out of the filesystem primitive. + +**Tests and exit criteria:** Add regular, leaf-symlink, ancestor-swap, +post-enumeration replacement, non-regular, and undeclared-file cases. A hostile +workspace must never cause bytes outside the run root to be uploaded. + +## Stage 11 — Make manifest and run identity singular + +**Read first:** `audit-findings.md` lines 1731–1769 (COR-001) and 3670–3690 +(TST-006); configuration, environment, run-state, and manifest docs. + +**Depends on:** Stages 2–3. + +**Goal:** Use one validated campaign/session/run identity per invocation and +recompute all run-scoped manifest fields together. + +**Work:** + +- At invocation start, resolve and validate one effective configuration and bind + the runtime environment to it; remove later reads from stale or unvalidated + configuration instances. +- On every run, recompute campaign, session, run ID, source references, and other + run-scoped manifest identity as a coherent set. Do not retain fields from the + prior invocation merely because a manifest object is reused. +- Reject disagreement among configured, CLI-selected, persisted, and manifest + campaign/session identity with an actionable error before side effects. +- Preserve intentionally session-scoped history separately from the current + run-scoped projection. + +**Tests and exit criteria:** Add consecutive-run regression tests using one +process/environment, explicit mismatch tests for every authority pair, and TST-006 +tests proving downstream operations receive the resolved configuration instance. + +## Stage 12 — Centralize handled terminal-failure persistence + +**Read first:** `audit-findings.md` lines 2553–2589 (RSK-001), 3388–3406 +(TST-002), 3886–3906 (SIM-001), and 3980–3997 (COM-001); run/session lifecycle +docs. + +**Depends on:** Stages 3 and 11. + +**Goal:** Ensure every handled command failure terminalizes the authoritative +session record and best-effort diagnostic run record with correct compounded +errors. + +**Work:** + +- Introduce the narrow terminal-failure operation described by SIM-001: persist + the session's terminal failure first, then update the run diagnostic record. + Keep the running transition separate and run-first. +- Route every handled pipeline/stage error and cancellation path through it. + Return an error that preserves the operation failure plus any session/run store + disagreement; do not mask the primary failure or report success. +- Add the short rationale comment required by COM-001 explaining the deliberately + asymmetric two-ledger ordering. +- Keep abrupt process death as the accepted residual risk. Do not add heartbeats or + pretend a stale `running` record proves liveness. + +**Tests and exit criteria:** Inject session-store and run-store failures at each +step, including cancellation and already-terminal records. Assert call order, +persisted truth, and composed errors. Remove duplicate terminalization sequences. + +## Stage 13 — Introduce the immutable remote-commit model and legacy boundary + +**Read first:** `audit-findings.md` lines 1853–1890 (COR-004), 1992–2037 +(COR-008), and 3256–3279 (ARC-003); remote manifest, storage, restore, and +compatibility docs. + +**Depends on:** Stages 2–3 and 11. + +**Goal:** Define one versioned, immutable snapshot contract selected by a small +current pointer, without mixing local metadata or temporary compatibility into the +new model. + +**Work:** + +- Add a typed, versioned run-scoped remote commit manifest whose identity and + declared object set are immutable. Include campaign/session/run identity, + enough version/checksum/generation metadata to bind readers to the selected + snapshot, and an extensible typed source-to-destination artifact mapping. +- Define the final current pointer as the sole mutable selector of a commit. The + pointer must be small, parseable, and able to select exactly one immutable + commit; it is not a second mutable manifest. +- Separate local-only metadata from remote commit state. Remove or split + `current_pointer_written` as required by ARC-003; derive remote commitment from + the selected pointer/commit relationship rather than an independently mutable + boolean. +- Isolate coherent legacy-pair reading in a clearly named compatibility file or + package boundary with dedicated tests and an in-code comment stating that it is + temporary and removable after migration. Reject torn or ambiguous legacy pairs. + Do not add legacy writing. + +**Tests and exit criteria:** Unit-test model validation, versions, unknown fields +per repository policy, pointer/commit mismatch, checksum/generation mismatch, +legacy coherent/torn cases, and the removable boundary. This stage may introduce +the model and readers without switching publish; Stage 14 switches the writer. + +## Stage 14 — Publish through immutable commits and canonical mappings + +**Read first:** `audit-findings.md` lines 1853–1890 (COR-004), 2068–2103 +(COR-011), 3428–3447 (DUP-002), and 3618–3642 (TST-004); publish, artifact, +remote-manifest, and storage docs. + +**Depends on:** Stages 10 and 13. + +**Goal:** Make new publication an immutable-object transaction whose last remote +mutation selects the complete snapshot. + +**Work:** + +- Make the artifact package's canonical run-manifest path the only source used by + publish; remove the duplicate path construction identified by DUP-002. +- Before upload, build and validate the complete deterministic source-to-remote + destination mapping using exact artifact/source identity, not basename + heuristics. Reject collisions and ambiguity. +- Upload/verify every immutable data object and the immutable commit manifest + first. Update the current pointer exactly once, last. Never upload or mutate + current-state data after pointer selection. +- New publish writes only the new protocol. Preserve the isolated legacy reader + from Stage 13 solely for existing remote state. +- Make publish tests stateful enough to observe remote ordering, generations, + object contents, failure injection, and retry behavior. + +**Tests and exit criteria:** Prove failures before the pointer leave the old +snapshot selected; successful publication selects only a complete new snapshot; +same-basename sources remain distinct; retries are deterministic; and no +post-pointer upload occurs. This is the primary TST-004 stage; Stages 15–16 extend +the same stateful fake. + +## Stage 15 — Make remote locks generation-safe and harden pagination + +**Read first:** `audit-findings.md` lines 2694–2731 (RSK-005), 3007–3032 +(RSK-014), 3618–3642 (TST-004), and 3691–3718 (TST-007); storage adapter and +publish lock docs. + +**Depends on:** Stage 14. + +**Goal:** Make remote ownership effective immediately and replacement conditional, +and prevent non-progressing object-store pagination loops. + +**Work:** + +- Extend the object-store contract with the minimum conditional write/replace + metadata needed for generation/ETag-safe lock acquisition, renewal, and release. + Implement it in the S3 adapter and stateful fake without reducing provider + preconditions to a local check-then-write. +- Commit remote lock ownership immediately, retry conditional conflicts with + bounded/cancellable policy, and never release another owner's generation. +- Revalidate lock ownership at the protected publish commit point immediately + before the current pointer changes. Loss of ownership must leave the old + snapshot selected. +- In paginated listing, reject an empty or repeated continuation token when the + provider claims more results. Include adapter/bucket/prefix context without + secrets. + +**Tests and exit criteria:** Add concurrent-owner, stale-generation, +renewal/release conflict, lock-loss-at-commit, cancellation, empty token, repeated +token, and normal multi-page tests. Run relevant tests under `-race`. + +## Stage 16 — Persist retryable post-commit cleanup state + +**Read first:** `audit-findings.md` lines 1925–1991 (COR-006 and COR-007); +publish, remote manifest, cleanup, and run/session lifecycle docs. + +**Depends on:** Stages 5 and 12–15. + +**Goal:** Make deletion after publication a durable, idempotent obligation that +does not disappear when execution state is reset or an attempt fails. + +**Work:** + +- Persist a typed cleanup obligation independently of the current executed-step + list before destructive cleanup can begin. Bind it to the committed run and + exact declared objects/paths. +- On retry/resume, detect and continue outstanding cleanup even when the current + stage list is empty or rebuilt. Keep cleanup idempotent and confined through + Stage 5. +- Persist completion evidence only after deletion has succeeded or an + already-absent target has been safely verified. A failed evidence write must + leave a retryable obligation rather than falsely completed state. +- Define command outcomes when publication committed but cleanup failed: the + committed snapshot remains authoritative, while the invocation clearly reports + incomplete cleanup and can retry it. + +**Tests and exit criteria:** In the stateful publish fake, inject failures before, +during, and after deletion/evidence persistence; restart with empty execution +state; retry multiple times; and prove no unrelated target is deleted. + +## Stage 17 — Bind restore/status to a committed snapshot and reject conflicts + +**Read first:** `audit-findings.md` lines 1992–2067 (COR-008 and COR-009) and +3643–3669 (TST-005); restore, status, remote manifest, and workspace docs. + +**Depends on:** Stages 4, 13–16. + +**Goal:** Restore and status must observe one pointer-selected immutable snapshot, +and `--force` must never overwrite a directory conflict. + +**Work:** + +- Resolve the current pointer once, load and validate the selected immutable + commit, and use only its declared object set and generation/checksum metadata + throughout one restore/status operation. +- Do not reconstruct current truth from mutable prefixes or a mixture of runs. + Route coherent legacy state through only the isolated Stage 13 compatibility + reader. +- Define conflict types before mutation. `--force` may replace eligible regular + managed files but must reject directory/non-regular conflicts, leaving no + completed local manifest and no false success. +- Make the restore fake stateful and able to mutate remote objects/pointers between + calls so snapshot-binding failures are observable. + +**Tests and exit criteria:** Cover pointer changes during restore, undeclared and +missing objects, version/generation mismatch, coherent legacy restore, file and +directory conflicts with/without force, partial download, and manifest absence on +failure. This is the primary TST-005 stage; Stages 18–20 extend it. + +## Stage 18 — Serialize restore transitions and make restored paths portable + +**Read first:** `audit-findings.md` lines 2732–2769 (RSK-006) and 2803–2838 +(RSK-008), plus TST-005 at lines 3643–3669; restore, local lock, manifest, and +runner reuse docs. + +**Depends on:** Stages 5 and 17. + +**Goal:** Prevent runner/restore races and ensure restored manifests cannot retain +unsafe producer-machine absolute paths. + +**Work:** + +- Acquire the session/workspace lock before executable-state classification and + revalidate state under the lock. All runner reuse checks must participate in the + same transition protocol. +- Persist a durable incomplete-restore marker before the first local mutation. + Keep it blocking runner reuse after a partial forced restore; clear it only after + all verified installs and the completed local manifest are durable. +- Rebase restored artifact paths from portable logical references to the selected + local root. Reject absolute paths outside that root and unsafe relative paths. + Retain an original absolute path, if useful, only as bounded non-authoritative + provenance. +- Do not attempt rollback of arbitrary external effects; the durable incomplete + marker and safe retry are the recovery contract. + +**Tests and exit criteria:** Add concurrent runner/restore process tests, +classification races, forced partial restore/restart/retry, Unix/Windows absolute +path forms, traversal, safe rebasing, and provenance bounds. + +## Stage 19 — Bind audio cache reuse to remote object identity + +**Read first:** `audit-findings.md` lines 2770–2802 (RSK-007) and TST-005 at +lines 3643–3669; previous-audio, object-store, and restore docs. + +**Depends on:** Stages 4, 15, and 17–18. + +**Goal:** Reuse restored/downloaded audio only when it is a verified regular file +for the selected remote object version. + +**Work:** + +- Persist a sidecar or equivalent typed cache record binding local audio to remote + object key plus authoritative version/generation/checksum information. +- Verify the local candidate root-relatively as a regular no-follow file and + validate size/digest as required before reuse. Do not treat an unqualified + multipart ETag as a content hash. +- Refresh or fail clearly when identity evidence is absent, stale, ambiguous, or + unsupported; `--force` must not mean trust an unrelated local file. +- Install refreshed content using Stage 4's confined/durable download path. + +**Tests and exit criteria:** Cover valid reuse, changed generation with same name, +missing/stale sidecar, multipart ETag, symlink/non-regular cache entries, corrupted +bytes, interrupted refresh, and force behavior. + +## Stage 20 — Unify previous-source readiness and eliminate duplicate transfers + +**Read first:** `audit-findings.md` lines 2038–2103 (COR-010 and COR-011) and +3064–3112 (EFF-001), plus TST-005 at lines 3643–3669; previous-cache, restore, +prepare, and dry-run docs. + +**Depends on:** Stages 14 and 17–19. + +**Goal:** Give restore, prepare, run, and dry-run one authoritative previous-source +resolver and retain verified downloads for the later restore snapshot. + +**Work:** + +- Implement one resolver that evaluates required/optional previous inputs from the + pointer-selected commit's exact source-to-destination mapping. Required absence + is actionable; optional absence is a typed non-error. +- Eliminate basename fallback from the new protocol. Keep any necessary legacy + fallback only inside the isolated Stage 13 compatibility boundary and reject + ambiguous matches. +- Reuse the generation-bound, verified candidate downloaded while constructing the + previous-source view when the restore snapshot needs the same object. Do not + reopen an unverified path or perform a second network transfer. +- Count actual transferred bytes once and make dry-run/readiness documentation + accurately describe remote checks and potential download/cache work. + +**Tests and exit criteria:** Prove identical behavior across prepare/restore/run, +required/optional cases, same basenames, legacy ambiguity, generation changes, +single-transfer accounting, cancellation, and dry-run output. + +## Stage 21 — Tighten configuration parsing, values, and expectations + +**Read first:** `audit-findings.md` lines 2104–2253 (COR-012 through COR-015), +3805–3821 (TST-011), and 3853–3868 (TST-014); configuration and CLI docs. + +**Depends on:** Stages 2 and 11. + +**Goal:** Reject ambiguous or semantically invalid configuration before side +effects and give the configuration package focused, non-duplicative tests. + +**Work:** + +- Require exactly one YAML document, including rejection of a second empty or + comment-only document if the parser exposes it as another document. +- Validate all duration values that control retries, timeouts, polling, or leases + as positive and within any existing semantic bounds. +- Select storage backends only through explicit supported configuration; remove + inference from incidental S3-shaped fields and reject incomplete/unknown modes. +- Make the previous-session flag a strict expectation: when requested, absence or + mismatch is an actionable error, not a silent downgrade. +- Centralize test environment cleanup and reorganize configuration tests into + loader, defaulting, and validation ownership as required by TST-011/TST-014. + +**Tests and exit criteria:** Cover multi-document YAML, zero/negative/overflow-like +durations, backend combinations, strict previous expectations, environment +isolation, defaults, unknown fields, and stable actionable errors. + +## Stage 22 — Make product configuration truthful and own remote temp files + +**Read first:** `audit-findings.md` lines 2521–2552 (COR-024), 3157–3197 +(ARC-004), and 2839–2869 (RSK-009); configuration schema, notifications, +Scriptorium, remote config, and integration docs. + +**Depends on:** Stages 4, 7, and 21. + +**Goal:** Expose only implemented configuration behavior and keep remote-config +temporary paths within a command-scoped lifecycle. + +**Work:** + +- Remove inert Scriptorium `artifact` and `path` fields from schema, Go types, + examples, validation, and docs. Reject obsolete fields under the repository's + strict configuration policy with clear migration guidance where feasible. +- Support notifications only as an explicit no-op mode. Reject any configured + backend, recipient, or provider-shaped value until a provider exists; align CLI + and docs. +- Replace ad hoc remote-config temp paths with one owned command-scoped object or + lifecycle. Clean it on success, validation failure, adapter failure, + cancellation, and early returns using confined deletion. +- Never serialize ephemeral local temp paths into manifests, run records, or + diagnostic output that purports to be reusable state. + +**Tests and exit criteria:** Cover obsolete/unknown product fields, explicit no-op, +backend/recipient rejection, every remote-config exit path, cancellation, stale +cleanup, and manifest inspection for ephemeral paths. + +## Stage 23 — Stream WhisperX uploads and make the adapter race-safe + +**Read first:** `audit-findings.md` lines 2254–2281 (COR-016), 3113–3156 +(EFF-002), 3363–3387 (TST-001), and 3691–3718 (TST-007); WhisperX integration +docs. + +**Depends on:** Stages 7–9 and 21. + +**Goal:** Make WhisperX transport bounded, retry-correct, cancellable, and safe +under the race detector. + +**Work:** + +- Accept only absolute `http` or `https` WhisperX endpoints; reject file and other + schemes before execution. +- Stream multipart bodies through a pipe per attempt instead of buffering the + complete audio request. Each retry gets a fresh reader/writer and deterministic + metadata; producer failures and cancellation must reach the HTTP caller without + goroutine leaks. +- Retain bounded response/result acquisition from Stage 7 and make request/response + limits and errors adapter-specific. +- Synchronize the Whisper fake's request collection and expose snapshots through a + safe accessor. Migrate tests away from direct shared-slice reads. +- Add evidence that memory behavior is no longer proportional to complete upload + size, using deterministic backpressure/streaming tests rather than fragile heap + thresholds alone. + +**Tests and exit criteria:** Cover schemes, retries, mid-stream read failure, +server early close, cancellation, blocked producer, exact payloads, bounded +responses, and goroutine completion. Run Whisper packages and their callers under +`go test -race`; this stage unblocks the full race suite in Stage 31. + +## Stage 24 — Correct prepare/transcribe transition semantics + +**Read first:** `audit-findings.md` lines 2282–2379 (COR-017 through COR-019) +and 3719–3743 (TST-008); prepare, transcribe, previous-cache, artifacts, and stage +lifecycle docs. + +**Depends on:** Stages 12 and 20–23. + +**Goal:** Make prepare/transcribe state transitions exact, cancellation-safe, and +independent of accidental basename behavior. + +**Work:** + +- Clear the managed previous-session tree when the effective requirement set is + empty as well as when it is populated. Publish only the current invocation's + verified result set. +- Track transcribe work as planned, dispatched, and completed. Cancellation or a + missing completion may not be reported as success and may not materialize a + completed artifact/manifest entry. +- Detect duplicate cleaned-audio source identities before dispatch. Allow distinct + source paths with the same basename and preserve exact identity through outputs. +- Keep state mutations inside the established lifecycle/terminalization contract. + +**Tests and exit criteria:** Add no-requirements cleanup, stale prior tree, +cancellation at each phase, partial completion, duplicate source, same-basename +distinct source, retry, and no-false-manifest tests. This is the primary TST-008 +stage; Stage 25 adds output/resolver boundary cases. + +## Stage 25 — Enforce output-path authority and shared singleton resolution + +**Read first:** `audit-findings.md` lines 3306–3330 (ARC-006), 3515–3537 +(DUP-006), and 3719–3743 (TST-008); stage request/result, artifact resolver, and +transcription docs. + +**Depends on:** Stage 24. + +**Goal:** Make the stage-requested destination authoritative and resolve singleton +transcripts through one artifact-owned contract. + +**Work:** + +- Require an adapter result path to be empty (meaning the requested path) or + exactly equal to the normalized stage-requested path. Reject any other path + before materialization. +- Put the equality/empty contract in one shared stage/adapter boundary helper and + add conformance tests for all built-in adapters. +- Route singleton transcript resolution through the artifact package's canonical + resolver. Keep plural raw-source discovery separate; do not collapse different + cardinality semantics into one ambiguous helper. +- Remove duplicate singleton-selection code and stale error wording. + +**Tests and exit criteria:** Cover empty/exact/different/relative-equivalent paths, +no output, multiple transcripts, unsafe paths, adapter conformance, and plural raw +sources. No adapter-selected alternate path may become authoritative. + +## Stage 26 — Centralize typed extraction-bundle evidence + +**Read first:** `audit-findings.md` lines 3538–3568 (DUP-007) and 3907–3932 +(SIM-002, merged into DUP-007), plus extraction artifact and resume/catalog docs. + +**Depends on:** Stages 2, 4, and 7. + +**Goal:** Evaluate extraction-bundle evidence once while preserving the different +policy decisions made by resume and catalog consumers. + +**Work:** + +- Add an artifact-owned typed proof/result that classifies extraction evidence as + valid, absent, obsolete, or unsafe and carries only verified metadata/paths. +- Centralize structural and filesystem proof: expected files, manifest/bundle + coherence, safe root-relative regular files, versions, and existing identity + evidence. Do not encode resume/catalog policy into the proof. +- Make resume and catalog map the same proof to their own documented outcomes. + Remove duplicate validators and the merged SIM-002 opportunity for a broad + generic resolver. + +**Tests and exit criteria:** Use one shared evidence fixture matrix for both +consumers and assert their intentionally different outcomes for every typed state. +Include unsafe root, symlink, incomplete, obsolete-version, and valid cases. + +## Stage 27 — Bind extraction reuse to direct transcript identity + +**Read first:** `audit-findings.md` lines 2380–2423 (COR-020) and 3744–3774 +(TST-009); extraction fingerprint, artifact evidence, direct-input, and resume +docs. + +**Depends on:** Stages 25–26. + +**Goal:** Never reuse extraction output after the direct transcript input changes +while keeping the external producer's transitive dependencies explicit. + +**Work:** + +- Extend the extraction fingerprint with a deterministic checksum of the exact + trimmed direct transcript bytes and stable source/producer identity needed by + the contract. +- Recompute that direct-input identity on every resume decision and feed it into + the typed Stage 26 evidence proof. A mismatch is obsolete evidence, not success. +- Keep Notarius/external transitive dependency freshness under the existing + explicit `--force` contract unless stable producer evidence is available; do not + invent a false dependency graph. +- Ensure extraction failure or obsolescence does not leave newly valid-looking + catalog/manifest entries. + +**Tests and exit criteria:** Add unchanged/changed/trimmed transcript cases, +producer/source changes, resume/restart, force, unsafe root, over-limit lane +result, and failure-before-publication tests. This is the primary TST-009 stage. + +## Stage 28 — Establish one effective artifact set and catalog bootstrap + +**Read first:** `audit-findings.md` lines 2456–2491 (COR-022), 3331–3362 +(ARC-007), 3569–3595 (DUP-008), and 3775–3804 (TST-010); analyze, extraction +catalog, artifact selection, configuration, and publish-filter docs. + +**Depends on:** Stages 21 and 26–27. + +**Goal:** Resolve artifact selection once into a typed effective set used by every +analyze precondition and input-composition path. + +**Work:** + +- Define one typed effective artifact set. Explicit CLI/stage selection overrides + configured defaults even when the selected artifact is disabled by default. +- Use that exact set for validation, prerequisite planning, extraction catalog + loading, prompt/input composition, and analyze execution. Keep publication + filtering a separate typed decision. +- Centralize only deterministic catalog definitions/bootstrap ordering in the + artifact owner. Keep caller-specific missing/optional/error policy outside the + shared bootstrap. +- Remove duplicate catalog lists and boolean mixtures that can diverge. + +**Tests and exit criteria:** Table-test configured defaults versus explicit +selection, disabled overrides, empty selections, ordering, catalog versions, +publication filters, and the exact composed analyze inputs. This is the primary +TST-010 stage; Stage 29 adds resolution/error cases. + +## Stage 29 — Make analyze resolution typed, optional, actionable, and deterministic + +**Read first:** `audit-findings.md` lines 2424–2455 (COR-021), 2492–2520 +(COR-023), 3033–3063 (RSK-015), 3933–3961 (SIM-003), and 3775–3804 (TST-010); +analyze, stage command, artifact dependency, and troubleshooting docs. + +**Depends on:** Stage 28. + +**Goal:** Give analyze one typed resolution/execution plan with correct optional +semantics, deterministic diagnostics, and executable remediation guidance. + +**Work:** + +- Represent built-in artifact resolution as typed present/absent/error results. + Optional absence is omitted; required absence returns a stable actionable error. +- Build one analyze execution context/effective plan from the Stage 28 artifact + set and typed resolutions. Do not add a generic resolver unrelated to analyze. +- Sort aggregated missing/dependency diagnostics by stable artifact/source keys. + Never expose map iteration order. +- Correct remediation text to the real command form: + `narratio run-stage prepare --force`, with context-specific + alternatives only when they are actually valid. + +**Tests and exit criteria:** Cover optional missing built-ins, required missing, +mixed custom/built-in artifacts, repeated randomized map insertion, exact guidance, +context construction, and final prompt inputs. Run deterministic tests repeatedly. + +## Stage 30 — Remove misleading contracts and align Audita ownership + +**Read first:** `audit-findings.md` lines 3198–3228 (ARC-001), 3280–3305 +(ARC-005), 3962–3979 (SIM-004), 4021–4037 (COM-003), and 4085–4101 +(COM-006); adapter, stage API, Audita, extraction source, and developer docs. + +**Depends on:** Stages 23–29. + +**Goal:** Remove dead or misleading surfaces after the behavioral contracts have +stabilized and place static Audita policy in its constructor. + +**Work:** + +- Remove the unused `IODecl`/`Declares` surface and update tests/docs rather than + preserving a compatibility shim for an internal unused contract. +- Move static Audita settings to constructor/configuration ownership. Per-run + requests carry only run-specific paths, modules, and execution data. Update the + fake and all call sites. +- Delete dead helpers made obsolete by the remediation. Do not delete a narrow + helper merely because it has one caller when it still owns a real invariant. +- Remove the stale Audita TODO and add a rationale comment only if ownership is not + self-evident. Add the extraction-source doc comment required by COM-006. + +**Tests and exit criteria:** Compile all adapters and fakes, run their focused +tests, search for removed symbols/TODOs/stale request fields, and confirm public +package documentation matches the remaining contract. + +## Stage 31 — Enforce CI validation and streamline the assembled test suite + +**Read first:** `audit-findings.md` lines 3822–3836 (TST-012) and 3869–3885 +(TST-015); testing policy, development workflow, Woodpecker configuration, and +release docs. + +**Depends on:** All production stages, especially Stage 23. + +**Goal:** Make ordinary automated validation unavoidable and consolidate broad +stage matrices only after the focused regression protections exist. + +**Work:** + +- Make every change and release path run ordinary tests, `go vet`, `go build`, and + documentation/example validation. A release must consume a revision that passed + the required validation, not merely run alongside an optional test pipeline. +- Run the full `go test -race ./...` suite on every change now that TST-001 is + fixed. Add scheduled shuffled/repeated execution for order dependence. +- Preserve focused owner tests from prior stages, then consolidate redundant broad + stage matrices as directed by TST-015. Do not reduce behavior coverage or hide + platform cases in an opaque mega-test. +- Add native macOS/Windows jobs only when authoritative runner labels and + successful native runners are available. Until then, retain explicit + cross-build checks and document the native evidence gap without inventing + labels or marking cross-compilation as equivalent. + +**Tests and exit criteria:** Validate Woodpecker syntax and dependency graph, +exercise the local equivalents of every job, prove failure propagation to release +eligibility, and inspect the final suite for duplicated matrices and lost cases. + +## Stage 32 — Reconcile lifecycle/analyze documentation and close the remediation + +**Read first:** `audit-findings.md` lines 3229–3255 (ARC-002, merged into +COM-002), 3998–4020 (COM-002), 4038–4055 (COM-004, merged into COM-002), and +4056–4084 (COM-005); all current-behavior policy and package documents modified by +the preceding stages. + +**Depends on:** Stages 1–31. + +**Goal:** Make the repository's normative documentation describe the implemented +architecture exactly and mechanically verify that every actionable audit finding +has a completed primary stage. + +**Work:** + +- Document the distinction between successful no-output, optional absence, and a + stage's explicit self-skip. Keep the production lifecycle semantics implemented + in earlier stages; do not add a new lifecycle abstraction for COM-002. +- Reconcile extraction catalog schema, effective selection/override semantics, + analyze consumption, optional absence, force/freshness rules, remote commit and + compatibility behavior, restore recovery, collaborative modes, secret handling, + output limits, and operations quota guidance. +- Review examples, troubleshooting commands, package docs, diagrams, and comments + against the current code. Remove superseded statements rather than layering new + caveats over them. +- Verify the traceability inventory below. Every confirmed actionable finding must + have one completed primary stage; merged/rejected findings must retain their + recorded disposition; accepted risks must not have been silently expanded. + +**Tests and exit criteria:** Run the full Stage 31 validation locally, run all +documentation/example checks, verify clean generated/example state, inspect the +final diff and repository status, and mark this stage complete only when no +required remediation remains. + +## Finding traceability inventory + +This inventory assigns exactly one primary implementation stage to each confirmed +actionable finding. A stage may read or test a finding owned by another stage when +the behavior crosses a boundary, but it must not create a second competing fix. + +| Finding(s) | Primary stage | +| --- | ---: | +| COR-001 | 11 | +| COR-002 | 2 | +| COR-003 | 4 | +| COR-004 | 14 | +| COR-005 | 10 | +| COR-006, COR-007 | 16 | +| COR-008, COR-009 | 17 | +| COR-010 | 20 | +| COR-011 | 14 | +| COR-012, COR-013, COR-014, COR-015 | 21 | +| COR-016 | 23 | +| COR-017, COR-018, COR-019 | 24 | +| COR-020 | 27 | +| COR-021, COR-023 | 29 | +| COR-022 | 28 | +| COR-024 | 22 | +| RSK-001 | 12 | +| RSK-002 | 3 | +| RSK-003 | 5 | +| RSK-004 | 1 | +| RSK-005 | 15 | +| RSK-006, RSK-008 | 18 | +| RSK-007 | 19 | +| RSK-009 | 22 | +| RSK-010 | 6 | +| RSK-011 | 8 | +| RSK-012 | 9 | +| RSK-013 | 7 | +| RSK-014 | 15 | +| RSK-015 | 29 | +| EFF-001 | 20 | +| EFF-002 | 23 | +| ARC-001 | 30 | +| ARC-003 | 13 | +| ARC-004 | 22 | +| ARC-005 | 30 | +| ARC-006 | 25 | +| ARC-007 | 28 | +| TST-001 | 23 | +| TST-002 | 12 | +| TST-003 | 4 | +| TST-004 | 14 | +| TST-005 | 17 | +| TST-006 | 11 | +| TST-007 | 7 | +| TST-008 | 24 | +| TST-009 | 27 | +| TST-010 | 28 | +| TST-011, TST-014 | 21 | +| TST-012, TST-015 | 31 | +| TST-013 | 2 | +| DUP-001, DUP-005 | 3 | +| DUP-002 | 14 | +| DUP-003 | 4 | +| DUP-006 | 25 | +| DUP-007 | 26 | +| DUP-008 | 28 | +| SIM-001 | 12 | +| SIM-003 | 29 | +| SIM-004 | 30 | +| COM-001 | 12 | +| COM-002, COM-005 | 32 | +| COM-003, COM-006 | 30 | + +The remaining confirmed IDs have non-independent dispositions and must not receive +separate implementation work: ARC-002 and COM-004 are merged into COM-002; +SIM-002 is merged into DUP-007; DUP-004 is rejected. Their audit rationale remains +authoritative. + +## Open Questions + +None. The governing decisions above make this plan decision-complete. Native +macOS/Windows CI activation remains contingent on external runner availability, +but the required implementation, cross-build checks, and rule against inventing +runner labels are already specified and do not require an architectural decision.