1152 lines
56 KiB
Markdown
1152 lines
56 KiB
Markdown
# Audit Remediation Implementation Plan
|
||
|
||
## Purpose and status
|
||
|
||
This document is the executable roadmap for remediating the confirmed findings in
|
||
[`audit-findings.md`](audit-findings.md). It is written for a `gpt-5.6-terra`
|
||
coding agent that will implement exactly one stage per prompt, in order.
|
||
|
||
The audit is an immutable requirements and evidence ledger. Do not edit
|
||
`audit-findings.md`, reinterpret accepted findings, or copy its detailed analysis
|
||
into this document. The line ranges below are stable references to the evidence
|
||
and acceptance criteria that the implementing agent must read for each stage.
|
||
|
||
This plan owns remediation progress; a separate `audit.md` tracker is not needed.
|
||
All stages are pending when this plan is created.
|
||
|
||
| Stage | Summary | Primary findings | Status |
|
||
| ---: | --- | --- | --- |
|
||
| 1 | Align data classification and group workspace modes | RSK-004 | Completed |
|
||
| 2 | Enforce safe identifiers and fuzz path/source contracts | COR-002, TST-013 | Completed |
|
||
| 3 | Consolidate crash-durable atomic file replacement | RSK-002, DUP-001, DUP-005 | Completed |
|
||
| 4 | Add confined destination and download/install capabilities | COR-003, DUP-003, TST-003 | Completed |
|
||
| 5 | Confine recursive cleanup and replace sentinel locks | RSK-003 | Completed |
|
||
| 6 | Harden API-key file acquisition | RSK-010 | Completed |
|
||
| 7 | Bound and verify external result acquisition | RSK-013, TST-007 | Completed |
|
||
| 8 | Terminate owned subprocess trees | RSK-011 | Completed |
|
||
| 9 | Redact and cap subprocess diagnostics | RSK-012 | Completed |
|
||
| 10 | Confine publish archive reads | COR-005 | Completed |
|
||
| 11 | Make manifest and run identity singular | COR-001, TST-006 | Completed |
|
||
| 12 | Centralize handled terminal-failure persistence | RSK-001, TST-002, SIM-001, COM-001 | Completed |
|
||
| 13 | Introduce the immutable remote-commit model and legacy boundary | ARC-003 | Completed |
|
||
| 14 | Publish through immutable commits and canonical mappings | COR-004, COR-011, DUP-002, TST-004 | Completed |
|
||
| 15 | Make remote locks generation-safe and harden pagination | RSK-005, RSK-014 | Completed |
|
||
| 16 | Persist retryable post-commit cleanup state | COR-006, COR-007 | Completed |
|
||
| 17 | Bind restore/status to a committed snapshot and reject conflicts | COR-008, COR-009, TST-005 | Completed |
|
||
| 18 | Serialize restore transitions and make restored paths portable | RSK-006, RSK-008 | Completed |
|
||
| 19 | Bind audio cache reuse to remote object identity | RSK-007 | Completed |
|
||
| 20 | Unify previous-source readiness and eliminate duplicate transfers | COR-010, EFF-001 | Completed |
|
||
| 21 | Tighten configuration parsing, values, and expectations | COR-012–COR-015, TST-011, TST-014 | Pending |
|
||
| 22 | Make product configuration truthful and own remote temp files | COR-024, RSK-009, ARC-004 | Pending |
|
||
| 23 | Stream WhisperX uploads and make the adapter race-safe | COR-016, EFF-002, TST-001 | Pending |
|
||
| 24 | Correct prepare/transcribe transition semantics | COR-017–COR-019, TST-008 | Pending |
|
||
| 25 | Enforce output-path authority and shared singleton resolution | ARC-006, DUP-006 | Pending |
|
||
| 26 | Centralize typed extraction-bundle evidence | DUP-007 | Pending |
|
||
| 27 | Bind extraction reuse to direct transcript identity | COR-020, TST-009 | Pending |
|
||
| 28 | Establish one effective artifact set and catalog bootstrap | COR-022, ARC-007, DUP-008 | Pending |
|
||
| 29 | Make analyze resolution typed, optional, actionable, and deterministic | COR-021, COR-023, RSK-015, SIM-003, TST-010 | Pending |
|
||
| 30 | Remove misleading contracts and align Audita ownership | ARC-001, ARC-005, SIM-004, COM-003, COM-006 | Pending |
|
||
| 31 | Enforce CI validation and streamline the assembled test suite | TST-012, TST-015 | Pending |
|
||
| 32 | Reconcile lifecycle/analyze documentation and close the remediation | COM-002, COM-005 | Pending |
|
||
|
||
## Governing decisions
|
||
|
||
The following decisions are settled requirements, not questions for the
|
||
implementing agent:
|
||
|
||
1. Ordinary Narratio and Notarius campaign/session data is deliberately
|
||
shareable, not private or sensitive. Group-owned, group-writable workspaces are
|
||
an operational requirement. API keys are the only sensitive data handled by
|
||
these applications and must remain private.
|
||
2. On POSIX systems, Narratio-managed ordinary workspace directories and files
|
||
should converge on setgid `02775` and `0664` respectively, inheriting the
|
||
workspace's existing group. API-key directories and files must be `0700` and
|
||
`0600`. Do not add ownership-changing behavior or assume Narratio may `chown`.
|
||
Windows behavior must preserve the same collaboration/security intent using
|
||
the platform's available guarantees and explicit operational guidance.
|
||
3. Remote current-state publication uses an immutable run-scoped commit manifest
|
||
selected by one final pointer update. Readers may accept coherent legacy state,
|
||
but all compatibility code must be isolated behind a clearly named boundary,
|
||
covered by dedicated tests, and carry an in-code removal comment. New writers
|
||
must never produce the legacy format.
|
||
4. Explicit notification no-op is supported. Configured notification backends or
|
||
recipients are rejected until a real provider exists.
|
||
5. External-output limits are runaway safeguards, not normal operating limits.
|
||
Each limit must be a named, centrally discoverable package constant and at
|
||
least an order of magnitude above expected ordinary output. Every limit error
|
||
must identify the enforcing adapter/contract, the configured byte limit, and
|
||
what exceeded it. Narratio-owned stdout/stderr capture is streaming and capped;
|
||
reaching its cap terminates the owned process tree. Hard whole-disk protection
|
||
is an operations concern implemented with filesystem/service/container/volume
|
||
quotas, not output-root polling.
|
||
6. Retain every accepted-risk disposition in the audit. In particular, do not
|
||
turn abrupt process-death detection into a distributed liveness system, do not
|
||
add rollback of arbitrary external side effects, and do not add generic
|
||
speculative abstractions rejected by the audit.
|
||
7. CI must run ordinary tests, vet, build, and example checks on every change and
|
||
release path; the complete race suite runs on every change once TST-001 is
|
||
repaired; shuffled tests run on a schedule. Add native macOS/Windows jobs only
|
||
when authoritative Woodpecker runner labels and usable runners are available.
|
||
Do not invent labels. Cross-compilation is useful but does not count as native
|
||
filesystem/process evidence.
|
||
|
||
## Instructions for every stage
|
||
|
||
For each implementation prompt, the coding agent must:
|
||
|
||
1. Read `docs/development.md`, then the exact audit ranges and canonical documents
|
||
named by that stage. Follow the task-specific reading guide in
|
||
`docs/development.md`. Do not read all 4,000+ audit lines unless a stage
|
||
explicitly requires them.
|
||
2. Inspect the current tree before editing. Earlier stages may have changed names
|
||
and ownership boundaries. Use the repository knowledge graph first for code
|
||
discovery and call tracing, then use text search for documentation, string
|
||
literals, configuration, and evidence the graph cannot supply.
|
||
3. Confirm the worktree state and preserve unrelated user changes. Do not rewrite
|
||
or amend earlier commits unless explicitly instructed.
|
||
4. Implement the stage completely, including production code, focused regression
|
||
tests, platform-specific implementations where applicable, current-behavior
|
||
documentation, and removal of code made obsolete by the stage. Do not leave a
|
||
second competing path or defer required tests to a later stage unless this plan
|
||
explicitly says so.
|
||
5. Prefer typed, owner-specific contracts and existing package ownership. Do not
|
||
create a generic abstraction merely because two call sites look similar. Keep
|
||
policy at the owner and mechanism in `internal/fileops` or the relevant
|
||
adapter/package.
|
||
6. Treat path/race/security checks as production behavior. Validation followed by
|
||
an unconfined pathname operation is not a completed fix. Unsupported safe
|
||
behavior must fail closed with an actionable error.
|
||
7. Run focused tests while developing, then at minimum `go test ./...` for every
|
||
stage. For production-code changes also run `go vet ./...`, `go build ./...`,
|
||
and the repository's documentation/example validation command. Run targeted
|
||
`-race` tests for concurrency changes and platform build/test checks named by
|
||
the stage. Do not require live cloud or third-party services.
|
||
8. Compare the final diff with the stage's finding list and exit criteria. Update
|
||
only that stage's row in the table from `Pending` to `Completed`, adding the
|
||
implementing commit hash if the workflow supplies one. Do not mark a stage
|
||
complete while a required check is failing or a required behavior is missing.
|
||
9. Leave `docs/roadmap/audit-findings.md` unchanged. If implementation evidence
|
||
reveals that a confirmed finding is factually impossible to satisfy as written,
|
||
stop and report the contradiction instead of silently changing its meaning.
|
||
|
||
## Stage 1 — Align data classification and group workspace modes
|
||
|
||
**Read first:** `audit-findings.md` lines 2660–2693 (RSK-004), 4171–4224
|
||
(resolved decisions and accepted risks); `docs/policy/architecture.md`;
|
||
`docs/operations.md`; and the workspace/artifact/file-operation package docs named
|
||
by `docs/development.md`.
|
||
|
||
**Goal:** Make the architecture and implementation consistently describe and
|
||
support shareable, group-owned workspaces while keeping API keys outside that
|
||
ordinary-data policy.
|
||
|
||
**Work:**
|
||
|
||
- Revise the architecture policy's current-behavior contract: campaign/session,
|
||
manifests, transcripts, generated configs, logs, and Notarius artifacts are
|
||
intentionally shareable; API-key material is sensitive.
|
||
- Identify every Narratio-owned ordinary workspace creation/promotion path. Give
|
||
the owning packages named, discoverable mode constants and converge final POSIX
|
||
directories/files to setgid `02775`/`0664`, without changing ownership.
|
||
- Preserve the parent workspace group through nested creation and atomic
|
||
replacement. Account for restrictive caller umasks explicitly; do not rely on a
|
||
favorable umask to add group write or setgid bits.
|
||
- Keep temporary-file exposure no broader than the intended final object and do
|
||
not weaken the separate API-key rules implemented in Stage 6.
|
||
- Document Windows ACL/ownership expectations and any mode-bit limitations; do
|
||
not claim POSIX mode semantics on Windows.
|
||
|
||
**Tests and exit criteria:** Add focused tests under restrictive umasks for nested
|
||
directories, final files, promoted Notarius trees, and repeated replacement.
|
||
Verify that group-write/setgid behavior is correct on POSIX and platform behavior
|
||
is explicit elsewhere. The architecture, operations, and security statements must
|
||
agree. No test should assert that ordinary campaign output is private.
|
||
|
||
## Stage 2 — Enforce safe identifiers and fuzz path/source contracts
|
||
|
||
**Read first:** `audit-findings.md` lines 1770–1810 (COR-002) and 3837–3852
|
||
(TST-013); configuration, manifest, workspace, artifact, and source-grammar docs.
|
||
|
||
**Depends on:** Stage 1.
|
||
|
||
**Goal:** Prevent externally influenced identity tokens from becoming absolute,
|
||
traversing, platform-ambiguous, or control-bearing filesystem segments.
|
||
|
||
**Work:**
|
||
|
||
- Define one strict portable segment grammar for campaign IDs, session IDs, run
|
||
IDs, artifact identifiers, and other opaque filesystem identity tokens: ASCII
|
||
letters, digits, `.`, `_`, and `-` only; reject empty, `.`/`..`, separators,
|
||
drive/volume syntax, absolute forms, control characters, and all other bytes.
|
||
- Enforce the contract at configuration/model construction and again at the
|
||
artifact/path boundary. Return actionable errors; never sanitize or silently
|
||
rewrite identity.
|
||
- Audit manifest/source parsing for equivalent tokens. Unsafe legacy values must
|
||
fail with migration guidance, not be accepted because an older file contains
|
||
them.
|
||
- Preserve intentional multi-segment paths only in typed path contracts that are
|
||
separately confined; do not force path-valued fields through the opaque-ID API.
|
||
|
||
**Tests and exit criteria:** Add table and fuzz tests for Unix/Windows separators,
|
||
drive forms, Unicode lookalikes, controls, empty/dot segments, and valid boundary
|
||
values. Include source-token parsers as required by TST-013. Prove all path builders
|
||
reject unsafe identifiers before filesystem access.
|
||
|
||
## Stage 3 — Consolidate crash-durable atomic file replacement
|
||
|
||
**Read first:** `audit-findings.md` lines 2590–2624 (RSK-002), 3407–3427
|
||
(DUP-001), and 3493–3514 (DUP-005); `internal/fileops` and manifest persistence
|
||
docs.
|
||
|
||
**Depends on:** Stages 1–2.
|
||
|
||
**Goal:** Provide one owner-neutral byte-file installation mechanism with explicit
|
||
durability semantics, while leaving serialization and policy with callers.
|
||
|
||
**Work:**
|
||
|
||
- Implement or complete a narrowly scoped `internal/fileops` byte-write/replace
|
||
primitive: sibling temporary file, complete write, file sync, atomic rename or
|
||
documented platform replacement, then parent-directory sync where supported.
|
||
- Accept the caller-owned final mode; make cleanup and error composition explicit.
|
||
Preserve cancellation checkpoints in callers before durable commit.
|
||
- Migrate duplicate local manifest, artifact, and adapter byte-write helpers to the
|
||
primitive. Keep manifest encoding/versioning out of `fileops`.
|
||
- Define and test the platform contract for directory sync and replacement. A
|
||
platform limitation must be explicit, not silently treated as equivalent.
|
||
|
||
**Tests and exit criteria:** Exercise short writes, write/sync/rename/directory-sync
|
||
failures through injected seams, repeated replacement, mode preservation, and
|
||
temporary cleanup. Existing manifest and adapter behavior must remain compatible.
|
||
No duplicate sibling-temp byte writer should remain.
|
||
|
||
## Stage 4 — Add confined destination and download/install capabilities
|
||
|
||
**Read first:** `audit-findings.md` lines 1811–1852 (COR-003), 3448–3471
|
||
(DUP-003), and 3596–3617 (TST-003); filesystem threat model and `internal/fileops`
|
||
docs.
|
||
|
||
**Depends on:** Stages 2–3.
|
||
|
||
**Goal:** Make destination mutation resistant to ancestor and leaf symlink races,
|
||
and share the safe remote-download installation mechanism without sharing caller
|
||
policy.
|
||
|
||
**Work:**
|
||
|
||
- Add root-relative, handle-confined creation, replacement, and promotion
|
||
capabilities that verify/traverse every ancestor without following symlinks.
|
||
Use platform-specific implementations where necessary and fail closed if the
|
||
required guarantee is unavailable.
|
||
- Replace validate-then-use pathname mutations in artifact creation, manifest
|
||
installation, Notarius tree promotion, restore/download installation, and
|
||
related destinations.
|
||
- Implement one destination-confined sibling-temp download/install mechanism.
|
||
Callers continue to own object identity, verification, force/conflict policy,
|
||
and final mode.
|
||
- Do not implement recursive deletion here; Stage 5 owns that distinct capability.
|
||
|
||
**Tests and exit criteria:** Add adversarial tests that replace intermediate
|
||
directories and leaves with symlinks between logical phases, plus regular success,
|
||
collision, cleanup, and platform tests. Prove no external target is modified.
|
||
TST-003 is primarily owned by this stage and may receive additional cleanup cases
|
||
in Stage 5.
|
||
|
||
## Stage 5 — Confine recursive cleanup and replace sentinel locks
|
||
|
||
**Read first:** `audit-findings.md` lines 1811–1852 (COR-003), 2625–2659
|
||
(RSK-003), and 3596–3617 (TST-003); local locking and workspace lifecycle docs.
|
||
|
||
**Depends on:** Stage 4.
|
||
|
||
**Goal:** Make deletion root-confined and make local crash recovery rely on
|
||
OS-released locks rather than stale file existence.
|
||
|
||
**Work:**
|
||
|
||
- Add a root-relative, no-follow cleanup/delete capability and migrate recursive
|
||
cleanup paths. Never pass an unchecked or re-resolved absolute path to broad
|
||
recursive deletion.
|
||
- Replace existence-based sentinel locking with a held OS lock whose ownership is
|
||
released by the kernel on process death. Retain useful owner/run metadata
|
||
without using metadata-file existence as the lock decision.
|
||
- Implement explicit Unix and Windows lock backends, wait/contention behavior,
|
||
cancellation, and release-error reporting. Do not auto-delete another process's
|
||
active lock.
|
||
- Ensure lock files and cleanup roots follow the group workspace policy where they
|
||
are ordinary runtime data.
|
||
|
||
**Tests and exit criteria:** Cover contention across independent processes,
|
||
process-death recovery, cancellation, release failures, symlink swaps, malicious
|
||
cleanup inputs, and idempotent cleanup. Run native tests where available and at
|
||
least cross-build all platform-specific files.
|
||
|
||
## Stage 6 — Harden API-key file acquisition
|
||
|
||
**Read first:** `audit-findings.md` lines 2870–2900 (RSK-010); architecture,
|
||
configuration, and integration docs governing API keys.
|
||
|
||
**Depends on:** Stages 1, 2, and 4.
|
||
|
||
**Goal:** Make the sole sensitive-data path private, bounded, regular-file-only,
|
||
and resistant to path races.
|
||
|
||
**Work:**
|
||
|
||
- Resolve API-key directories and files through a root-confined, no-follow read
|
||
path. Reject symlinks, non-regular files, unsafe relative names, replacement
|
||
races, and oversized content.
|
||
- Define the small API-key byte limit as a named, discoverable constant in the
|
||
owning package. Errors must name the API-key loader and the limit.
|
||
- Enforce `0700` directory and `0600` file privacy for Narratio-created secret
|
||
storage on POSIX, and reject unsafe existing modes where Narratio reads key
|
||
files. Document Windows ownership/ACL requirements without pretending mode bits
|
||
are authoritative there.
|
||
- Preserve documented environment/file precedence without persisting or logging
|
||
secret values.
|
||
|
||
**Tests and exit criteria:** Cover mode rejection, symlink/ancestor swaps,
|
||
non-regular files, over-limit reads, valid newline trimming, precedence, and error
|
||
redaction. No API-key content may appear in failures or fixtures intended for logs.
|
||
|
||
## Stage 7 — Bound and verify external result acquisition
|
||
|
||
**Read first:** `audit-findings.md` lines 2963–3006 (RSK-013), 3691–3718
|
||
(TST-007), and 3744–3774 (TST-009); adapter contracts and integration docs.
|
||
|
||
**Depends on:** Stages 4 and 6.
|
||
|
||
**Goal:** Prevent a runaway adapter or producer from forcing unbounded memory or
|
||
post-process disk consumption while leaving abundant headroom for real sessions.
|
||
|
||
**Work:**
|
||
|
||
- Factor the non-secret portions of the Stage 6 read mechanism into an
|
||
owner-neutral bounded, regular-file, no-follow acquisition primitive if that
|
||
does not already exist. Keep each contract's limit in its owning package.
|
||
- Inventory every externally produced file Narratio reads after a process exits:
|
||
adapter JSON, Audita output, Scriptorium output, WhisperX result, Notarius lane
|
||
result/catalog/bundle evidence, and any equivalent stage handoff. Migrate all of
|
||
them to bounded verified acquisition.
|
||
- Set generous documented constants at least ten times larger than credible
|
||
ordinary output. Do not embed numeric byte limits in call sites or hide them in
|
||
a generic helper.
|
||
- Every over-limit error must state the responsible adapter or producer contract,
|
||
the configured limit, and the file/result category. Never return partial parsed
|
||
output.
|
||
- Document that hard disk exhaustion is prevented operationally with filesystem,
|
||
service, container, or volume quotas; do not add polling of output-root size.
|
||
|
||
**Tests and exit criteria:** Add exact-limit and limit-plus-one tests, growth/race
|
||
cases, non-regular and symlink cases, malformed/truncated results, and clear error
|
||
assertions for each adapter family. TST-007 is primarily owned here; later adapter
|
||
stages add their specialized liveness cases.
|
||
|
||
## Stage 8 — Terminate owned subprocess trees
|
||
|
||
**Read first:** `audit-findings.md` lines 2901–2931 (RSK-011); adapter process and
|
||
platform-support docs.
|
||
|
||
**Depends on:** Stage 7.
|
||
|
||
**Goal:** Ensure cancellation, timeout, and enforced output limits stop and reap
|
||
the complete subprocess tree Narratio owns.
|
||
|
||
**Work:**
|
||
|
||
- Introduce a narrow platform-specific process-runner capability that starts an
|
||
owned process in a Unix process group or Windows job object and can terminate
|
||
and reap the tree.
|
||
- Migrate Notarius, Audita, Scriptorium, WhisperX, and other owned external command
|
||
execution to it without broadening environment inheritance.
|
||
- Define graceful-then-forceful cancellation behavior, bounded waits, error
|
||
composition, and already-exited handling. Ensure descendants cannot retain
|
||
pipes indefinitely after the leader exits.
|
||
|
||
**Tests and exit criteria:** Use helper processes that spawn descendants and hold
|
||
stdout/stderr open. Prove cancellation/timeout kills descendants and reaps the
|
||
leader. Add native Unix/Windows tests where runners exist and cross-build all
|
||
platform code; record no unsupported fallback as safe.
|
||
|
||
## Stage 9 — Redact and cap subprocess diagnostics
|
||
|
||
**Read first:** `audit-findings.md` lines 2932–2962 (RSK-012) and 2963–3006
|
||
(RSK-013); logging, adapter, and operations docs.
|
||
|
||
**Depends on:** Stages 6–8.
|
||
|
||
**Goal:** Make persisted stdout/stderr and diagnostic tails bounded and unable to
|
||
disclose known credentials.
|
||
|
||
**Work:**
|
||
|
||
- Define an explicit set of sensitive environment-variable names and pass their
|
||
values only to a streaming redaction layer; never persist the values or include
|
||
them in error text.
|
||
- Minimize inherited environments for external processes while preserving the
|
||
documented variables needed for execution.
|
||
- Stream stdout and stderr through redaction into capped diagnostic files using
|
||
named, centrally discoverable limits in the owning process/adapter package.
|
||
Redaction must work when a secret spans write chunks. Diagnostic tails must be
|
||
derived from already-redacted content.
|
||
- If either capture limit is reached, terminate and reap the process tree through
|
||
Stage 8 and return an unmistakable error naming stdout/stderr, the responsible
|
||
adapter, and the byte limit. Do not report successful partial execution.
|
||
|
||
**Tests and exit criteria:** Cover exact cap, cap-plus-one, split-token secrets,
|
||
overlapping secret values, empty values, tails, cancellation, descendant cleanup,
|
||
and on-disk inspection. Assert that neither raw nor transformed error paths expose
|
||
test credentials.
|
||
|
||
## Stage 10 — Confine publish archive reads
|
||
|
||
**Read first:** `audit-findings.md` lines 1891–1924 (COR-005); publish, artifact,
|
||
manifest, and storage docs.
|
||
|
||
**Depends on:** Stages 2, 4, and 7.
|
||
|
||
**Goal:** Ensure publish uploads exactly the verified regular files inside the
|
||
selected run root, despite symlink or replacement races.
|
||
|
||
**Work:**
|
||
|
||
- Enumerate publish candidates through the artifact/run manifest rather than an
|
||
untrusted tree walk.
|
||
- Open each candidate root-relatively without following ancestors or the leaf;
|
||
verify regular-file type and any declared identity immediately on the opened
|
||
object, then stream from that verified handle.
|
||
- Reject undeclared files, directories, devices, sockets, symlinks, and identity
|
||
changes. Preserve deterministic ordering and actionable source-relative errors.
|
||
- Keep object-store destination policy out of the filesystem primitive.
|
||
|
||
**Tests and exit criteria:** Add regular, leaf-symlink, ancestor-swap,
|
||
post-enumeration replacement, non-regular, and undeclared-file cases. A hostile
|
||
workspace must never cause bytes outside the run root to be uploaded.
|
||
|
||
## Stage 11 — Make manifest and run identity singular
|
||
|
||
**Read first:** `audit-findings.md` lines 1731–1769 (COR-001) and 3670–3690
|
||
(TST-006); configuration, environment, run-state, and manifest docs.
|
||
|
||
**Depends on:** Stages 2–3.
|
||
|
||
**Goal:** Use one validated campaign/session/run identity per invocation and
|
||
recompute all run-scoped manifest fields together.
|
||
|
||
**Work:**
|
||
|
||
- At invocation start, resolve and validate one effective configuration and bind
|
||
the runtime environment to it; remove later reads from stale or unvalidated
|
||
configuration instances.
|
||
- On every run, recompute campaign, session, run ID, source references, and other
|
||
run-scoped manifest identity as a coherent set. Do not retain fields from the
|
||
prior invocation merely because a manifest object is reused.
|
||
- Reject disagreement among configured, CLI-selected, persisted, and manifest
|
||
campaign/session identity with an actionable error before side effects.
|
||
- Preserve intentionally session-scoped history separately from the current
|
||
run-scoped projection.
|
||
|
||
**Tests and exit criteria:** Add consecutive-run regression tests using one
|
||
process/environment, explicit mismatch tests for every authority pair, and TST-006
|
||
tests proving downstream operations receive the resolved configuration instance.
|
||
|
||
## Stage 12 — Centralize handled terminal-failure persistence
|
||
|
||
**Read first:** `audit-findings.md` lines 2553–2589 (RSK-001), 3388–3406
|
||
(TST-002), 3886–3906 (SIM-001), and 3980–3997 (COM-001); run/session lifecycle
|
||
docs.
|
||
|
||
**Depends on:** Stages 3 and 11.
|
||
|
||
**Goal:** Ensure every handled command failure terminalizes the authoritative
|
||
session record and best-effort diagnostic run record with correct compounded
|
||
errors.
|
||
|
||
**Work:**
|
||
|
||
- Introduce the narrow terminal-failure operation described by SIM-001: persist
|
||
the session's terminal failure first, then update the run diagnostic record.
|
||
Keep the running transition separate and run-first.
|
||
- Route every handled pipeline/stage error and cancellation path through it.
|
||
Return an error that preserves the operation failure plus any session/run store
|
||
disagreement; do not mask the primary failure or report success.
|
||
- Add the short rationale comment required by COM-001 explaining the deliberately
|
||
asymmetric two-ledger ordering.
|
||
- Keep abrupt process death as the accepted residual risk. Do not add heartbeats or
|
||
pretend a stale `running` record proves liveness.
|
||
|
||
**Tests and exit criteria:** Inject session-store and run-store failures at each
|
||
step, including cancellation and already-terminal records. Assert call order,
|
||
persisted truth, and composed errors. Remove duplicate terminalization sequences.
|
||
|
||
## Stage 13 — Introduce the immutable remote-commit model and legacy boundary
|
||
|
||
**Read first:** `audit-findings.md` lines 1853–1890 (COR-004), 1992–2037
|
||
(COR-008), and 3256–3279 (ARC-003); remote manifest, storage, restore, and
|
||
compatibility docs.
|
||
|
||
**Depends on:** Stages 2–3 and 11.
|
||
|
||
**Goal:** Define one versioned, immutable snapshot contract selected by a small
|
||
current pointer, without mixing local metadata or temporary compatibility into the
|
||
new model.
|
||
|
||
**Work:**
|
||
|
||
- Add a typed, versioned run-scoped remote commit manifest whose identity and
|
||
declared object set are immutable. Include campaign/session/run identity,
|
||
enough version/checksum/generation metadata to bind readers to the selected
|
||
snapshot, and an extensible typed source-to-destination artifact mapping.
|
||
- Define the final current pointer as the sole mutable selector of a commit. The
|
||
pointer must be small, parseable, and able to select exactly one immutable
|
||
commit; it is not a second mutable manifest.
|
||
- Separate local-only metadata from remote commit state. Remove or split
|
||
`current_pointer_written` as required by ARC-003; derive remote commitment from
|
||
the selected pointer/commit relationship rather than an independently mutable
|
||
boolean.
|
||
- Isolate coherent legacy-pair reading in a clearly named compatibility file or
|
||
package boundary with dedicated tests and an in-code comment stating that it is
|
||
temporary and removable after migration. Reject torn or ambiguous legacy pairs.
|
||
Do not add legacy writing.
|
||
|
||
**Tests and exit criteria:** Unit-test model validation, versions, unknown fields
|
||
per repository policy, pointer/commit mismatch, checksum/generation mismatch,
|
||
legacy coherent/torn cases, and the removable boundary. This stage may introduce
|
||
the model and readers without switching publish; Stage 14 switches the writer.
|
||
|
||
## Stage 14 — Publish through immutable commits and canonical mappings
|
||
|
||
**Read first:** `audit-findings.md` lines 1853–1890 (COR-004), 2068–2103
|
||
(COR-011), 3428–3447 (DUP-002), and 3618–3642 (TST-004); publish, artifact,
|
||
remote-manifest, and storage docs.
|
||
|
||
**Depends on:** Stages 10 and 13.
|
||
|
||
**Goal:** Make new publication an immutable-object transaction whose last remote
|
||
mutation selects the complete snapshot.
|
||
|
||
**Work:**
|
||
|
||
- Make the artifact package's canonical run-manifest path the only source used by
|
||
publish; remove the duplicate path construction identified by DUP-002.
|
||
- Before upload, build and validate the complete deterministic source-to-remote
|
||
destination mapping using exact artifact/source identity, not basename
|
||
heuristics. Reject collisions and ambiguity.
|
||
- Upload/verify every immutable data object and the immutable commit manifest
|
||
first. Update the current pointer exactly once, last. Never upload or mutate
|
||
current-state data after pointer selection.
|
||
- New publish writes only the new protocol. Preserve the isolated legacy reader
|
||
from Stage 13 solely for existing remote state.
|
||
- Make publish tests stateful enough to observe remote ordering, generations,
|
||
object contents, failure injection, and retry behavior.
|
||
|
||
**Tests and exit criteria:** Prove failures before the pointer leave the old
|
||
snapshot selected; successful publication selects only a complete new snapshot;
|
||
same-basename sources remain distinct; retries are deterministic; and no
|
||
post-pointer upload occurs. This is the primary TST-004 stage; Stages 15–16 extend
|
||
the same stateful fake.
|
||
|
||
## Stage 15 — Make remote locks generation-safe and harden pagination
|
||
|
||
**Read first:** `audit-findings.md` lines 2694–2731 (RSK-005), 3007–3032
|
||
(RSK-014), 3618–3642 (TST-004), and 3691–3718 (TST-007); storage adapter and
|
||
publish lock docs.
|
||
|
||
**Depends on:** Stage 14.
|
||
|
||
**Goal:** Make remote ownership effective immediately and replacement conditional,
|
||
and prevent non-progressing object-store pagination loops.
|
||
|
||
**Work:**
|
||
|
||
- Extend the object-store contract with the minimum conditional write/replace
|
||
metadata needed for generation/ETag-safe lock acquisition, renewal, and release.
|
||
Implement it in the S3 adapter and stateful fake without reducing provider
|
||
preconditions to a local check-then-write.
|
||
- Commit remote lock ownership immediately, retry conditional conflicts with
|
||
bounded/cancellable policy, and never release another owner's generation.
|
||
- Revalidate lock ownership at the protected publish commit point immediately
|
||
before the current pointer changes. Loss of ownership must leave the old
|
||
snapshot selected.
|
||
- In paginated listing, reject an empty or repeated continuation token when the
|
||
provider claims more results. Include adapter/bucket/prefix context without
|
||
secrets.
|
||
|
||
**Tests and exit criteria:** Add concurrent-owner, stale-generation,
|
||
renewal/release conflict, lock-loss-at-commit, cancellation, empty token, repeated
|
||
token, and normal multi-page tests. Run relevant tests under `-race`.
|
||
|
||
## Stage 16 — Persist retryable post-commit cleanup state
|
||
|
||
**Read first:** `audit-findings.md` lines 1925–1991 (COR-006 and COR-007);
|
||
publish, remote manifest, cleanup, and run/session lifecycle docs.
|
||
|
||
**Depends on:** Stages 5 and 12–15.
|
||
|
||
**Goal:** Make deletion after publication a durable, idempotent obligation that
|
||
does not disappear when execution state is reset or an attempt fails.
|
||
|
||
**Work:**
|
||
|
||
- Persist a typed cleanup obligation independently of the current executed-step
|
||
list before destructive cleanup can begin. Bind it to the committed run and
|
||
exact declared objects/paths.
|
||
- On retry/resume, detect and continue outstanding cleanup even when the current
|
||
stage list is empty or rebuilt. Keep cleanup idempotent and confined through
|
||
Stage 5.
|
||
- Persist completion evidence only after deletion has succeeded or an
|
||
already-absent target has been safely verified. A failed evidence write must
|
||
leave a retryable obligation rather than falsely completed state.
|
||
- Define command outcomes when publication committed but cleanup failed: the
|
||
committed snapshot remains authoritative, while the invocation clearly reports
|
||
incomplete cleanup and can retry it.
|
||
|
||
**Tests and exit criteria:** In the stateful publish fake, inject failures before,
|
||
during, and after deletion/evidence persistence; restart with empty execution
|
||
state; retry multiple times; and prove no unrelated target is deleted.
|
||
|
||
## Stage 17 — Bind restore/status to a committed snapshot and reject conflicts
|
||
|
||
**Read first:** `audit-findings.md` lines 1992–2067 (COR-008 and COR-009) and
|
||
3643–3669 (TST-005); restore, status, remote manifest, and workspace docs.
|
||
|
||
**Depends on:** Stages 4, 13–16.
|
||
|
||
**Goal:** Restore and status must observe one pointer-selected immutable snapshot,
|
||
and `--force` must never overwrite a directory conflict.
|
||
|
||
**Work:**
|
||
|
||
- Resolve the current pointer once, load and validate the selected immutable
|
||
commit, and use only its declared object set and generation/checksum metadata
|
||
throughout one restore/status operation.
|
||
- Do not reconstruct current truth from mutable prefixes or a mixture of runs.
|
||
Route coherent legacy state through only the isolated Stage 13 compatibility
|
||
reader.
|
||
- Define conflict types before mutation. `--force` may replace eligible regular
|
||
managed files but must reject directory/non-regular conflicts, leaving no
|
||
completed local manifest and no false success.
|
||
- Make the restore fake stateful and able to mutate remote objects/pointers between
|
||
calls so snapshot-binding failures are observable.
|
||
|
||
**Tests and exit criteria:** Cover pointer changes during restore, undeclared and
|
||
missing objects, version/generation mismatch, coherent legacy restore, file and
|
||
directory conflicts with/without force, partial download, and manifest absence on
|
||
failure. This is the primary TST-005 stage; Stages 18–20 extend it.
|
||
|
||
## Stage 18 — Serialize restore transitions and make restored paths portable
|
||
|
||
**Read first:** `audit-findings.md` lines 2732–2769 (RSK-006) and 2803–2838
|
||
(RSK-008), plus TST-005 at lines 3643–3669; restore, local lock, manifest, and
|
||
runner reuse docs.
|
||
|
||
**Depends on:** Stages 5 and 17.
|
||
|
||
**Goal:** Prevent runner/restore races and ensure restored manifests cannot retain
|
||
unsafe producer-machine absolute paths.
|
||
|
||
**Work:**
|
||
|
||
- Acquire the session/workspace lock before executable-state classification and
|
||
revalidate state under the lock. All runner reuse checks must participate in the
|
||
same transition protocol.
|
||
- Persist a durable incomplete-restore marker before the first local mutation.
|
||
Keep it blocking runner reuse after a partial forced restore; clear it only after
|
||
all verified installs and the completed local manifest are durable.
|
||
- Rebase restored artifact paths from portable logical references to the selected
|
||
local root. Reject absolute paths outside that root and unsafe relative paths.
|
||
Retain an original absolute path, if useful, only as bounded non-authoritative
|
||
provenance.
|
||
- Do not attempt rollback of arbitrary external effects; the durable incomplete
|
||
marker and safe retry are the recovery contract.
|
||
|
||
**Tests and exit criteria:** Add concurrent runner/restore process tests,
|
||
classification races, forced partial restore/restart/retry, Unix/Windows absolute
|
||
path forms, traversal, safe rebasing, and provenance bounds.
|
||
|
||
## Stage 19 — Bind audio cache reuse to remote object identity
|
||
|
||
**Read first:** `audit-findings.md` lines 2770–2802 (RSK-007) and TST-005 at
|
||
lines 3643–3669; previous-audio, object-store, and restore docs.
|
||
|
||
**Depends on:** Stages 4, 15, and 17–18.
|
||
|
||
**Goal:** Reuse restored/downloaded audio only when it is a verified regular file
|
||
for the selected remote object version.
|
||
|
||
**Work:**
|
||
|
||
- Persist a sidecar or equivalent typed cache record binding local audio to remote
|
||
object key plus authoritative version/generation/checksum information.
|
||
- Verify the local candidate root-relatively as a regular no-follow file and
|
||
validate size/digest as required before reuse. Do not treat an unqualified
|
||
multipart ETag as a content hash.
|
||
- Refresh or fail clearly when identity evidence is absent, stale, ambiguous, or
|
||
unsupported; `--force` must not mean trust an unrelated local file.
|
||
- Install refreshed content using Stage 4's confined/durable download path.
|
||
|
||
**Tests and exit criteria:** Cover valid reuse, changed generation with same name,
|
||
missing/stale sidecar, multipart ETag, symlink/non-regular cache entries, corrupted
|
||
bytes, interrupted refresh, and force behavior.
|
||
|
||
## Stage 20 — Unify previous-source readiness and eliminate duplicate transfers
|
||
|
||
**Read first:** `audit-findings.md` lines 2038–2103 (COR-010 and COR-011) and
|
||
3064–3112 (EFF-001), plus TST-005 at lines 3643–3669; previous-cache, restore,
|
||
prepare, and dry-run docs.
|
||
|
||
**Depends on:** Stages 14 and 17–19.
|
||
|
||
**Goal:** Give restore, prepare, run, and dry-run one authoritative previous-source
|
||
resolver and retain verified downloads for the later restore snapshot.
|
||
|
||
**Work:**
|
||
|
||
- Implement one resolver that evaluates required/optional previous inputs from the
|
||
pointer-selected commit's exact source-to-destination mapping. Required absence
|
||
is actionable; optional absence is a typed non-error.
|
||
- Eliminate basename fallback from the new protocol. Keep any necessary legacy
|
||
fallback only inside the isolated Stage 13 compatibility boundary and reject
|
||
ambiguous matches.
|
||
- Reuse the generation-bound, verified candidate downloaded while constructing the
|
||
previous-source view when the restore snapshot needs the same object. Do not
|
||
reopen an unverified path or perform a second network transfer.
|
||
- Count actual transferred bytes once and make dry-run/readiness documentation
|
||
accurately describe remote checks and potential download/cache work.
|
||
|
||
**Tests and exit criteria:** Prove identical behavior across prepare/restore/run,
|
||
required/optional cases, same basenames, legacy ambiguity, generation changes,
|
||
single-transfer accounting, cancellation, and dry-run output.
|
||
|
||
## Stage 21 — Tighten configuration parsing, values, and expectations
|
||
|
||
**Read first:** `audit-findings.md` lines 2104–2253 (COR-012 through COR-015),
|
||
3805–3821 (TST-011), and 3853–3868 (TST-014); configuration and CLI docs.
|
||
|
||
**Depends on:** Stages 2 and 11.
|
||
|
||
**Goal:** Reject ambiguous or semantically invalid configuration before side
|
||
effects and give the configuration package focused, non-duplicative tests.
|
||
|
||
**Work:**
|
||
|
||
- Require exactly one YAML document, including rejection of a second empty or
|
||
comment-only document if the parser exposes it as another document.
|
||
- Validate all duration values that control retries, timeouts, polling, or leases
|
||
as positive and within any existing semantic bounds.
|
||
- Select storage backends only through explicit supported configuration; remove
|
||
inference from incidental S3-shaped fields and reject incomplete/unknown modes.
|
||
- Make the previous-session flag a strict expectation: when requested, absence or
|
||
mismatch is an actionable error, not a silent downgrade.
|
||
- Centralize test environment cleanup and reorganize configuration tests into
|
||
loader, defaulting, and validation ownership as required by TST-011/TST-014.
|
||
|
||
**Tests and exit criteria:** Cover multi-document YAML, zero/negative/overflow-like
|
||
durations, backend combinations, strict previous expectations, environment
|
||
isolation, defaults, unknown fields, and stable actionable errors.
|
||
|
||
**Status:** Completed.
|
||
|
||
## Stage 22 — Make product configuration truthful and own remote temp files
|
||
|
||
**Read first:** `audit-findings.md` lines 2521–2552 (COR-024), 3157–3197
|
||
(ARC-004), and 2839–2869 (RSK-009); configuration schema, notifications,
|
||
Scriptorium, remote config, and integration docs.
|
||
|
||
**Depends on:** Stages 4, 7, and 21.
|
||
|
||
**Goal:** Expose only implemented configuration behavior and keep remote-config
|
||
temporary paths within a command-scoped lifecycle.
|
||
|
||
**Work:**
|
||
|
||
- Remove inert Scriptorium `artifact` and `path` fields from schema, Go types,
|
||
examples, validation, and docs. Reject obsolete fields under the repository's
|
||
strict configuration policy with clear migration guidance where feasible.
|
||
- Support notifications only as an explicit no-op mode. Reject any configured
|
||
backend, recipient, or provider-shaped value until a provider exists; align CLI
|
||
and docs.
|
||
- Replace ad hoc remote-config temp paths with one owned command-scoped object or
|
||
lifecycle. Clean it on success, validation failure, adapter failure,
|
||
cancellation, and early returns using confined deletion.
|
||
- Never serialize ephemeral local temp paths into manifests, run records, or
|
||
diagnostic output that purports to be reusable state.
|
||
|
||
**Tests and exit criteria:** Cover obsolete/unknown product fields, explicit no-op,
|
||
backend/recipient rejection, every remote-config exit path, cancellation, stale
|
||
cleanup, and manifest inspection for ephemeral paths.
|
||
|
||
**Status:** Completed.
|
||
|
||
## Stage 23 — Stream WhisperX uploads and make the adapter race-safe
|
||
|
||
**Read first:** `audit-findings.md` lines 2254–2281 (COR-016), 3113–3156
|
||
(EFF-002), 3363–3387 (TST-001), and 3691–3718 (TST-007); WhisperX integration
|
||
docs.
|
||
|
||
**Depends on:** Stages 7–9 and 21.
|
||
|
||
**Goal:** Make WhisperX transport bounded, retry-correct, cancellable, and safe
|
||
under the race detector.
|
||
|
||
**Work:**
|
||
|
||
- Accept only absolute `http` or `https` WhisperX endpoints; reject file and other
|
||
schemes before execution.
|
||
- Stream multipart bodies through a pipe per attempt instead of buffering the
|
||
complete audio request. Each retry gets a fresh reader/writer and deterministic
|
||
metadata; producer failures and cancellation must reach the HTTP caller without
|
||
goroutine leaks.
|
||
- Retain bounded response/result acquisition from Stage 7 and make request/response
|
||
limits and errors adapter-specific.
|
||
- Synchronize the Whisper fake's request collection and expose snapshots through a
|
||
safe accessor. Migrate tests away from direct shared-slice reads.
|
||
- Add evidence that memory behavior is no longer proportional to complete upload
|
||
size, using deterministic backpressure/streaming tests rather than fragile heap
|
||
thresholds alone.
|
||
|
||
**Tests and exit criteria:** Cover schemes, retries, mid-stream read failure,
|
||
server early close, cancellation, blocked producer, exact payloads, bounded
|
||
responses, and goroutine completion. Run Whisper packages and their callers under
|
||
`go test -race`; this stage unblocks the full race suite in Stage 31.
|
||
|
||
**Status:** Completed.
|
||
|
||
## Stage 24 — Correct prepare/transcribe transition semantics
|
||
|
||
**Read first:** `audit-findings.md` lines 2282–2379 (COR-017 through COR-019)
|
||
and 3719–3743 (TST-008); prepare, transcribe, previous-cache, artifacts, and stage
|
||
lifecycle docs.
|
||
|
||
**Depends on:** Stages 12 and 20–23.
|
||
|
||
**Goal:** Make prepare/transcribe state transitions exact, cancellation-safe, and
|
||
independent of accidental basename behavior.
|
||
|
||
**Work:**
|
||
|
||
- Clear the managed previous-session tree when the effective requirement set is
|
||
empty as well as when it is populated. Publish only the current invocation's
|
||
verified result set.
|
||
- Track transcribe work as planned, dispatched, and completed. Cancellation or a
|
||
missing completion may not be reported as success and may not materialize a
|
||
completed artifact/manifest entry.
|
||
- Detect duplicate cleaned-audio source identities before dispatch. Allow distinct
|
||
source paths with the same basename and preserve exact identity through outputs.
|
||
- Keep state mutations inside the established lifecycle/terminalization contract.
|
||
|
||
**Tests and exit criteria:** Add no-requirements cleanup, stale prior tree,
|
||
cancellation at each phase, partial completion, duplicate source, same-basename
|
||
distinct source, retry, and no-false-manifest tests. This is the primary TST-008
|
||
stage; Stage 25 adds output/resolver boundary cases.
|
||
|
||
## Stage 25 — Enforce output-path authority and shared singleton resolution
|
||
|
||
**Read first:** `audit-findings.md` lines 3306–3330 (ARC-006), 3515–3537
|
||
(DUP-006), and 3719–3743 (TST-008); stage request/result, artifact resolver, and
|
||
transcription docs.
|
||
|
||
**Depends on:** Stage 24.
|
||
|
||
**Goal:** Make the stage-requested destination authoritative and resolve singleton
|
||
transcripts through one artifact-owned contract.
|
||
|
||
**Work:**
|
||
|
||
- Require an adapter result path to be empty (meaning the requested path) or
|
||
exactly equal to the normalized stage-requested path. Reject any other path
|
||
before materialization.
|
||
- Put the equality/empty contract in one shared stage/adapter boundary helper and
|
||
add conformance tests for all built-in adapters.
|
||
- Route singleton transcript resolution through the artifact package's canonical
|
||
resolver. Keep plural raw-source discovery separate; do not collapse different
|
||
cardinality semantics into one ambiguous helper.
|
||
- Remove duplicate singleton-selection code and stale error wording.
|
||
|
||
**Tests and exit criteria:** Cover empty/exact/different/relative-equivalent paths,
|
||
no output, multiple transcripts, unsafe paths, adapter conformance, and plural raw
|
||
sources. No adapter-selected alternate path may become authoritative.
|
||
|
||
## Stage 26 — Centralize typed extraction-bundle evidence
|
||
|
||
**Read first:** `audit-findings.md` lines 3538–3568 (DUP-007) and 3907–3932
|
||
(SIM-002, merged into DUP-007), plus extraction artifact and resume/catalog docs.
|
||
|
||
**Depends on:** Stages 2, 4, and 7.
|
||
|
||
**Goal:** Evaluate extraction-bundle evidence once while preserving the different
|
||
policy decisions made by resume and catalog consumers.
|
||
|
||
**Work:**
|
||
|
||
- Add an artifact-owned typed proof/result that classifies extraction evidence as
|
||
valid, absent, obsolete, or unsafe and carries only verified metadata/paths.
|
||
- Centralize structural and filesystem proof: expected files, manifest/bundle
|
||
coherence, safe root-relative regular files, versions, and existing identity
|
||
evidence. Do not encode resume/catalog policy into the proof.
|
||
- Make resume and catalog map the same proof to their own documented outcomes.
|
||
Remove duplicate validators and the merged SIM-002 opportunity for a broad
|
||
generic resolver.
|
||
|
||
**Tests and exit criteria:** Use one shared evidence fixture matrix for both
|
||
consumers and assert their intentionally different outcomes for every typed state.
|
||
Include unsafe root, symlink, incomplete, obsolete-version, and valid cases.
|
||
|
||
## Stage 27 — Bind extraction reuse to direct transcript identity
|
||
|
||
**Read first:** `audit-findings.md` lines 2380–2423 (COR-020) and 3744–3774
|
||
(TST-009); extraction fingerprint, artifact evidence, direct-input, and resume
|
||
docs.
|
||
|
||
**Depends on:** Stages 25–26.
|
||
|
||
**Goal:** Never reuse extraction output after the direct transcript input changes
|
||
while keeping the external producer's transitive dependencies explicit.
|
||
|
||
**Work:**
|
||
|
||
- Extend the extraction fingerprint with a deterministic checksum of the exact
|
||
trimmed direct transcript bytes and stable source/producer identity needed by
|
||
the contract.
|
||
- Recompute that direct-input identity on every resume decision and feed it into
|
||
the typed Stage 26 evidence proof. A mismatch is obsolete evidence, not success.
|
||
- Keep Notarius/external transitive dependency freshness under the existing
|
||
explicit `--force` contract unless stable producer evidence is available; do not
|
||
invent a false dependency graph.
|
||
- Ensure extraction failure or obsolescence does not leave newly valid-looking
|
||
catalog/manifest entries.
|
||
|
||
**Tests and exit criteria:** Add unchanged/changed/trimmed transcript cases,
|
||
producer/source changes, resume/restart, force, unsafe root, over-limit lane
|
||
result, and failure-before-publication tests. This is the primary TST-009 stage.
|
||
|
||
## Stage 28 — Establish one effective artifact set and catalog bootstrap
|
||
|
||
**Read first:** `audit-findings.md` lines 2456–2491 (COR-022), 3331–3362
|
||
(ARC-007), 3569–3595 (DUP-008), and 3775–3804 (TST-010); analyze, extraction
|
||
catalog, artifact selection, configuration, and publish-filter docs.
|
||
|
||
**Depends on:** Stages 21 and 26–27.
|
||
|
||
**Goal:** Resolve artifact selection once into a typed effective set used by every
|
||
analyze precondition and input-composition path.
|
||
|
||
**Work:**
|
||
|
||
- Define one typed effective artifact set. Explicit CLI/stage selection overrides
|
||
configured defaults even when the selected artifact is disabled by default.
|
||
- Use that exact set for validation, prerequisite planning, extraction catalog
|
||
loading, prompt/input composition, and analyze execution. Keep publication
|
||
filtering a separate typed decision.
|
||
- Centralize only deterministic catalog definitions/bootstrap ordering in the
|
||
artifact owner. Keep caller-specific missing/optional/error policy outside the
|
||
shared bootstrap.
|
||
- Remove duplicate catalog lists and boolean mixtures that can diverge.
|
||
|
||
**Tests and exit criteria:** Table-test configured defaults versus explicit
|
||
selection, disabled overrides, empty selections, ordering, catalog versions,
|
||
publication filters, and the exact composed analyze inputs. This is the primary
|
||
TST-010 stage; Stage 29 adds resolution/error cases.
|
||
|
||
## Stage 29 — Make analyze resolution typed, optional, actionable, and deterministic
|
||
|
||
**Read first:** `audit-findings.md` lines 2424–2455 (COR-021), 2492–2520
|
||
(COR-023), 3033–3063 (RSK-015), 3933–3961 (SIM-003), and 3775–3804 (TST-010);
|
||
analyze, stage command, artifact dependency, and troubleshooting docs.
|
||
|
||
**Depends on:** Stage 28.
|
||
|
||
**Goal:** Give analyze one typed resolution/execution plan with correct optional
|
||
semantics, deterministic diagnostics, and executable remediation guidance.
|
||
|
||
**Work:**
|
||
|
||
- Represent built-in artifact resolution as typed present/absent/error results.
|
||
Optional absence is omitted; required absence returns a stable actionable error.
|
||
- Build one analyze execution context/effective plan from the Stage 28 artifact
|
||
set and typed resolutions. Do not add a generic resolver unrelated to analyze.
|
||
- Sort aggregated missing/dependency diagnostics by stable artifact/source keys.
|
||
Never expose map iteration order.
|
||
- Correct remediation text to the real command form:
|
||
`narratio run-stage prepare <session_id> --force`, with context-specific
|
||
alternatives only when they are actually valid.
|
||
|
||
**Tests and exit criteria:** Cover optional missing built-ins, required missing,
|
||
mixed custom/built-in artifacts, repeated randomized map insertion, exact guidance,
|
||
context construction, and final prompt inputs. Run deterministic tests repeatedly.
|
||
|
||
## Stage 30 — Remove misleading contracts and align Audita ownership
|
||
|
||
**Read first:** `audit-findings.md` lines 3198–3228 (ARC-001), 3280–3305
|
||
(ARC-005), 3962–3979 (SIM-004), 4021–4037 (COM-003), and 4085–4101
|
||
(COM-006); adapter, stage API, Audita, extraction source, and developer docs.
|
||
|
||
**Depends on:** Stages 23–29.
|
||
|
||
**Goal:** Remove dead or misleading surfaces after the behavioral contracts have
|
||
stabilized and place static Audita policy in its constructor.
|
||
|
||
**Work:**
|
||
|
||
- Remove the unused `IODecl`/`Declares` surface and update tests/docs rather than
|
||
preserving a compatibility shim for an internal unused contract.
|
||
- Move static Audita settings to constructor/configuration ownership. Per-run
|
||
requests carry only run-specific paths, modules, and execution data. Update the
|
||
fake and all call sites.
|
||
- Delete dead helpers made obsolete by the remediation. Do not delete a narrow
|
||
helper merely because it has one caller when it still owns a real invariant.
|
||
- Remove the stale Audita TODO and add a rationale comment only if ownership is not
|
||
self-evident. Add the extraction-source doc comment required by COM-006.
|
||
|
||
**Tests and exit criteria:** Compile all adapters and fakes, run their focused
|
||
tests, search for removed symbols/TODOs/stale request fields, and confirm public
|
||
package documentation matches the remaining contract.
|
||
|
||
## Stage 31 — Enforce CI validation and streamline the assembled test suite
|
||
|
||
**Read first:** `audit-findings.md` lines 3822–3836 (TST-012) and 3869–3885
|
||
(TST-015); testing policy, development workflow, Woodpecker configuration, and
|
||
release docs.
|
||
|
||
**Depends on:** All production stages, especially Stage 23.
|
||
|
||
**Goal:** Make ordinary automated validation unavoidable and consolidate broad
|
||
stage matrices only after the focused regression protections exist.
|
||
|
||
**Work:**
|
||
|
||
- Make every change and release path run ordinary tests, `go vet`, `go build`, and
|
||
documentation/example validation. A release must consume a revision that passed
|
||
the required validation, not merely run alongside an optional test pipeline.
|
||
- Run the full `go test -race ./...` suite on every change now that TST-001 is
|
||
fixed. Add scheduled shuffled/repeated execution for order dependence.
|
||
- Preserve focused owner tests from prior stages, then consolidate redundant broad
|
||
stage matrices as directed by TST-015. Do not reduce behavior coverage or hide
|
||
platform cases in an opaque mega-test.
|
||
- Add native macOS/Windows jobs only when authoritative runner labels and
|
||
successful native runners are available. Until then, retain explicit
|
||
cross-build checks and document the native evidence gap without inventing
|
||
labels or marking cross-compilation as equivalent.
|
||
|
||
**Tests and exit criteria:** Validate Woodpecker syntax and dependency graph,
|
||
exercise the local equivalents of every job, prove failure propagation to release
|
||
eligibility, and inspect the final suite for duplicated matrices and lost cases.
|
||
|
||
## Stage 32 — Reconcile lifecycle/analyze documentation and close the remediation
|
||
|
||
**Read first:** `audit-findings.md` lines 3229–3255 (ARC-002, merged into
|
||
COM-002), 3998–4020 (COM-002), 4038–4055 (COM-004, merged into COM-002), and
|
||
4056–4084 (COM-005); all current-behavior policy and package documents modified by
|
||
the preceding stages.
|
||
|
||
**Depends on:** Stages 1–31.
|
||
|
||
**Goal:** Make the repository's normative documentation describe the implemented
|
||
architecture exactly and mechanically verify that every actionable audit finding
|
||
has a completed primary stage.
|
||
|
||
**Work:**
|
||
|
||
- Document the distinction between successful no-output, optional absence, and a
|
||
stage's explicit self-skip. Keep the production lifecycle semantics implemented
|
||
in earlier stages; do not add a new lifecycle abstraction for COM-002.
|
||
- Reconcile extraction catalog schema, effective selection/override semantics,
|
||
analyze consumption, optional absence, force/freshness rules, remote commit and
|
||
compatibility behavior, restore recovery, collaborative modes, secret handling,
|
||
output limits, and operations quota guidance.
|
||
- Review examples, troubleshooting commands, package docs, diagrams, and comments
|
||
against the current code. Remove superseded statements rather than layering new
|
||
caveats over them.
|
||
- Verify the traceability inventory below. Every confirmed actionable finding must
|
||
have one completed primary stage; merged/rejected findings must retain their
|
||
recorded disposition; accepted risks must not have been silently expanded.
|
||
|
||
**Tests and exit criteria:** Run the full Stage 31 validation locally, run all
|
||
documentation/example checks, verify clean generated/example state, inspect the
|
||
final diff and repository status, and mark this stage complete only when no
|
||
required remediation remains.
|
||
|
||
## Finding traceability inventory
|
||
|
||
This inventory assigns exactly one primary implementation stage to each confirmed
|
||
actionable finding. A stage may read or test a finding owned by another stage when
|
||
the behavior crosses a boundary, but it must not create a second competing fix.
|
||
|
||
| Finding(s) | Primary stage |
|
||
| --- | ---: |
|
||
| COR-001 | 11 |
|
||
| COR-002 | 2 |
|
||
| COR-003 | 4 |
|
||
| COR-004 | 14 |
|
||
| COR-005 | 10 |
|
||
| COR-006, COR-007 | 16 |
|
||
| COR-008, COR-009 | 17 |
|
||
| COR-010 | 20 |
|
||
| COR-011 | 14 |
|
||
| COR-012, COR-013, COR-014, COR-015 | 21 |
|
||
| COR-016 | 23 |
|
||
| COR-017, COR-018, COR-019 | 24 |
|
||
| COR-020 | 27 |
|
||
| COR-021, COR-023 | 29 |
|
||
| COR-022 | 28 |
|
||
| COR-024 | 22 |
|
||
| RSK-001 | 12 |
|
||
| RSK-002 | 3 |
|
||
| RSK-003 | 5 |
|
||
| RSK-004 | 1 |
|
||
| RSK-005 | 15 |
|
||
| RSK-006, RSK-008 | 18 |
|
||
| RSK-007 | 19 |
|
||
| RSK-009 | 22 |
|
||
| RSK-010 | 6 |
|
||
| RSK-011 | 8 |
|
||
| RSK-012 | 9 |
|
||
| RSK-013 | 7 |
|
||
| RSK-014 | 15 |
|
||
| RSK-015 | 29 |
|
||
| EFF-001 | 20 |
|
||
| EFF-002 | 23 |
|
||
| ARC-001 | 30 |
|
||
| ARC-003 | 13 |
|
||
| ARC-004 | 22 |
|
||
| ARC-005 | 30 |
|
||
| ARC-006 | 25 |
|
||
| ARC-007 | 28 |
|
||
| TST-001 | 23 |
|
||
| TST-002 | 12 |
|
||
| TST-003 | 4 |
|
||
| TST-004 | 14 |
|
||
| TST-005 | 17 |
|
||
| TST-006 | 11 |
|
||
| TST-007 | 7 |
|
||
| TST-008 | 24 |
|
||
| TST-009 | 27 |
|
||
| TST-010 | 28 |
|
||
| TST-011, TST-014 | 21 |
|
||
| TST-012, TST-015 | 31 |
|
||
| TST-013 | 2 |
|
||
| DUP-001, DUP-005 | 3 |
|
||
| DUP-002 | 14 |
|
||
| DUP-003 | 4 |
|
||
| DUP-006 | 25 |
|
||
| DUP-007 | 26 |
|
||
| DUP-008 | 28 |
|
||
| SIM-001 | 12 |
|
||
| SIM-003 | 29 |
|
||
| SIM-004 | 30 |
|
||
| COM-001 | 12 |
|
||
| COM-002, COM-005 | 32 |
|
||
| COM-003, COM-006 | 30 |
|
||
|
||
The remaining confirmed IDs have non-independent dispositions and must not receive
|
||
separate implementation work: ARC-002 and COM-004 are merged into COM-002;
|
||
SIM-002 is merged into DUP-007; DUP-004 is rejected. Their audit rationale remains
|
||
authoritative.
|
||
|
||
## Open Questions
|
||
|
||
None. The governing decisions above make this plan decision-complete. Native
|
||
macOS/Windows CI activation remains contingent on external runner availability,
|
||
but the required implementation, cross-build checks, and rule against inventing
|
||
runner labels are already specified and do not require an architectural decision.
|