From 51edd384c0f2b2e736bfe7b4d02f6c27169bf2b2 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sat, 29 Aug 2026 23:11:19 +0000 Subject: [PATCH] Remove completed feature roadmaps --- docs/roadmap/implementation.md | 810 ------------------ docs/roadmap/notarius-v0.6-cli-references.md | 274 ------ .../post-transcript-artifact-workflow.md | 409 --------- 3 files changed, 1493 deletions(-) delete mode 100644 docs/roadmap/implementation.md delete mode 100644 docs/roadmap/notarius-v0.6-cli-references.md delete mode 100644 docs/roadmap/post-transcript-artifact-workflow.md diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md deleted file mode 100644 index 8e5cb02..0000000 --- a/docs/roadmap/implementation.md +++ /dev/null @@ -1,810 +0,0 @@ -# Post-Transcript Artifact Workflow Implementation Plan - -## Purpose And Status - -This document is the executable implementation plan for -[`post-transcript-artifact-workflow.md`](post-transcript-artifact-workflow.md). -That roadmap owns the accepted user intent, compatibility policy, and target -state. This plan translates it into bounded stages suitable for one -`gpt-5.6-terra` implementation prompt apiece. - -All stages below are completed. They were implemented in numeric order, with -each later stage relying on the tested and documented contracts established by -its predecessors. This accepted roadmap remains available for closeout history; -it does not replace the canonical current-behavior documents linked from -`docs/development.md`. - -## Settled Implementation Decisions - -The following choices make the plan decision-complete: - -- The canonical order is `prepare`, `transcribe`, `merge`, `polish`, - `normalize`, `trim`, `render`, `extract`, `analyze`, `publish`, `notify`. -- Execution order remains a flat, fixed registry. Invalidation uses a separate - fixed direct-dependency table whose transitive closure is returned in - canonical order. It is not configurable and is not an execution DAG. -- `run` and `session plan` share one inclusive contiguous-range selector. - Omitted endpoints select the beginning or end of the canonical registry. -- A bounded `run` requires every excluded prefix stage to have a terminal - manifest status of `succeeded` or `skipped`. The first absent, pending, - running, failed, stale, or interrupted prerequisite is an actionable error. - Excluded prefix stages are not resume-validated or repaired; selected stages - still validate the concrete inputs they consume. Stages after `--through` - are not prerequisites and receive no analogous check. -- `--force` affects only selected pipeline stages. Invalidation may mark a - dependent outside the range stale, but cannot execute it. -- `--artifacts` is valid only when the selected range contains `analyze` or - `publish`. Its existing repeatable, comma-separated normalization and - deduplication semantics remain unchanged. -- `regenerate-artifacts` performs argument expansion and calls the canonical - `run` handler. It owns no planner, runner, prerequisite, or force behavior. - Duplicate `--force`, `--from`, or `--through` options are rejected by the - shared bounded-run parser, including `--name=value` spellings. -- Incremental state is specific to configured Scriptorium artifacts inside the - fixed `analyze` stage. Do not create dynamic stages, generic jobs, or a - general subtask framework. -- The session manifest owns current artifact availability. A configured output - file found on disk without current manifest evidence is unavailable. -- Analysis fingerprints contain only Narratio-observable semantic inputs. They - exclude run IDs, absolute workspace paths, timeouts, and executable contents. - They also cannot observe arbitrary files or transitive configuration loaded - privately by Scriptorium; those changes require explicit force. -- Partial analysis preserves unrelated current records. On an error, the - runner accepts a restricted analyze-state result alongside the error so it - can record completed, failed, and newly stale artifact state before marking - the aggregate stage failed. It must never promote unvalidated files. -- Legacy aggregate-only analysis output remains readable but is not current - evidence. Only regenerated artifacts enter the new per-artifact state. - -## Instructions For Every Stage - -Before changing code in each stage: - -1. Read `docs/development.md`, its task-specific references, all documents in - `docs/policy/`, this plan, and the relevant portions of the feature roadmap. -2. Inspect the current implementation before editing. Prefer the codebase - knowledge graph for code discovery, then read the exact owning files. -3. Confirm the worktree state and preserve unrelated user changes. - -During each stage: - -- Keep changes within that stage's scope and the accepted roadmap. -- Follow the architecture policy's dependency direction, filesystem safety, - manifest authority, and application-owned lifecycle rules. -- Follow the testing policy: place behavior at its narrowest public owner, use - fakes at owned boundaries, keep the default suite offline and deterministic, - and avoid duplicating the same assertion across layers. -- Update a canonical current-behavior document in the same stage when that - stage changes already usable behavior. Do not describe unimplemented later - stages as current. -- Preserve backward-compatible manifest reads and emit only the new canonical - representation after its writer is introduced. -- Run focused tests during development, then `go test ./...` before completing - the stage. Run the repository's required formatting, static checks, and - documentation checks identified by `docs/development.md`. - -At the end of each stage, leave a cohesive change that can be reviewed and -committed independently. Do not begin a later stage while an earlier stage has -failing tests or incomplete acceptance criteria. - -## Stage 1 — Canonical Order And Dependency-Aware Invalidation - -**Status: Completed** - -### Goal - -Move `render` before `extract` and replace suffix-based invalidation with the -fixed dependency relation required by the feature roadmap. - -### Required Work - -1. Change the single canonical stage registry so `render` immediately follows - `trim` and `extract` immediately follows `render`. Do not add a second stage - list in a command or test helper. -2. In the application orchestration owner, define a fixed table of direct - invalidation edges: - - `prepare -> transcribe` - - `transcribe -> merge` - - `merge -> polish` - - `polish -> normalize` - - `normalize -> trim` - - `trim -> render, extract` - - `render -> analyze` - - `extract -> analyze` - - `analyze -> publish` - - `publish -> notify` - - `notify ->` none -3. Compute transitive dependents from that table and return them in canonical - order. Unknown stages must return an error; callers must not silently treat - them as having no dependents. -4. Validate the relation against the canonical registry when it is constructed - or first used. Reject duplicate registry names, unknown edge endpoints, - cycles, missing stage classifications, and registry/table drift. -5. Route every existing invalidation trigger through this owner: force, - non-resumable success, failure, and changed effective outcome. Retain the - existing rule that only succeeded dependent records become stale and retain - the existing no-op treatment for an identical repeated self-skip. -6. Remove assumptions that invalidation is the canonical suffix. In - particular, `render` and `extract` must never invalidate one another, while - either invalidates `analyze`, `publish`, and `notify`. -7. Update the architecture policy, internal overview, and focused render, - extract, and manifest documentation for the implemented order and separate - invalidation owner. - -### Tests And Exit Criteria - -- Registry/planner tests assert the complete new order. -- Table-driven invalidation tests assert every transitive set in the feature - roadmap, canonical output ordering, render/extract independence, unknown - stages, cycle rejection, and inventory drift rejection. -- Representative runner tests prove each invalidation trigger uses the new - relation and does not execute newly stale stages implicitly. -- Existing manifests with stable stage names remain readable and resumable; - no version-based invalidation or manifest migration is introduced. - -## Stage 2 — Shared Bounded Plan Selection - -**Status: Completed** - -### Goal - -Give the application one validated representation of a contiguous canonical -stage range, shared by execution and plan preview. - -### Required Work - -1. Extend the application planner with a bounded-plan constructor accepting - optional `from` and `through` names. The returned plan contains an inclusive - contiguous slice of the canonical registry. -2. Default an omitted `from` to the first stage and an omitted `through` to the - last stage. Preserve the current full plan when both are omitted. -3. Reject unknown endpoints and a `from` endpoint occurring after `through`. - Errors must name the bad value or reversed pair and list or point to valid - canonical stage names. -4. Put range membership and endpoint information on one application-level - value so `run`, `session plan`, prerequisite validation, and adapter - composition do not independently recalculate bounds. -5. Keep `run-stage` and the dedicated `analyze` and `publish` conveniences on - their existing single-stage paths. Do not reinterpret them through new - range flags. - -### Tests And Exit Criteria - -- Planner tests cover every endpoint, one-stage ranges, omitted starts, omitted - ends, a full omitted range, unknown names, reversed ranges, and stable order. -- Tests assert that selecting a range cannot create a non-contiguous plan or - mutate the canonical registry. -- No CLI behavior changes in this stage; the new planner API is ready for both - `run` and `session plan` to consume next. - -## Stage 3 — Bounded `run` And `session plan` Command Contracts - -**Status: Completed** - -### Goal - -Expose the shared range through both commands with one parsing and structural -validation contract. - -### Required Work - -1. Add repeat-safe singleton option parsing for `--from`, `--through`, and - `--force` to the shared bounded-run command layer. Reject duplicates whether - written as separate arguments or `--name=value`; never use argument order to - choose a winner. -2. Preserve the existing session-oriented argument conventions and common - configuration flags. Preserve repeatable/comma-separated `--artifacts` - normalization and deduplication. -3. Add `--from` and `--through` to `run`, and pass the resulting bounded plan to - the existing runner rather than filtering stages after planning. -4. Add the same bounds, force flag, and artifact selection to `session plan`. - Both commands must call the Stage 2 selector and surface identical range - validation errors. -5. Reject `--artifacts` when the range contains neither `analyze` nor `publish`. - Accept it when either consumer is present, including a single-stage range. -6. Update command help, usage errors, `docs/cli.md`, and `docs/operations.md` for - the inclusive/defaulting behavior. Do not document the alias until Stage 5. - -### Tests And Exit Criteria - -- Command tests cover valid ranges, defaults, invalid/reversed ranges, - singleton duplicates in both syntaxes, and artifact/range compatibility. -- Unbounded `run` and unbounded `session plan` retain their prior behavior - except for the canonical order implemented in Stage 1. -- Parsing tests assert structural equivalence between the run and plan range - values, without duplicating all planner cases at the command layer. - -## Stage 4 — Bounded Runner Prerequisites And Composition - -**Status: Completed** - -### Goal - -Make bounded execution honor its mutation boundary while still failing safely -when excluded upstream work cannot support the selected stages. - -### Required Work - -1. Before starting a bounded run, inspect the session manifest for every - canonical stage before `--from`. Accept only `succeeded` or `skipped` as a - terminal prerequisite status. Treat an absent record and every other status - as unusable. -2. Report the first unusable prefix stage in canonical order, its status (or - absence), the selected start, and an actionable suggestion to widen the - range or recover that stage explicitly. Do not mutate the manifest, create a - run record, or invoke an adapter before this check succeeds. -3. Do not call resume validators for excluded prefix stages. Concrete selected - stages continue to resolve and validate their own manifest-authoritative - inputs, so an unsafe or missing artifact still fails at its owning boundary. -4. Do not inspect stages after `--through` as prerequisites. They may become - stale through Stage 1 invalidation, but must not be scheduled. -5. Ensure force decisions are calculated only for stages in the bounded plan. - A forced selected stage may stale dependencies outside the range but cannot - execute them. -6. Make production composition plan-aware. Initialize and validate only the - external adapters, object storage, remote locks, and other collaborators - needed by selected stages or by shared session lifecycle requirements. - Preserve each selected stage's current fail-fast configuration validation. -7. Keep post-publish cleanup conditional on publish actually executing. -8. Update operational and troubleshooting documentation for prerequisite - failures, recovery, excluded-stage behavior, and plan-aware composition. - -### Tests And Exit Criteria - -- Runner tests cover every unacceptable prefix status, accepted skipped - prerequisites, no prefix for a first-stage run, and no check after the end. -- Tests prove excluded stages are neither executed nor resume-validated and - that a failed prerequisite check performs no persistent run mutation. -- Adapter-composition tests prove a bounded render-only run does not require - Notarius or Scriptorium, an extract-only run requires Notarius but not - Scriptorium, and analyze requires only its actual collaborators. -- Tests prove force/invalidation can stale an out-of-range dependent without - executing it and that stop-on-selected-stage-failure remains unchanged. - -## Stage 5 — Exact `regenerate-artifacts` Alias - -**Status: Completed** - -### Goal - -Add the transparent convenience command without creating another orchestration -path. - -### Required Work - -1. Register the top-level `regenerate-artifacts` command. -2. Implement it only by constructing the equivalent canonical arguments and - invoking the shared `run` parser/handler: - - ```text - run --force --from extract --through analyze [caller options] - ``` - - Preserve common session/configuration arguments and all repeatable - `--artifacts` values. -3. Do not add an alias-specific loader, plan, runner, prerequisite check, - summary, artifact rule, or force rule. Canonical run diagnostics may call - the operation `run`. -4. Let the shared duplicate-singleton validation from Stage 3 reject a caller's - `--force`, `--from`, or `--through`, since the alias already supplies them. -5. Give the alias concise help that states the exact expansion, that extraction - always runs, that selected analysis artifacts and required prerequisites are - rebuilt, and that publish/notify never run. -6. Add the alias to `docs/cli.md` and the development workflow in - `docs/operations.md`. - -### Tests And Exit Criteria - -- A narrow command test captures the forwarded arguments or resulting shared - command request and proves exact equivalence, including option pass-through. -- Tests prove alias help does not execute, duplicate owned options fail through - the shared parser, and unknown/private alias options are not accepted. -- Do not duplicate runner integration cases under the alias name; its only - behavior is expansion. - -## Stage 6 — Versioned Analyze-Artifact Manifest State - -**Status: Completed** - -### Goal - -Introduce a backward-compatible, analyze-specific session-manifest model that -can represent independently current artifacts without treating them as stages. - -### Required Work - -1. Add an explicit analysis-state contract version to the `analyze` stage - record. Absence means legacy aggregate-only state; a present supported - version distinguishes a valid empty set from legacy data. -2. Add a map keyed by normalized configured artifact key. Each record must - contain: - - key and disposition/status (`current`, `stale`, `missing`, `failed`, or - `unselected`); - - fingerprint contract version and fingerprint when evaluated; - - normalized configured dependency keys; - - the existing output artifact record when current, including source ID, - contract, canonical confined path, and checksum, plus a separate - analyze-record output-size field (do not broaden every artifact schema - solely to carry this analyze-specific evidence); - - producer run ID and update time; - - a bounded non-secret error for failed work; and - - useful non-secret Scriptorium provenance, logs, and generated config paths - already allowed by manifest policy. -3. Add the corresponding analyze-artifact collection to invocation-stage - records. A session record describes the reconciled current set; an - invocation record describes only work evaluated or attempted by that run. -4. Keep the schema explicitly owned by analyze. Do not add dynamic stage names - or generic pipeline-subtask abstractions. -5. Extend validation and canonical serialization for the new fields. Validate - normalized unique keys, supported versions, status-specific required and - forbidden fields, output identity, checksum/size, and non-secret bounded - metadata. Maintain deterministic JSON output. -6. Readers must accept old manifests with no new fields. Writers must not - fabricate fingerprints from legacy aggregate outputs or emit parallel - legacy state as current evidence. -7. Preserve per-artifact analysis state when ordinary stage lifecycle helpers - clear aggregate result details during running, failure, or skip. Other stage - records keep their existing behavior. -8. Update `docs/internal/manifest.md` with the new authority, legacy meaning, - status model, and session-versus-invocation distinction. - -### Tests And Exit Criteria - -- Manifest round-trip tests cover every status and deterministic map output. -- Validation tests cover malformed keys, unsupported versions, impossible - status/field combinations, incomplete current output, and bad checksum/size. -- Legacy fixtures remain readable and are explicitly identified as lacking - current artifact evidence. -- Lifecycle tests prove unrelated current records survive transitions while - aggregate output/log/config/metadata fields retain their prior clearing - semantics. - -## Stage 7 — Runner Projection And Partial-Error State Boundary - -**Status: Completed** - -### Goal - -Give analyze one safe way to promote reconciled session state and invocation -history through the existing application-owned runner transaction. - -### Required Work - -1. Extend the stage result contract with an optional analyze-specific state - projection containing: - - the complete reconciled session analysis state; and - - the invocation subset attempted or evaluated in the current run. - Keep ordinary stage results unchanged. -2. On successful analyze completion, have the runner validate and apply the - session projection to the session manifest and the invocation projection to - the run manifest. Rebuild aggregate session `Outputs` deterministically from - current per-artifact records only. Keep invocation `Outputs` limited to - artifacts actually produced by that invocation. -3. Add a restricted result-plus-error path for analyze. If a stage returns an - error with an analyze-state projection, validate and persist only that state - before marking the aggregate stage failed. Ignore/reject ordinary success - outputs, success disposition, or unrelated stage projections alongside an - error. -4. Apply state only after the stage has validated run-local outputs and safely - materialized the records it marks current. The runner must never derive - records by scanning output directories. -5. On projection validation or persistence failure, fail conservatively and do - not advertise newly attempted artifacts as current. Preserve the last - durable unrelated current records. -6. Add `Force` to the stage environment as application-owned invocation - context, set it from the selected stage decision, and leave unselected - stages unaffected. This will distinguish explicit targets from ordinary - stale prerequisite rebuilding in later stages. - -### Tests And Exit Criteria - -- Runner tests prove separate session and invocation projections on success. -- Error-path tests prove completed artifacts can be durably represented, the - failed target is not current, unrelated prior current records survive, and - aggregate analyze/publish state remains conservative. -- Tests reject state projection from non-analyze stages and malformed or - contradictory result-plus-error payloads. -- Existing stages and ordinary error behavior remain unchanged. - -## Stage 8 — Manifest-Authoritative Configured Artifact Evidence - -**Status: Completed** - -### Goal - -Make configured analysis outputs available to analyze and publish only through -validated current manifest evidence. - -### Required Work - -1. Add an analysis-evidence inspector and catalog hydrator analogous to the - extraction-evidence owner, but specific to configured Scriptorium artifacts. -2. For a `current` record, verify the supported state/fingerprint version, - configured key and source ID, contract, configured canonical path, confined - no-follow regular file, stored size, and stored checksum. Return a typed - current/non-current result with an actionable reason. -3. Treat stale, missing, failed, unselected, legacy, removed, malformed, - unsafe, missing, or checksum-mismatched evidence as unavailable. Do not - silently rewrite status during read-only catalog hydration. -4. Remove current-session configured-artifact fallbacks that call `stat` and - mark canonical files available merely because they exist. Hydrate analyze, - publish, operator display, and artifact resolution from the evidence owner. -5. Retain existing behavior for prior-session inputs or other source kinds only - where their canonical owner already has an explicit compatibility policy; - do not broaden filesystem fallback. -6. When current configuration removes or renames a key, omit its old record - from the current catalog even if the manifest retains history until the next - analyze reconciliation. -7. Update `docs/internal/artifacts.md` and `docs/internal/stage-publish.md` for - manifest-authoritative configured results. - -### Tests And Exit Criteria - -- Evidence tests cover valid current output, each non-current status, legacy - absence, config/path/source mismatch, symlink/non-regular files, missing - files, size mismatch, and checksum mismatch. -- Analyze and publish catalog tests prove an incidental file is unavailable and - a validated current record is available. -- Publish tests prove stale configured artifacts cannot be selected or uploaded - while unrelated current artifacts remain publishable. - -## Stage 9 — Deterministic Analyze Input Identity - -**Status: Completed** - -### Goal - -Resolve every Narratio-visible analyze input into a stable semantic identity -that the fingerprint engine can consume without depending on workspace paths or -producer runs. - -### Required Work - -1. Define an ordered input-identity record containing the configured input - name, source ID, required/optional policy, presence/absence, contract, - content checksum, and size. Include a stable session-relative or source-based - logical identity where needed; never include an absolute path. -2. Resolve transcript, prepared-input, extraction-lane, previous-session, and - configured-artifact sources through their existing artifact resolvers and - catalogs. Configured-artifact dependencies must pass Stage 8 evidence. -3. Hash validated regular files using the repository's streaming file-safety - primitives and central size limits. Reuse an already validated manifest - checksum when its owning evidence contract proves it represents the same - bytes; do not read whole artifacts into memory. -4. Represent an absent optional input explicitly so appearance/disappearance - changes identity. A missing required input remains an error. -5. Normalize input ordering independently of Go map iteration while preserving - any configured order whose semantics are observable to Scriptorium. -6. Keep identity resolution read-only. It may inspect files and manifests but - cannot materialize output, update status, invoke adapters, or create run - records. - -### Tests And Exit Criteria - -- Table-driven tests cover every supported source kind, optional absence, - missing required input, unsafe path/type, checksum reuse, and changed bytes. -- Identical bytes at relocated workspace roots produce identical semantic - identities; changed bytes or contracts produce different identities. -- Configured dependencies cannot resolve from incidental or stale files. -- Ordering remains deterministic across randomized map insertion. - -## Stage 10 — Versioned Analysis Fingerprints And Reconciliation - -**Status: Completed** - -### Goal - -Classify configured artifacts as current or requiring work from one deterministic -fingerprint contract shared by resume validation, planning, and execution. - -### Required Work - -1. Define a centrally named fingerprint contract version and a canonical - serialization used only as hash input. Do not use ad hoc string concatenation - or Go map serialization. -2. Include all Narratio-observable result-affecting fields: - - normalized artifact key and normalized Scriptorium artifact configuration; - - deterministic dependency keys; - - effective prompt/profile identifiers, render-debug behavior, normalized - output identity, ordered input declarations, and sorted effective vars; - - Stage 9 resolved input identities and current configured-dependency output - identities; and - - result-affecting global Scriptorium configuration that Narratio directly - passes or interprets. -3. Exclude timeout/retry settings, absolute binary/config/output/workspace - paths, run IDs, timestamps, log/config output locations, and executable or - arbitrary transitive file contents. Document that explicit force is required - when an unobserved Scriptorium-private input changes. -4. Compute dependency artifact fingerprints in deterministic topological order - and reject unknown dependencies and cycles through the existing configuration - validation owner. -5. Build a read-only reconciliation function that compares current config, - stored record/version/fingerprint, Stage 8 output evidence, and newly resolved - identities. Return a typed reason for `current`, `stale`, `missing`, `failed`, - `legacy`, `removed`, or otherwise non-resumable state. -6. Fingerprint byte-identical input content identically even when it came from - a forced upstream run with a different producer run ID. - -### Tests And Exit Criteria - -- Golden or table-driven tests prove deterministic fingerprints across map - order, workspace relocation, and producer run IDs. -- Sensitivity tests change each included semantic field independently and - assert a new fingerprint. Exclusion tests cover timeout, timestamp, run ID, - and absolute-root-only changes. -- Reconciliation tests cover current, tampered output, changed dependency, - optional input transition, legacy record, removed config, version mismatch, - and byte-identical upstream replacement. -- `docs/internal/stage-analyze.md` records the implemented fingerprint boundary - and its explicit limitations. - -## Stage 11 — Incremental Analysis Work Planning - -**Status: Completed** - -### Goal - -Turn reconciliation results, explicit selection, dependencies, and force into -a deterministic artifact execution plan without invoking Scriptorium. - -### Required Work - -1. Define explicit targets as the normalized `--artifacts` selection when - present, otherwise all enabled configured artifacts. Preserve existing - unknown/ambiguous-selection validation. -2. Compute the transitive configured prerequisite closure of explicit targets - and order the closure deterministically and topologically. -3. Reuse a prerequisite when Stage 10 classifies it current. Schedule a stale, - missing, failed, legacy, or invalid prerequisite before its dependent. -4. Force only explicit targets. A prerequisite is forced only when it is also - an explicit target; otherwise a current prerequisite is reused. -5. Preserve the existing selection rule: default selection includes enabled - artifacts, while an explicitly named disabled artifact is a valid target. - A disabled configured prerequisite may be reused when current or executed - when it is in a selected target's required closure. Never execute an - unrelated disabled artifact. -6. Retain valid unselected records. Classify artifacts removed or renamed from - current configuration as unavailable in the reconciled session projection. -7. Produce a typed plan containing explicit targets, prerequisite-only work, - reused current artifacts, invalidated/removed records, reasons, and the - deterministic execution order. Keep the function read-only and free of - adapter calls. - -### Tests And Exit Criteria - -- Planning tests cover full/default selection, partial selection, dependency - closure, nested dependencies, current prerequisite reuse, stale prerequisite - rebuilding, target-only force, disabled prerequisites, unknown keys, cycles, - removed configuration, and deterministic order. -- Tests prove valid unrelated records survive the projected result and legacy - unselected outputs do not become current. -- The planner exposes enough typed information for `session plan` and analyze - execution to share decisions rather than recomputing them differently. - -## Stage 12 — Incremental Analyze Execution And Promotion - -**Status: Completed** - -### Goal - -Execute the Stage 11 work plan on the successful path, safely promote validated -results, and preserve or invalidate records according to actual output identity. - -### Required Work - -1. Refactor analyze to execute only scheduled artifacts in deterministic order. - Reused current artifacts must be exposed to later scheduled dependents - through the runtime catalog without invoking Scriptorium. -2. For each scheduled artifact, keep output run-local until the adapter result - and output pass existing safety, contract, size, and checksum validation. - Materialize canonically only through the established file-operation owner. -3. Record the new fingerprint, full output evidence, producer run ID, bounded - non-secret provenance, logs, and generated configs in the Stage 7 projection. -4. After replacement, compare semantic output identity. If bytes/contract are - unchanged, allow an unselected dependent whose recomputed fingerprint is - equal to remain current. If identity changes, mark every unselected - configured dependent stale without executing it. -5. Preserve valid unrelated current records and outputs during partial runs. - Reconstruct aggregate session outputs from all current records; report only - newly produced outputs in the invocation manifest. -6. A partial invocation succeeds when every explicit target and required - prerequisite succeeds, even if unrelated configured records remain stale. -7. Never synthesize a current record from an existing canonical output. - -### Tests And Exit Criteria - -- Analyze tests cover current reuse, partial rerun preservation, nested stale - prerequisite rebuilding, selected force, dependency ordering, changed-output - dependent invalidation, and identical-output dependent preservation. -- Legacy tests prove full selection rebuilds the effective set, while partial - selection promotes only targets/prerequisites and leaves legacy unselected - outputs unavailable. -- Adapter fakes remain offline and assertions focus on requests, resulting - files, manifest records, and catalogs rather than private implementation maps. - -## Stage 13 — Incremental Analyze Failure Safety - -**Status: Completed** - -### Goal - -Complete the incremental executor with conservative, durable behavior for -partial adapter, validation, materialization, and persistence failures. - -### Required Work - -1. At each artifact boundary, retain enough reconciled state to return the - restricted Stage 7 projection if later work fails. Do not report a completion - until its run-local output has been validated and canonically materialized. -2. On failure, mark the active target `failed` with a bounded non-secret error. - Mark any artifact whose current identity depends on the unavailable result - stale, including unselected dependents, without executing them. -3. Preserve unrelated previous current records. Preserve earlier completions - from this invocation only when they crossed the defined materialization and - validation boundary; include them in invocation history. -4. Return the restricted state projection alongside the original error so the - runner can persist artifact state before marking aggregate analyze failed. - Dependency-aware invalidation must keep publish and notify conservative. -5. If persistence of the partial projection itself fails, surface that failure - with the original context, retain the last durable manifest, and treat any - newly materialized file as incidental rather than current evidence. -6. Never mark a failed or unverified artifact current merely because old bytes - remain at its canonical path. Never reconstruct partial success by scanning - output directories after an error. - -### Tests And Exit Criteria - -- Tests cover first, middle, and last artifact adapter failures; unsafe or - invalid adapter output; canonical materialization failure; and session/run - manifest persistence failure. -- Each case proves the failed target is unavailable, dependent state is stale, - unrelated current work survives, and only durably completed work appears in - invocation history. -- Tests prove old canonical bytes and incidental newly materialized bytes do - not override the durable manifest authority. -- Error wrapping remains actionable and non-secret, and the original adapter or - filesystem cause remains discoverable. - -## Stage 14 — Analyze Resume Validation And Plan/Run Parity - -**Status: Completed** - -### Goal - -Make aggregate analyze skipping and `session plan` reflect artifact-level -freshness using the same read-only decision engine as execution. - -### Required Work - -1. Implement an analyze-specific resume validator that runs Stages 8–11 - reconciliation for the requested artifact set. It is resumable only when all - explicit targets and required prerequisites are current and no selected work - is scheduled. -2. A succeeded aggregate analyze record with stale unrelated artifacts may - still skip for a partial selection that does not require them. A later full - selection must not skip them. -3. If coarse stage invalidation marked analyze stale but recomputation proves - selected fingerprints and outputs unchanged, allow analyze execution to - perform zero Scriptorium calls and restore the correct successful aggregate - state through the ordinary runner boundary. -4. Extend `session plan` to invoke the same stage resume validators and analyze - work planner as `run`, using a cloned/in-memory manifest transition model to - account for earlier selected stages and their invalidation. It must not - persist a session manifest, create an invocation directory, materialize a - file, or call an external adapter. -5. Plan output must distinguish pipeline-stage run/skip decisions and, for - analyze, explicit targets, prerequisite-only rebuilds, and current reuse with - concise reasons. Do not promise output identities that require execution. -6. Preserve the rule that excluded stages are never resume-validated. The - Stage 4 prefix status check remains structural and read-only. - -### Tests And Exit Criteria - -- Resume tests cover full and partial selections, force, current/stale/legacy - mixtures, current prerequisites, changed inputs, output tampering, and a - stale aggregate record with semantically current selected artifacts. -- Paired plan/run tests feed the same fixture and assert identical selected - stage and analyze-artifact decisions before execution. -- Side-effect tests prove planning performs no manifest write, run-directory - creation, canonical materialization, or adapter invocation. -- Existing extract resume validation remains functional in bounded and - unbounded plans. - -## Stage 15 — Assembled Workflow And Compatibility Coverage - -**Status: Completed** - -### Goal - -Verify the complete feature through production composition and representative -historical state without adding another behavior path. - -### Required Work - -1. Add assembled application tests for: - - an ordinary full run in the new canonical order; - - `run --from extract --through analyze --force` with and without artifact - selection; - - the exact alias reaching the same bounded-run request; - - render-only and extract-only forced runs demonstrating sibling - invalidation independence; - - a later publish consuming only current configured artifacts; and - - stop-on-failure with dependent stages outside the range left stale but - unexecuted. -2. Add compatibility fixtures or focused tests for manifests created under the - old render/extract order and aggregate-only analyze state. Confirm no - transcript stage is invalidated solely by version or old relative order. -3. Exercise a complete legacy transition: partial regeneration, unavailable - unselected legacy output, later full regeneration, then successful publish - from current per-artifact records. -4. Verify production composition does not eagerly require excluded adapters - and that generated run manifests list only the bounded requested stages and - actual invocation artifacts. -5. Remove obsolete suffix-invalidation, aggregate-analysis-replacement, and - filesystem-discovery compatibility code that no current path requires. Keep - any unavoidable compatibility shim narrow, clearly commented with its - removal condition, and covered by a legacy test. - -### Tests And Exit Criteria - -- Assembled tests use controlled fakes and real application wiring; they remain - offline and deterministic. -- The full suite passes under the standard test command and repository race or - platform checks required by policy. -- Coverage is allocated to behavior owners; this stage adds only integration - assertions that smaller tests cannot prove. - -## Stage 16 — Canonical Documentation And Quality Closure - -**Status: Completed** - -### Goal - -Bring every current-behavior owner into agreement and perform the final -repository-wide policy and regression check. - -### Required Work - -1. Audit current documentation for the old extract/render order, suffix-only - invalidation, unbounded-only run behavior, aggregate analysis replacement, - and filesystem-discovered configured artifacts. -2. Finalize: - - `docs/cli.md` for range syntax, duplicate singleton rejection, artifact - selection, exact alias expansion, and examples; - - `docs/operations.md` for the post-transcript development loop, force scope, - plan preview, explicit publish, and recovery; - - `docs/troubleshooting.md` for bounded prerequisite failures, stale or - tampered analysis evidence, legacy regeneration, fingerprint limitations, - and explicit force for unobserved Scriptorium inputs; - - `docs/internal/overview.md`, `stage-render.md`, `stage-extract.md`, - `stage-analyze.md`, `stage-publish.md`, `artifacts.md`, `manifest.md`, and - `adapters.md` for their implemented contracts; and - - `docs/policy/architecture.md` for flat canonical execution plus separate - dependency-aware invalidation and contiguous bounded runs. -3. Keep volatile syntax and schema details in their canonical owners. Other - documents should link rather than duplicate large flag or field inventories. -4. Confirm all examples use the new order and that none describe the fixed - dependency relation as a configurable DAG or claim perfect observation of - Scriptorium-private inputs. -5. Run formatting, documentation checks, static analysis, `go test ./...`, and - any race/platform checks required by repository policy. Review the final diff - for accidental generated files, secrets, broad refactors, stale compatibility - branches, and policy violations. -6. Update this plan's status only after all prior acceptance criteria pass. Do - not delete the accepted feature roadmap until the project's normal roadmap - closeout process authorizes it. - -### Tests And Exit Criteria - -- All maintained documentation agrees with implemented behavior and passes the - documentation policy's discoverability and ownership requirements. -- All repository-required checks pass from a clean checkout without installed - external adapter binaries. -- The worktree contains only intentional feature changes, no unresolved TODOs - standing in for this plan, and no known divergence from the feature roadmap. - -## Open Questions - -None. The accepted feature roadmap and the settled decisions above are -sufficient to implement every stage without a further product or architecture -choice. diff --git a/docs/roadmap/notarius-v0.6-cli-references.md b/docs/roadmap/notarius-v0.6-cli-references.md deleted file mode 100644 index 789c67c..0000000 --- a/docs/roadmap/notarius-v0.6-cli-references.md +++ /dev/null @@ -1,274 +0,0 @@ -# Notarius v0.6 CLI Reference Integration - -## Status - -Accepted target state. Delivery sequencing and implementation status are owned -by [implementation.md](implementation.md). - -## Purpose - -Upgrade Narratio's extraction boundary to the Notarius v0.6.0 subprocess -contract and supply session reference documents explicitly with repeatable -`--reference selector=path` arguments. - -The maintained D&D integration must make the prepared party roster, player -context, glossary, and optional spell catalog available to every compatible -Notarius target. Notarius must continue to own pipeline topology, reference-slot -compatibility, generated artifact handoffs, prompts, and D&D schemas. Narratio -owns selection and preparation of its external reference files, exact CLI -invocation, provenance, and extraction reuse correctness. - -## Current State And Gap - -Narratio currently invokes Notarius as: - -```text -notarius run --config --input --output-dir --json -``` - -The `prepare` stage already materializes campaign/session party, players, and -glossary files under the session `inputs/` directory, but `extract` does not -pass them to Notarius. Narratio also has no stable spell-catalog input. As a -result, a Notarius deployment must duplicate these paths in its own -configuration, cannot reliably receive session overrides, and may extract -without the same campaign context supplied to Narratio's analysis stage. - -Notarius v0.6.0 makes an unqualified CLI selector pipeline-scoped. For example, -`--reference party=/absolute/path/party.yml` supplies the file to every -selected target that declares `party`. Scoped selectors remain available for -exceptional overrides. CLI paths are resolved from the Notarius process working -directory, so subprocess callers are expected to provide absolute paths. - -The v0.6.0 receipt, index, warning, diagnostic, and ten-lane D&D artifact -contracts remain compatible with Narratio's current v0.5 integration. This -feature changes the invocation and input-provenance contract rather than the -accepted output inventory. - -## User Outcome - -With the maintained complete D&D configuration, an operator can declare the -campaign reference sources once in Narratio. For each extraction Narratio will: - -1. materialize the effective campaign/session files during `prepare`; -2. resolve those prepared files by stable Narratio source ID; -3. pass absolute paths for `party`, `players`, `glossary`, and, when configured, - `spell_catalog` to Notarius through repeatable CLI arguments; -4. fail before launching Notarius when a configured reference is unavailable; -5. rerun extraction when a selector, source binding, or reference file changes; - and -6. retain bounded reference identities and checksums for diagnosis and - provenance without copying reference contents into manifest metadata. - -Session-level stable-input overrides must flow through the same mechanism. A -custom Notarius pipeline may bind different external slots without requiring a -Narratio code change. - -## Chosen Architecture - -### Explicit Reference Bindings - -Extend `pipeline.notarius` with an explicit map from a Notarius CLI selector to -a prepared Narratio input source: - -```yaml -notarius: - enabled: true - binary: notarius - config_path: /usr/local/etc/notarius/config.yml - pipeline_id: dnd-session - working_directory: /usr/local/etc/notarius - references: - party: narratio.input.party - players: narratio.input.players - glossary: narratio.input.glossary - spell_catalog: narratio.input.spell_catalog - outputs: - # Existing required lane contracts remain unchanged. -``` - -Each configured binding is required. An operator who does not maintain an -optional Notarius reference, such as a spell catalog, omits that binding. This -keeps missing-input behavior explicit and avoids a second required/optional -policy inside each entry. - -The maintained complete D&D example will show all four external reference -slots. The three existing campaign context bindings use the canonical `party`, -`players`, and `glossary` spellings. Narratio will not emit the deprecated -`roster` alias. - -The binding is deliberately source-based rather than path-based. Pipeline -configuration should not reconstruct session workspace paths or bypass -`prepare`; it names the stable input whose effective campaign/session value is -already owned by Narratio. The map also avoids hard-coded behavior keyed to the -literal `dnd-session` pipeline ID, preserving custom-pipeline support. - -Narratio accepts the selector forms published by Notarius v0.6.0: - -- `slot`; -- `chunk.slot`; -- `lane.slot`; and -- `lane.extract.slot`, `lane.merge.slot`, or `lane.normalize.slot`. - -Configuration validation will reject empty or structurally invalid selectors, -selectors containing `=`, unsupported source IDs, and duplicate YAML keys. -Notarius remains authoritative for whether a selected target actually declares -the slot and whether a file satisfies that slot's media type and size contract. -Narratio will not duplicate the Notarius module registry. - -### Stable Reference Inputs - -Continue to use the existing prepared sources and canonical files: - -| Narratio source | Prepared file | Notarius slot | -| --- | --- | --- | -| `narratio.input.party` | `inputs/party.yml` | `party` | -| `narratio.input.players` | `inputs/players.yml` | `players` | -| `narratio.input.glossary` | `inputs/glossary.yml` | `glossary` | -| `narratio.input.spell_catalog` | `inputs/spell_catalog.json` | `spell_catalog` | - -Add optional `spell_catalog_file` fields to campaign and session inputs, with -the existing campaign-default/session-override resolution behavior. When -provided, `prepare` copies it into the session input area and records its -origin and checksum consistently with the other stable inputs. The prepared -filename remains JSON so Notarius can apply its published spell-catalog media -contract. - -The new source must be added everywhere stable inputs are enumerated: strict -configuration decoding and merging, validation, prepare materialization, -artifact policy/source descriptions, operator inspection, manifest input -records, examples, and canonical documentation. It remains optional at the -campaign level; a configured Notarius binding makes it mandatory for that -extraction. - -Extract and analyze should use one shared prepared-input source resolver rather -than maintain separate source-to-filename tables. The resolver must return an -absolute, regular, non-empty file beneath the current session workspace and -produce actionable `prepare --force` guidance when a configured source is -missing. It must not fall back to the original campaign path after preparation. - -### Adapter Request And CLI Construction - -Extend the transport-neutral Notarius run request with an ordered collection of -resolved reference bindings. Each binding contains only its selector and -absolute prepared-file path. The extraction stage resolves source IDs and file -identity; the subprocess adapter validates and serializes the request. - -The production command becomes: - -```text -notarius run - --config - --input - --output-dir - --reference party= - --reference players= - --reference glossary= - --reference spell_catalog= - --json -``` - -Only configured bindings are emitted. Selectors are sorted before request -construction so argument order, tests, logs, and fingerprints are deterministic. -Arguments are passed directly to the subprocess without shell interpretation; -paths containing spaces or platform-specific separators remain one argument. - -CLI bindings intentionally override matching external paths in the deployed -Notarius configuration. Narratio must not pass `--without-reference` and must -not synthesize CLI bindings for `location_registry`, `item_registry`, -`npc_registry`, `scene_descriptions`, `combat_turns`, or `npc_occurrences`. -Those are generated same-run artifact handoffs in the complete D&D pipeline and -remain entirely under Notarius configuration and execution control. A custom -configuration that collides an external CLI binding with a generated handoff is -expected to fail with Notarius's normal resolution error. - -### Fingerprints, Resume, And Provenance - -Reference identity is part of the extraction input contract. The extraction -fingerprint and resume validator must include, in deterministic selector order: - -- the selector; -- the configured Narratio source ID; -- the resolved prepared path identity; and -- the prepared file's content checksum and size. - -This is required even though Notarius generates a prompt session ID from the -input module and transcript bytes: Notarius intentionally does not include -references in that identifier. Narratio must therefore prevent an old -extraction from being reused after a roster, player list, glossary, spell -catalog, selector, or source mapping changes. - -A changed reference makes the prior `extract` result non-reusable and follows -Narratio's normal downstream invalidation rules. A failed reference-resolution -or checksum check also prevents reuse; it must not silently accept the prior -bundle. - -Successful extract metadata should record a bounded, deterministic list of -selector, source ID, workspace-relative path, checksum, and size. It must not -record reference contents, original absolute operator paths, or values from the -files. Existing receipt and bundle provenance behavior remains unchanged. - -### Error And Compatibility Behavior - -Narratio's documented minimum supported Notarius version becomes v0.6.0 for an -enabled reference binding. Compatibility remains contract-based rather than -dependent on parsing `notarius --version`: an older or incompatible executable -will fail at the CLI boundary with captured diagnostics. - -Errors must identify the responsible selector and Narratio source without -including file contents. Configuration errors are reported before pipeline -execution. Missing, empty, non-regular, unsafe, or unreadable prepared files -fail extraction before the Notarius subprocess starts. Notarius continues to -report undeclared slots, media incompatibility, size limits, required-slot -failures, and generated-handoff collisions. - -When Notarius is disabled, extraction retains its current explicit skip -behavior and does not resolve reference inputs. Receipt v2 ingestion, bundle -confinement, ten-lane selection, and downstream artifact source IDs are not -otherwise changed by this feature. - -## Target End State - -Narratio and Notarius have a clear orchestration boundary: - -- `prepare` owns the effective, immutable session copies of external campaign - context; -- `extract` maps configured stable source IDs to Notarius v0.6 CLI selectors, - supplies absolute file paths, and owns reuse/provenance policy; -- the Notarius adapter owns exact subprocess serialization and supported result - decoding; -- Notarius owns slot compatibility, reference precedence within its pipeline, - generated artifact handoffs, and output schemas; and -- `analyze` consumes the resulting ten structured lane artifacts exactly as it - does today. - -The maintained complete D&D workflow passes party, players, glossary, and spell -catalog context from the same prepared session inputs used elsewhere in -Narratio. Updating any of those documents deterministically causes fresh -extraction, and operators can diagnose the effective bindings without exposing -file contents. - -## Out Of Scope - -- Reproducing Notarius pipeline, lane, binding, or media-type validation in - Narratio. -- Passing or overriding Notarius generated artifact handoffs. -- Adding `--without-reference`, Notarius resume/recompute controls, lane - selection, model selection, profile selection, or session-ID overrides. -- Changing the ten accepted D&D lane contracts or the Scriptorium analysis - design. -- Reading reference payloads into Narratio manifests or logs. -- Automatically running `notarius config validate` for every session. - -## Settled Policy Choices - -The implementation must preserve these choices unless implementation evidence -shows a contract conflict: - -- explicit selector-to-source mappings are preferred over pipeline-ID-specific - defaults; -- every configured mapping is required; -- `spell_catalog_file` is optional until a mapping requests its prepared - source; -- the complete D&D example demonstrates all four external references; and -- Notarius v0.6.0 is the minimum supported CLI contract for reference-enabled - extraction. diff --git a/docs/roadmap/post-transcript-artifact-workflow.md b/docs/roadmap/post-transcript-artifact-workflow.md deleted file mode 100644 index cf3345f..0000000 --- a/docs/roadmap/post-transcript-artifact-workflow.md +++ /dev/null @@ -1,409 +0,0 @@ -# Post-Transcript Artifact Development Workflow - -## Status - -Implemented. The completed delivery sequence is retained in -[`implementation.md`](implementation.md). - -## Goal - -Make repeated development of extraction and analysis artifacts fast, explicit, -and safe after a session's transcripts are complete. Narratio should expose a -clear transcript/post-transcript boundary, allow operators to run a bounded -part of its canonical pipeline, provide one transparent convenience alias for -the common forced-regeneration workflow, and reuse analysis artifacts whose -observable inputs have not changed. - -The feature must preserve Narratio's intentionally simple orchestration model: -one fixed stage sequence, explicit stages, manifest-authoritative state, and no -configurable workflow graph. - -## User Intent - -Transcript production is comparatively infrequent after a session reaches a -good final transcript. Development of Notarius extraction and Scriptorium -artifacts continues much longer and commonly requires repeated execution. - -The normal development workflow should therefore: - -- treat completed transcript work as read-only unless the operator explicitly - selects transcript stages; -- regenerate extraction and analysis without attempting earlier stages; -- make forced scope visible in the command itself; -- support focused Scriptorium artifact selection; and -- avoid rerunning unrelated analysis artifacts when their meaningful inputs - and dependencies are unchanged. - -Persistent transcript seals and run-to-run comparison tools are intentionally -deferred. Bounded execution provides the immediate mutation boundary without -adding another kind of durable lock. - -## Canonical Pipeline And Phase Boundary - -Move `render` before `extract` so all transcript representations are complete -before post-transcript artifact generation begins. The canonical order becomes: - -1. `prepare` -2. `transcribe` -3. `merge` -4. `polish` -5. `normalize` -6. `trim` -7. `render` -8. `extract` -9. `analyze` -10. `publish` -11. `notify` - -This creates two useful conceptual regions without making phases configurable: - -- transcript production: `prepare` through `render`; and -- post-transcript processing and delivery: `extract` through `notify`. - -`render` and `extract` are independent sibling consumers of completed -transcript data. Render uses the canonical final and final-trimmed transcripts -to create Markdown representations. Extract uses the canonical final-trimmed -transcript and prepared references to create Notarius artifacts. Neither stage -consumes the other's output. - -Narratio's render path is deterministic. A render failure will stop a full run -before extraction under the new order, and that behavior is accepted: a full -run should complete transcript representations before starting post-transcript -work. Recovery remains available through continuation and bounded or -single-stage execution. - -## Execution Order And Invalidation - -Execution order and invalidation dependencies must have separate explicit -owners. The canonical stage registry defines when stages are attempted. A -central, fixed, configuration-independent invalidation relation defines which -recorded results may no longer be trustworthy after a stage outcome changes. - -The relation is conservative across supported configurations. For example, -`analyze` remains dependent on `render` because a configured artifact may -consume rendered Markdown, even if one particular pipeline does not. - -The required transitive invalidation sets, returned in canonical execution -order, are: - -| Changed stage | Succeeded stages eligible to become stale | -| --- | --- | -| `prepare` | `transcribe`, `merge`, `polish`, `normalize`, `trim`, `render`, `extract`, `analyze`, `publish`, `notify` | -| `transcribe` | `merge`, `polish`, `normalize`, `trim`, `render`, `extract`, `analyze`, `publish`, `notify` | -| `merge` | `polish`, `normalize`, `trim`, `render`, `extract`, `analyze`, `publish`, `notify` | -| `polish` | `normalize`, `trim`, `render`, `extract`, `analyze`, `publish`, `notify` | -| `normalize` | `trim`, `render`, `extract`, `analyze`, `publish`, `notify` | -| `trim` | `render`, `extract`, `analyze`, `publish`, `notify` | -| `render` | `analyze`, `publish`, `notify` | -| `extract` | `analyze`, `publish`, `notify` | -| `analyze` | `publish`, `notify` | -| `publish` | `notify` | -| `notify` | none | - -In particular, render and extract must not invalidate one another. A change to -either still invalidates analysis and delivery, while a change to trim -invalidates both branches and their consumers. - -The relation applies to every existing invalidation trigger, including forced -replacement, a non-resumable success, failure, and a changed effective outcome. -Only succeeded dependent stage records become stale under the existing status -rules. Failed and incomplete records retain their meaning, and an identical -repeated self-skip does not cause perpetual reruns. - -The application owner must validate the fixed relation against the canonical -stage inventory so a stage addition, removal, rename, duplication, or missing -classification cannot silently produce incorrect invalidation behavior. This -relation is not configurable and is not an alternate execution planner. - -## Bounded Canonical Execution - -Extend `run` with inclusive canonical bounds: - -```text -narratio run [--from ] [--through ] [--force] -``` - -Examples: - -```bash -narratio run SESSION --from extract --through analyze --force -narratio run SESSION --from render --through render --force -narratio run SESSION --from analyze --through analyze -``` - -The bounds have these settled semantics: - -- they select one contiguous slice of the fixed canonical stage sequence; -- `--from` defaults to the first stage and `--through` defaults to the last - stage when omitted; -- both stage names must exist, and `--from` must not occur after `--through`; -- with neither option, `run` retains its current full-pipeline behavior; -- `--force` applies only to stages inside the selected range; -- stages before and after the range are not executed or resume-validated; -- excluded upstream records and artifacts may be resolved and validated as - stage inputs, but Narratio must not repair or regenerate them implicitly; -- missing, stale, unsafe, or otherwise unusable prerequisites produce an - actionable error rather than widening the requested range; -- invalidation caused by an executed stage may mark dependent stages outside - the range stale, but those stages are not executed; and -- stage failure retains the existing stop-on-failure behavior. - -`session plan` must accept the same bounds, force scope, and artifact selection -needed to preview the corresponding `run` without executing stages. Plan and -run must use one selection implementation so their range validation and -run/skip decisions cannot drift. - -`--artifacts` retains its existing meaning for `analyze` and `publish` when -those stages are inside the selected range. Supplying artifact selection for a -range containing neither consumer is an error rather than a silent no-op. -Repeated artifact-selection flags retain their existing normalization and -deduplication behavior. - -Production composition should follow the bounded plan. Selecting a range must -not require an adapter used only by an excluded stage, while a selected stage -continues to require and validate its own collaborators. - -## `regenerate-artifacts` Convenience Alias - -Add this top-level command: - -```text -narratio regenerate-artifacts [--artifacts ] -``` - -It is exactly a convenience alias for: - -```text -narratio run --force --from extract --through analyze [--artifacts ] -``` - -The alias has no independent orchestration semantics, prerequisites, force -rules, or execution path. Its implementation must rewrite or construct the -equivalent `run` arguments before invoking the shared run parser and handler. -All common session/configuration arguments and repeatable `--artifacts` values -pass through to `run` unchanged. - -The shared parser owns validation, planning, execution, errors, and summaries. -Alias help must state the exact equivalence. It is acceptable and desirable for -runtime errors and summaries to identify the canonical `run` operation. The -alias must not gain private flags or behavior; a future capability belongs on -`run` first. - -Because `--force`, `--from`, and `--through` define the alias, callers must not -override them. The shared command parsing layer should reject duplicate -singleton options rather than use ordering to choose a winner. That rule should -apply consistently to bounded `run` itself, not only to the alias. - -Without `--artifacts`, the alias force-runs extraction and all enabled -configured analysis artifacts, matching the existing default analysis -selection. With `--artifacts`, extraction still produces its complete -configured Notarius bundle, while forced analysis targets only the selected -Scriptorium artifacts and any prerequisites required to build them. An -explicitly selected disabled artifact remains a valid target under the existing -selection rules. The alias never runs publish or notify. Changed results may -correctly leave those later stages stale. - -The existing `analyze` command remains the convenience path for forcing -analysis without rerunning Notarius. - -## Incremental Analysis Artifacts - -### Analyze-Owned State - -`analyze` currently has one aggregate stage result. Extend its manifest-owned -state so each configured Scriptorium artifact has an explicit current result -identity. Keep this model specific to analysis artifacts; do not introduce -dynamic pipeline stages or a generic subtask framework without another proven -consumer. - -Each current artifact result must identify at least: - -- the normalized configured artifact key; -- a versioned input fingerprint; -- the output source ID, contract, confined canonical path, checksum, and size; -- the producing Narratio run identity and useful non-secret Scriptorium - provenance; and -- enough status or disposition information to distinguish current, stale, - missing, failed, and intentionally unselected work. - -The session manifest remains the authority for current availability. An -incidental output file is not current merely because it exists. Invocation -manifests continue to record what one run attempted and produced. - -### Artifact Fingerprints - -Define one deterministic, versioned fingerprint per configured artifact using -all Narratio-observable inputs that can change its result: - -- its normalized Scriptorium artifact configuration; -- its ordered input names, source IDs, required/optional policy, and resolved - input content identities; -- transcript, prepared-input, previous-session, extraction-lane, and other - configured artifact contracts and content checksums; -- the current content identities of configured artifact dependencies; -- result-affecting Scriptorium adapter configuration visible to Narratio; and -- an explicit fingerprint contract version. - -Fingerprint ordering must be deterministic. Identity must not change solely -because a workspace moved, an absolute path changed, or an otherwise identical -producer used a new run ID. In particular, a forced Notarius invocation that -produces byte-identical lanes must not make unrelated analysis artifacts stale -solely because the extraction run identity changed. - -Narratio cannot observe arbitrary files, prompts, modules, executable contents, -or transitive configuration loaded privately by Scriptorium. Documentation must -state that changing an unobserved external input requires explicit force. Do -not claim perfect content-addressed reuse beyond Narratio's declared inputs. - -### Freshness And Selection - -Before skipping a succeeded `analyze` stage, an analyze-specific resume -validator must reconcile the requested artifact set against current -configuration, dependencies, input fingerprints, output records, confined -regular files, and stored output checksums. - -The execution rules are: - -- an ordinary run executes only requested artifacts that are missing, stale, - invalid, or no longer resumable; -- forcing analyze rebuilds all requested targets even when their fingerprints - are current; -- `--artifacts` identifies explicit targets, not the complete set of records - that may remain current; -- a selected target's configured prerequisites are processed in deterministic - dependency order, reusing them when current and rebuilding them when stale; -- forcing a target does not force an otherwise current prerequisite unless it - was also explicitly selected; -- valid unselected artifact records and outputs survive a partial rerun; -- artifacts removed or renamed in current configuration cease to be advertised - as current; -- an artifact whose dependency or resolved input changes becomes stale unless - the new semantic content identity is unchanged; and -- stale, missing, failed, or unverified artifacts are unavailable to downstream - catalog and publish resolution even if an older file remains on disk. - -If a rebuilt artifact changes, configured dependents that were not part of the -invocation are not silently rebuilt. They become stale and will be rebuilt by a -later run that selects them. If the rebuilt output is content-identical and the -dependent fingerprint remains equal, the dependent may remain current. - -A partial invocation succeeds when its explicit targets and required -prerequisites succeed. The aggregate stage record may therefore describe a -successful partial invocation while other configured artifacts are stale. The -resume validator, not aggregate status alone, must ensure a later full run does -not skip unresolved artifact work. - -### Replacement And Failure Safety - -Artifact replacement must preserve unrelated current results while ensuring a -failed target is not presented as freshly generated. Run-local output must be -validated before canonical materialization and manifest promotion, consistent -with existing stage safety policy. - -On partial failure: - -- successfully completed and validated targets may be recorded in the - invocation history according to existing runner transaction boundaries; -- the failed target and any result whose current identity depends on it must - not be advertised as current; -- unrelated previously validated artifacts must not be erased merely because - they share the `analyze` stage; and -- publish and later stage state must remain conservatively stale or failed. - -The implementation must define one clear manifest transition boundary and must -not synthesize current output records from directory contents. - -### Legacy Analyze Results - -Existing manifests may contain only an aggregate analyze success and outputs, -without versioned per-artifact fingerprints. They remain readable, but Narratio -must not invent trustworthy fingerprints for work whose exact inputs were not -recorded. - -On first incremental evaluation, legacy analysis artifacts are non-resumable. -A full analysis selection rebuilds the effective configured set. A partial -selection may rebuild its targets and prerequisites, but unselected legacy -outputs remain stale and unavailable until regenerated. Old files and -invocation manifests may remain for inspection under existing retention rules. -No wholesale manifest rewrite or version-based transcript invalidation is -required. - -## Resume And Existing Pipeline Manifests - -The render/extract order change itself requires no manifest migration because -stage records use stable names. Under the new sequence: - -- succeeded render and extract records remain eligible for ordinary reuse and - their stage-specific validation; -- stale, failed, interrupted, and absent records execute in the new order; and -- neither result is discarded merely because its relative position changed. - -Bounded execution does not rewrite excluded stage records. Compatibility logic -must remain name- and evidence-based; do not invalidate historical transcript -work solely because it was produced by an earlier Narratio version. - -The incremental-analysis model may add backward-compatible manifest fields or -versioned metadata. Readers must accept older manifests, while new writers must -emit one canonical representation and must not maintain parallel legacy and new -analysis state indefinitely. - -## Compatibility And Operational Effects - -- Existing unbounded `run`, `run-stage`, `analyze`, and `publish` commands keep - their current meanings except for the accepted render/extract order change - and more precise analysis reuse. -- `--from` and `--through` are additive CLI options; configuration does not gain - stage-order or range fields. -- `regenerate-artifacts` adds no semantics beyond its documented `run` alias. -- Bounded forced runs cannot mutate transcript stages outside their range. -- Full runs attempt deterministic render before invoking Notarius. -- Forcing render no longer regenerates an otherwise valid Notarius bundle, and - forcing extraction no longer regenerates Markdown. -- Artifact-level validation adds filesystem hashing and fingerprint work before - some analyze skips, trading modest local inspection cost for fewer - Scriptorium invocations. -- The first analysis evaluation after upgrade may require regeneration because - legacy aggregate results do not contain sufficient freshness evidence. - -## Out Of Scope - -- Persistent transcript seals, transcript freeze state, or another lock type. -- Run-history listing, run-to-run artifact comparison, or draft promotion. -- Parallel execution of render and extract or of analysis artifacts. -- Non-contiguous stage selection. -- User-configurable stage order or invalidation dependencies. -- A generic DAG, phase, job, workflow, or manifest-subtask framework. -- New Seriatim, Notarius, or Scriptorium CLI capabilities. -- Changes to Notarius lane contracts or Scriptorium output schemas. -- Automatic observation of arbitrary transitive Scriptorium files or executable - contents. -- New render retry, caching, resumability, or fingerprint behavior. -- Automatic publish or notification as part of `regenerate-artifacts`. - -## Target End State - -Narratio has one comprehensible pipeline in which transcript production ends at -render and post-transcript generation begins at extract. Operators can run any -contiguous canonical range without accidentally executing stages outside it, -and force applies only within the requested range. - -The common development command: - -```bash -narratio regenerate-artifacts SESSION -``` - -is transparently identical to a forced bounded run from extract through -analyze. It preserves transcript state, regenerates the complete Notarius -bundle, rebuilds the requested Scriptorium targets, and leaves delivery as a -separate explicit action. - -Analysis artifacts have independent, manifest-authoritative freshness within -the fixed `analyze` stage. Narratio reuses valid unselected work, rebuilds stale -dependencies and selected targets deterministically, withholds stale outputs -from downstream consumers, and recognizes content-identical upstream results -without tying reuse to ephemeral run paths or IDs. - -Together, canonical ordering, dependency-aware invalidation, bounded execution, -the transparent alias, and artifact-level analysis reuse provide an ergonomic -development loop without turning Narratio into a general workflow engine.