From 9bc1b0fedacebaa3b8aa52e8b4c5ddb012c7e936 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sat, 29 Aug 2026 17:29:08 +0000 Subject: [PATCH] Plan post-transcript artifact regeneration --- .../post-transcript-artifact-workflow.md | 460 ++++++++++++++++++ 1 file changed, 460 insertions(+) create mode 100644 docs/roadmap/post-transcript-artifact-workflow.md diff --git a/docs/roadmap/post-transcript-artifact-workflow.md b/docs/roadmap/post-transcript-artifact-workflow.md new file mode 100644 index 0000000..461f480 --- /dev/null +++ b/docs/roadmap/post-transcript-artifact-workflow.md @@ -0,0 +1,460 @@ +# Post-Transcript Artifact Development Workflow + +## Status + +Accepted target state; not yet implemented. + +## Goal + +Make repeated development of extraction and analysis artifacts fast, explicit, +and safe after a session's transcripts are complete. Narratio should expose a +clear transcript/post-transcript boundary, allow operators to run a bounded +part of its canonical pipeline, provide one transparent convenience alias for +the common forced-regeneration workflow, and reuse analysis artifacts whose +observable inputs have not changed. + +The feature must preserve Narratio's intentionally simple orchestration model: +one fixed stage sequence, explicit stages, manifest-authoritative state, and no +configurable workflow graph. + +## User Intent + +Transcript production is comparatively infrequent after a session reaches a +good final transcript. Development of Notarius extraction and Scriptorium +artifacts continues much longer and commonly requires repeated execution. + +The normal development workflow should therefore: + +- treat completed transcript work as read-only unless the operator explicitly + selects transcript stages; +- regenerate extraction and analysis without attempting earlier stages; +- make forced scope visible in the command itself; +- support focused Scriptorium artifact selection; and +- avoid rerunning unrelated analysis artifacts when their meaningful inputs + and dependencies are unchanged. + +Persistent transcript seals and run-to-run comparison tools are intentionally +deferred. Bounded execution provides the immediate mutation boundary without +adding another kind of durable lock. + +## Canonical Pipeline And Phase Boundary + +Move `render` before `extract` so all transcript representations are complete +before post-transcript artifact generation begins. The canonical order becomes: + +1. `prepare` +2. `transcribe` +3. `merge` +4. `polish` +5. `normalize` +6. `trim` +7. `render` +8. `extract` +9. `analyze` +10. `publish` +11. `notify` + +This creates two useful conceptual regions without making phases configurable: + +- transcript production: `prepare` through `render`; and +- post-transcript processing and delivery: `extract` through `notify`. + +`render` and `extract` are independent sibling consumers of completed +transcript data. Render uses the canonical final and final-trimmed transcripts +to create Markdown representations. Extract uses the canonical final-trimmed +transcript and prepared references to create Notarius artifacts. Neither stage +consumes the other's output. + +Narratio's render path is deterministic. A render failure will stop a full run +before extraction under the new order, and that behavior is accepted: a full +run should complete transcript representations before starting post-transcript +work. Recovery remains available through continuation and bounded or +single-stage execution. + +## Execution Order And Invalidation + +Execution order and invalidation dependencies must have separate explicit +owners. The canonical stage registry defines when stages are attempted. A +central, fixed, configuration-independent invalidation relation defines which +recorded results may no longer be trustworthy after a stage outcome changes. + +The relation is conservative across supported configurations. For example, +`analyze` remains dependent on `render` because a configured artifact may +consume rendered Markdown, even if one particular pipeline does not. + +The required transitive invalidation sets, returned in canonical execution +order, are: + +| Changed stage | Succeeded stages eligible to become stale | +| --- | --- | +| `prepare` | `transcribe`, `merge`, `polish`, `normalize`, `trim`, `render`, `extract`, `analyze`, `publish`, `notify` | +| `transcribe` | `merge`, `polish`, `normalize`, `trim`, `render`, `extract`, `analyze`, `publish`, `notify` | +| `merge` | `polish`, `normalize`, `trim`, `render`, `extract`, `analyze`, `publish`, `notify` | +| `polish` | `normalize`, `trim`, `render`, `extract`, `analyze`, `publish`, `notify` | +| `normalize` | `trim`, `render`, `extract`, `analyze`, `publish`, `notify` | +| `trim` | `render`, `extract`, `analyze`, `publish`, `notify` | +| `render` | `analyze`, `publish`, `notify` | +| `extract` | `analyze`, `publish`, `notify` | +| `analyze` | `publish`, `notify` | +| `publish` | `notify` | +| `notify` | none | + +In particular, render and extract must not invalidate one another. A change to +either still invalidates analysis and delivery, while a change to trim +invalidates both branches and their consumers. + +The relation applies to every existing invalidation trigger, including forced +replacement, a non-resumable success, failure, and a changed effective outcome. +Only succeeded dependent stage records become stale under the existing status +rules. Failed and incomplete records retain their meaning, and an identical +repeated self-skip does not cause perpetual reruns. + +The application owner must validate the fixed relation against the canonical +stage inventory so a stage addition, removal, rename, duplication, or missing +classification cannot silently produce incorrect invalidation behavior. This +relation is not configurable and is not an alternate execution planner. + +## Bounded Canonical Execution + +Extend `run` with inclusive canonical bounds: + +```text +narratio run [--from ] [--through ] [--force] +``` + +Examples: + +```bash +narratio run SESSION --from extract --through analyze --force +narratio run SESSION --from render --through render --force +narratio run SESSION --from analyze --through analyze +``` + +The bounds have these settled semantics: + +- they select one contiguous slice of the fixed canonical stage sequence; +- `--from` defaults to the first stage and `--through` defaults to the last + stage when omitted; +- both stage names must exist, and `--from` must not occur after `--through`; +- with neither option, `run` retains its current full-pipeline behavior; +- `--force` applies only to stages inside the selected range; +- stages before and after the range are not executed or resume-validated; +- excluded upstream records and artifacts may be resolved and validated as + stage inputs, but Narratio must not repair or regenerate them implicitly; +- missing, stale, unsafe, or otherwise unusable prerequisites produce an + actionable error rather than widening the requested range; +- invalidation caused by an executed stage may mark dependent stages outside + the range stale, but those stages are not executed; and +- stage failure retains the existing stop-on-failure behavior. + +`session plan` must accept the same bounds, force scope, and artifact selection +needed to preview the corresponding `run` without executing stages. Plan and +run must use one selection implementation so their range validation and +run/skip decisions cannot drift. + +`--artifacts` retains its existing meaning for `analyze` and `publish` when +those stages are inside the selected range. Supplying artifact selection for a +range containing neither consumer is an error rather than a silent no-op. +Repeated artifact-selection flags retain their existing normalization and +deduplication behavior. + +Production composition should follow the bounded plan. Selecting a range must +not require an adapter used only by an excluded stage, while a selected stage +continues to require and validate its own collaborators. + +## `regenerate-artifacts` Convenience Alias + +Add this top-level command: + +```text +narratio regenerate-artifacts [--artifacts ] +``` + +It is exactly a convenience alias for: + +```text +narratio run --force --from extract --through analyze [--artifacts ] +``` + +The alias has no independent orchestration semantics, prerequisites, force +rules, or execution path. Its implementation must rewrite or construct the +equivalent `run` arguments before invoking the shared run parser and handler. +All common session/configuration arguments and repeatable `--artifacts` values +pass through to `run` unchanged. + +The shared parser owns validation, planning, execution, errors, and summaries. +Alias help must state the exact equivalence. It is acceptable and desirable for +runtime errors and summaries to identify the canonical `run` operation. The +alias must not gain private flags or behavior; a future capability belongs on +`run` first. + +Because `--force`, `--from`, and `--through` define the alias, callers must not +override them. The shared command parsing layer should reject duplicate +singleton options rather than use ordering to choose a winner. That rule should +apply consistently to bounded `run` itself, not only to the alias. + +Without `--artifacts`, the alias force-runs extraction and all configured +analysis artifacts. With `--artifacts`, extraction still produces its complete +configured Notarius bundle, while forced analysis targets only the selected +Scriptorium artifacts and any prerequisites required to build them. It never +runs publish or notify. Changed results may correctly leave those later stages +stale. + +The existing `analyze` command remains the convenience path for forcing +analysis without rerunning Notarius. + +## Incremental Analysis Artifacts + +### Analyze-Owned State + +`analyze` currently has one aggregate stage result. Extend its manifest-owned +state so each configured Scriptorium artifact has an explicit current result +identity. Keep this model specific to analysis artifacts; do not introduce +dynamic pipeline stages or a generic subtask framework without another proven +consumer. + +Each current artifact result must identify at least: + +- the normalized configured artifact key; +- a versioned input fingerprint; +- the output source ID, contract, confined canonical path, checksum, and size; +- the producing Narratio run identity and useful non-secret Scriptorium + provenance; and +- enough status or disposition information to distinguish current, stale, + missing, failed, and intentionally unselected work. + +The session manifest remains the authority for current availability. An +incidental output file is not current merely because it exists. Invocation +manifests continue to record what one run attempted and produced. + +### Artifact Fingerprints + +Define one deterministic, versioned fingerprint per configured artifact using +all Narratio-observable inputs that can change its result: + +- its normalized Scriptorium artifact configuration; +- its ordered input names, source IDs, required/optional policy, and resolved + input content identities; +- transcript, prepared-input, previous-session, extraction-lane, and other + configured artifact contracts and content checksums; +- the current content identities of configured artifact dependencies; +- result-affecting Scriptorium adapter configuration visible to Narratio; and +- an explicit fingerprint contract version. + +Fingerprint ordering must be deterministic. Identity must not change solely +because a workspace moved, an absolute path changed, or an otherwise identical +producer used a new run ID. In particular, a forced Notarius invocation that +produces byte-identical lanes must not make unrelated analysis artifacts stale +solely because the extraction run identity changed. + +Narratio cannot observe arbitrary files, prompts, modules, executable contents, +or transitive configuration loaded privately by Scriptorium. Documentation must +state that changing an unobserved external input requires explicit force. Do +not claim perfect content-addressed reuse beyond Narratio's declared inputs. + +### Freshness And Selection + +Before skipping a succeeded `analyze` stage, an analyze-specific resume +validator must reconcile the requested artifact set against current +configuration, dependencies, input fingerprints, output records, confined +regular files, and stored output checksums. + +The execution rules are: + +- an ordinary run executes only requested artifacts that are missing, stale, + invalid, or no longer resumable; +- forcing analyze rebuilds all requested targets even when their fingerprints + are current; +- `--artifacts` identifies explicit targets, not the complete set of records + that may remain current; +- a selected target's configured prerequisites are processed in deterministic + dependency order, reusing them when current and rebuilding them when stale; +- forcing a target does not force an otherwise current prerequisite unless it + was also explicitly selected; +- valid unselected artifact records and outputs survive a partial rerun; +- artifacts removed or renamed in current configuration cease to be advertised + as current; +- an artifact whose dependency or resolved input changes becomes stale unless + the new semantic content identity is unchanged; and +- stale, missing, failed, or unverified artifacts are unavailable to downstream + catalog and publish resolution even if an older file remains on disk. + +If a rebuilt artifact changes, configured dependents that were not part of the +invocation are not silently rebuilt. They become stale and will be rebuilt by a +later run that selects them. If the rebuilt output is content-identical and the +dependent fingerprint remains equal, the dependent may remain current. + +A partial invocation succeeds when its explicit targets and required +prerequisites succeed. The aggregate stage record may therefore describe a +successful partial invocation while other configured artifacts are stale. The +resume validator, not aggregate status alone, must ensure a later full run does +not skip unresolved artifact work. + +### Replacement And Failure Safety + +Artifact replacement must preserve unrelated current results while ensuring a +failed target is not presented as freshly generated. Run-local output must be +validated before canonical materialization and manifest promotion, consistent +with existing stage safety policy. + +On partial failure: + +- successfully completed and validated targets may be recorded in the + invocation history according to existing runner transaction boundaries; +- the failed target and any result whose current identity depends on it must + not be advertised as current; +- unrelated previously validated artifacts must not be erased merely because + they share the `analyze` stage; and +- publish and later stage state must remain conservatively stale or failed. + +The implementation must define one clear manifest transition boundary and must +not synthesize current output records from directory contents. + +### Legacy Analyze Results + +Existing manifests may contain only an aggregate analyze success and outputs, +without versioned per-artifact fingerprints. They remain readable, but Narratio +must not invent trustworthy fingerprints for work whose exact inputs were not +recorded. + +On first incremental evaluation, legacy analysis artifacts are non-resumable. +A full analysis selection rebuilds the effective configured set. A partial +selection may rebuild its targets and prerequisites, but unselected legacy +outputs remain stale and unavailable until regenerated. Old files and +invocation manifests may remain for inspection under existing retention rules. +No wholesale manifest rewrite or version-based transcript invalidation is +required. + +## Resume And Existing Pipeline Manifests + +The render/extract order change itself requires no manifest migration because +stage records use stable names. Under the new sequence: + +- succeeded render and extract records remain eligible for ordinary reuse and + their stage-specific validation; +- stale, failed, interrupted, and absent records execute in the new order; and +- neither result is discarded merely because its relative position changed. + +Bounded execution does not rewrite excluded stage records. Compatibility logic +must remain name- and evidence-based; do not invalidate historical transcript +work solely because it was produced by an earlier Narratio version. + +The incremental-analysis model may add backward-compatible manifest fields or +versioned metadata. Readers must accept older manifests, while new writers must +emit one canonical representation and must not maintain parallel legacy and new +analysis state indefinitely. + +## Required Test Coverage + +Allocate tests to the behavior owner and avoid duplicating every case at every +layer: + +- planner and command tests prove the new canonical order, inclusive range + selection, omitted-bound defaults, invalid/reversed bounds, duplicate-option + rejection, artifact-range validation, and unchanged unbounded behavior; +- alias tests prove exact argument equivalence and pass-through without + re-testing the runner through an independent path; +- invalidation tests prove the complete fixed relation, inventory validation, + deterministic ordering, unknown-stage handling, and render/extract + independence; +- runner tests prove force is confined to the selected range, excluded stages + are not resume-validated or composed unnecessarily, outside dependents may be + staled but not executed, and unusable prerequisites do not widen the range; +- continuation tests cover representative manifests produced under the old + render/extract order; +- analyze tests prove fingerprint stability and sensitivity, dependency + ordering, selected force, stale-prerequisite rebuilding, valid unselected + preservation, changed-dependent invalidation, identical-content reuse, + removed configuration, output tampering, partial failure, and legacy-result + handling; and +- catalog and publish tests prove that only current per-artifact results are + exposed downstream. + +Tests must assert observable plans, manifest transitions, adapter requests, and +artifact availability rather than private map layout. The default suite remains +offline, deterministic, and independent of installed Seriatim, Notarius, or +Scriptorium binaries. + +## Documentation And Policy Changes + +When implementation lands, update all current-behavior owners in the same +change: + +- revise the architecture policy so canonical order owns execution, + dependency-aware invalidation owns affected work, and bounded runs remain + contiguous selections rather than arbitrary workflows; +- document the new order, range flags, exact alias expansion, force scope, + plan behavior, incremental reuse, and legacy-analysis transition in the CLI + and operations references; +- update the internal overview and the focused render, extract, analyze, + manifest, artifact, adapter, and publish documents where their implemented + contracts change; +- update troubleshooting guidance for incomplete bounded prerequisites, stale + analysis artifacts, fingerprint limitations, and forced recovery; and +- remove every current claim that extract runs between trim and render or that + all invalidation is a canonical-order suffix. + +Documentation should continue to describe one explicit pipeline, not a DAG +scheduler. Volatile command syntax and manifest details remain in their +canonical owners rather than being copied across orientation documents. + +## Compatibility And Operational Effects + +- Existing unbounded `run`, `run-stage`, `analyze`, and `publish` commands keep + their current meanings except for the accepted render/extract order change + and more precise analysis reuse. +- `--from` and `--through` are additive CLI options; configuration does not gain + stage-order or range fields. +- `regenerate-artifacts` adds no semantics beyond its documented `run` alias. +- Bounded forced runs cannot mutate transcript stages outside their range. +- Full runs attempt deterministic render before invoking Notarius. +- Forcing render no longer regenerates an otherwise valid Notarius bundle, and + forcing extraction no longer regenerates Markdown. +- Artifact-level validation adds filesystem hashing and fingerprint work before + some analyze skips, trading modest local inspection cost for fewer + Scriptorium invocations. +- The first analysis evaluation after upgrade may require regeneration because + legacy aggregate results do not contain sufficient freshness evidence. + +## Out Of Scope + +- Persistent transcript seals, transcript freeze state, or another lock type. +- Run-history listing, run-to-run artifact comparison, or draft promotion. +- Parallel execution of render and extract or of analysis artifacts. +- Non-contiguous stage selection. +- User-configurable stage order or invalidation dependencies. +- A generic DAG, phase, job, workflow, or manifest-subtask framework. +- New Seriatim, Notarius, or Scriptorium CLI capabilities. +- Changes to Notarius lane contracts or Scriptorium output schemas. +- Automatic observation of arbitrary transitive Scriptorium files or executable + contents. +- New render retry, caching, resumability, or fingerprint behavior. +- Automatic publish or notification as part of `regenerate-artifacts`. + +## Target End State + +Narratio has one comprehensible pipeline in which transcript production ends at +render and post-transcript generation begins at extract. Operators can run any +contiguous canonical range without accidentally executing stages outside it, +and force applies only within the requested range. + +The common development command: + +```bash +narratio regenerate-artifacts SESSION +``` + +is transparently identical to a forced bounded run from extract through +analyze. It preserves transcript state, regenerates the complete Notarius +bundle, rebuilds the requested Scriptorium targets, and leaves delivery as a +separate explicit action. + +Analysis artifacts have independent, manifest-authoritative freshness within +the fixed `analyze` stage. Narratio reuses valid unselected work, rebuilds stale +dependencies and selected targets deterministically, withholds stale outputs +from downstream consumers, and recognizes content-identical upstream results +without tying reuse to ephemeral run paths or IDs. + +Together, canonical ordering, dependency-aware invalidation, bounded execution, +the transparent alias, and artifact-level analysis reuse provide an ergonomic +development loop without turning Narratio into a general workflow engine.