narratio/docs/roadmap/post-transcript-artifact-workflow.md

Post-Transcript Artifact Development Workflow

Status

Accepted target state; not yet implemented.

Goal

Make repeated development of extraction and analysis artifacts fast, explicit, and safe after a session's transcripts are complete. Narratio should expose a clear transcript/post-transcript boundary, allow operators to run a bounded part of its canonical pipeline, provide one transparent convenience alias for the common forced-regeneration workflow, and reuse analysis artifacts whose observable inputs have not changed.

The feature must preserve Narratio's intentionally simple orchestration model: one fixed stage sequence, explicit stages, manifest-authoritative state, and no configurable workflow graph.

User Intent

Transcript production is comparatively infrequent after a session reaches a good final transcript. Development of Notarius extraction and Scriptorium artifacts continues much longer and commonly requires repeated execution.

The normal development workflow should therefore:

• treat completed transcript work as read-only unless the operator explicitly selects transcript stages;
• regenerate extraction and analysis without attempting earlier stages;
• make forced scope visible in the command itself;
• support focused Scriptorium artifact selection; and
• avoid rerunning unrelated analysis artifacts when their meaningful inputs and dependencies are unchanged.

Persistent transcript seals and run-to-run comparison tools are intentionally deferred. Bounded execution provides the immediate mutation boundary without adding another kind of durable lock.

Canonical Pipeline And Phase Boundary

Move render before extract so all transcript representations are complete before post-transcript artifact generation begins. The canonical order becomes:

1. prepare
2. transcribe
3. merge
4. polish
5. normalize
6. trim
7. render
8. extract
9. analyze
10. publish
11. notify

This creates two useful conceptual regions without making phases configurable:

• transcript production: prepare through render; and
• post-transcript processing and delivery: extract through notify.

render and extract are independent sibling consumers of completed transcript data. Render uses the canonical final and final-trimmed transcripts to create Markdown representations. Extract uses the canonical final-trimmed transcript and prepared references to create Notarius artifacts. Neither stage consumes the other's output.

Narratio's render path is deterministic. A render failure will stop a full run before extraction under the new order, and that behavior is accepted: a full run should complete transcript representations before starting post-transcript work. Recovery remains available through continuation and bounded or single-stage execution.

Execution Order And Invalidation

Execution order and invalidation dependencies must have separate explicit owners. The canonical stage registry defines when stages are attempted. A central, fixed, configuration-independent invalidation relation defines which recorded results may no longer be trustworthy after a stage outcome changes.

The relation is conservative across supported configurations. For example, analyze remains dependent on render because a configured artifact may consume rendered Markdown, even if one particular pipeline does not.

The required transitive invalidation sets, returned in canonical execution order, are:

Changed stage	Succeeded stages eligible to become stale
prepare	transcribe, merge, polish, normalize, trim, render, extract, analyze, publish, notify
transcribe	merge, polish, normalize, trim, render, extract, analyze, publish, notify
merge	polish, normalize, trim, render, extract, analyze, publish, notify
polish	normalize, trim, render, extract, analyze, publish, notify
normalize	trim, render, extract, analyze, publish, notify
trim	render, extract, analyze, publish, notify
render	analyze, publish, notify
extract	analyze, publish, notify
analyze	publish, notify
publish	notify
notify	none

In particular, render and extract must not invalidate one another. A change to either still invalidates analysis and delivery, while a change to trim invalidates both branches and their consumers.

The relation applies to every existing invalidation trigger, including forced replacement, a non-resumable success, failure, and a changed effective outcome. Only succeeded dependent stage records become stale under the existing status rules. Failed and incomplete records retain their meaning, and an identical repeated self-skip does not cause perpetual reruns.

The application owner must validate the fixed relation against the canonical stage inventory so a stage addition, removal, rename, duplication, or missing classification cannot silently produce incorrect invalidation behavior. This relation is not configurable and is not an alternate execution planner.

Bounded Canonical Execution

Extend run with inclusive canonical bounds:

narratio run <session_id> [--from <stage>] [--through <stage>] [--force]

Examples:

narratio run SESSION --from extract --through analyze --force
narratio run SESSION --from render --through render --force
narratio run SESSION --from analyze --through analyze

The bounds have these settled semantics:

• they select one contiguous slice of the fixed canonical stage sequence;
• --from defaults to the first stage and --through defaults to the last stage when omitted;
• both stage names must exist, and --from must not occur after --through;
• with neither option, run retains its current full-pipeline behavior;
• --force applies only to stages inside the selected range;
• stages before and after the range are not executed or resume-validated;
• excluded upstream records and artifacts may be resolved and validated as stage inputs, but Narratio must not repair or regenerate them implicitly;
• missing, stale, unsafe, or otherwise unusable prerequisites produce an actionable error rather than widening the requested range;
• invalidation caused by an executed stage may mark dependent stages outside the range stale, but those stages are not executed; and
• stage failure retains the existing stop-on-failure behavior.

session plan must accept the same bounds, force scope, and artifact selection needed to preview the corresponding run without executing stages. Plan and run must use one selection implementation so their range validation and run/skip decisions cannot drift.

--artifacts retains its existing meaning for analyze and publish when those stages are inside the selected range. Supplying artifact selection for a range containing neither consumer is an error rather than a silent no-op. Repeated artifact-selection flags retain their existing normalization and deduplication behavior.

Production composition should follow the bounded plan. Selecting a range must not require an adapter used only by an excluded stage, while a selected stage continues to require and validate its own collaborators.

regenerate-artifacts Convenience Alias

Add this top-level command:

narratio regenerate-artifacts <session_id> [--artifacts <name[,name...]>]

It is exactly a convenience alias for:

narratio run <session_id> --force --from extract --through analyze [--artifacts <name[,name...]>]

The alias has no independent orchestration semantics, prerequisites, force rules, or execution path. Its implementation must rewrite or construct the equivalent run arguments before invoking the shared run parser and handler. All common session/configuration arguments and repeatable --artifacts values pass through to run unchanged.

The shared parser owns validation, planning, execution, errors, and summaries. Alias help must state the exact equivalence. It is acceptable and desirable for runtime errors and summaries to identify the canonical run operation. The alias must not gain private flags or behavior; a future capability belongs on run first.

Because --force, --from, and --through define the alias, callers must not override them. The shared command parsing layer should reject duplicate singleton options rather than use ordering to choose a winner. That rule should apply consistently to bounded run itself, not only to the alias.

Without --artifacts, the alias force-runs extraction and all configured analysis artifacts. With --artifacts, extraction still produces its complete configured Notarius bundle, while forced analysis targets only the selected Scriptorium artifacts and any prerequisites required to build them. It never runs publish or notify. Changed results may correctly leave those later stages stale.

The existing analyze command remains the convenience path for forcing analysis without rerunning Notarius.

Incremental Analysis Artifacts

Analyze-Owned State

analyze currently has one aggregate stage result. Extend its manifest-owned state so each configured Scriptorium artifact has an explicit current result identity. Keep this model specific to analysis artifacts; do not introduce dynamic pipeline stages or a generic subtask framework without another proven consumer.

Each current artifact result must identify at least:

• the normalized configured artifact key;
• a versioned input fingerprint;
• the output source ID, contract, confined canonical path, checksum, and size;
• the producing Narratio run identity and useful non-secret Scriptorium provenance; and
• enough status or disposition information to distinguish current, stale, missing, failed, and intentionally unselected work.

The session manifest remains the authority for current availability. An incidental output file is not current merely because it exists. Invocation manifests continue to record what one run attempted and produced.

Artifact Fingerprints

Define one deterministic, versioned fingerprint per configured artifact using all Narratio-observable inputs that can change its result:

• its normalized Scriptorium artifact configuration;
• its ordered input names, source IDs, required/optional policy, and resolved input content identities;
• transcript, prepared-input, previous-session, extraction-lane, and other configured artifact contracts and content checksums;
• the current content identities of configured artifact dependencies;
• result-affecting Scriptorium adapter configuration visible to Narratio; and
• an explicit fingerprint contract version.

Fingerprint ordering must be deterministic. Identity must not change solely because a workspace moved, an absolute path changed, or an otherwise identical producer used a new run ID. In particular, a forced Notarius invocation that produces byte-identical lanes must not make unrelated analysis artifacts stale solely because the extraction run identity changed.

Narratio cannot observe arbitrary files, prompts, modules, executable contents, or transitive configuration loaded privately by Scriptorium. Documentation must state that changing an unobserved external input requires explicit force. Do not claim perfect content-addressed reuse beyond Narratio's declared inputs.

Freshness And Selection

Before skipping a succeeded analyze stage, an analyze-specific resume validator must reconcile the requested artifact set against current configuration, dependencies, input fingerprints, output records, confined regular files, and stored output checksums.

The execution rules are:

• an ordinary run executes only requested artifacts that are missing, stale, invalid, or no longer resumable;
• forcing analyze rebuilds all requested targets even when their fingerprints are current;
• --artifacts identifies explicit targets, not the complete set of records that may remain current;
• a selected target's configured prerequisites are processed in deterministic dependency order, reusing them when current and rebuilding them when stale;
• forcing a target does not force an otherwise current prerequisite unless it was also explicitly selected;
• valid unselected artifact records and outputs survive a partial rerun;
• artifacts removed or renamed in current configuration cease to be advertised as current;
• an artifact whose dependency or resolved input changes becomes stale unless the new semantic content identity is unchanged; and
• stale, missing, failed, or unverified artifacts are unavailable to downstream catalog and publish resolution even if an older file remains on disk.

If a rebuilt artifact changes, configured dependents that were not part of the invocation are not silently rebuilt. They become stale and will be rebuilt by a later run that selects them. If the rebuilt output is content-identical and the dependent fingerprint remains equal, the dependent may remain current.

A partial invocation succeeds when its explicit targets and required prerequisites succeed. The aggregate stage record may therefore describe a successful partial invocation while other configured artifacts are stale. The resume validator, not aggregate status alone, must ensure a later full run does not skip unresolved artifact work.

Replacement And Failure Safety

Artifact replacement must preserve unrelated current results while ensuring a failed target is not presented as freshly generated. Run-local output must be validated before canonical materialization and manifest promotion, consistent with existing stage safety policy.

On partial failure:

• successfully completed and validated targets may be recorded in the invocation history according to existing runner transaction boundaries;
• the failed target and any result whose current identity depends on it must not be advertised as current;
• unrelated previously validated artifacts must not be erased merely because they share the analyze stage; and
• publish and later stage state must remain conservatively stale or failed.

The implementation must define one clear manifest transition boundary and must not synthesize current output records from directory contents.

Legacy Analyze Results

Existing manifests may contain only an aggregate analyze success and outputs, without versioned per-artifact fingerprints. They remain readable, but Narratio must not invent trustworthy fingerprints for work whose exact inputs were not recorded.

On first incremental evaluation, legacy analysis artifacts are non-resumable. A full analysis selection rebuilds the effective configured set. A partial selection may rebuild its targets and prerequisites, but unselected legacy outputs remain stale and unavailable until regenerated. Old files and invocation manifests may remain for inspection under existing retention rules. No wholesale manifest rewrite or version-based transcript invalidation is required.

Resume And Existing Pipeline Manifests

The render/extract order change itself requires no manifest migration because stage records use stable names. Under the new sequence:

• succeeded render and extract records remain eligible for ordinary reuse and their stage-specific validation;
• stale, failed, interrupted, and absent records execute in the new order; and
• neither result is discarded merely because its relative position changed.

Bounded execution does not rewrite excluded stage records. Compatibility logic must remain name- and evidence-based; do not invalidate historical transcript work solely because it was produced by an earlier Narratio version.

The incremental-analysis model may add backward-compatible manifest fields or versioned metadata. Readers must accept older manifests, while new writers must emit one canonical representation and must not maintain parallel legacy and new analysis state indefinitely.

Required Test Coverage

Allocate tests to the behavior owner and avoid duplicating every case at every layer:

• planner and command tests prove the new canonical order, inclusive range selection, omitted-bound defaults, invalid/reversed bounds, duplicate-option rejection, artifact-range validation, and unchanged unbounded behavior;
• alias tests prove exact argument equivalence and pass-through without re-testing the runner through an independent path;
• invalidation tests prove the complete fixed relation, inventory validation, deterministic ordering, unknown-stage handling, and render/extract independence;
• runner tests prove force is confined to the selected range, excluded stages are not resume-validated or composed unnecessarily, outside dependents may be staled but not executed, and unusable prerequisites do not widen the range;
• continuation tests cover representative manifests produced under the old render/extract order;
• analyze tests prove fingerprint stability and sensitivity, dependency ordering, selected force, stale-prerequisite rebuilding, valid unselected preservation, changed-dependent invalidation, identical-content reuse, removed configuration, output tampering, partial failure, and legacy-result handling; and
• catalog and publish tests prove that only current per-artifact results are exposed downstream.

Tests must assert observable plans, manifest transitions, adapter requests, and artifact availability rather than private map layout. The default suite remains offline, deterministic, and independent of installed Seriatim, Notarius, or Scriptorium binaries.

Documentation And Policy Changes

When implementation lands, update all current-behavior owners in the same change:

• revise the architecture policy so canonical order owns execution, dependency-aware invalidation owns affected work, and bounded runs remain contiguous selections rather than arbitrary workflows;
• document the new order, range flags, exact alias expansion, force scope, plan behavior, incremental reuse, and legacy-analysis transition in the CLI and operations references;
• update the internal overview and the focused render, extract, analyze, manifest, artifact, adapter, and publish documents where their implemented contracts change;
• update troubleshooting guidance for incomplete bounded prerequisites, stale analysis artifacts, fingerprint limitations, and forced recovery; and
• remove every current claim that extract runs between trim and render or that all invalidation is a canonical-order suffix.

Documentation should continue to describe one explicit pipeline, not a DAG scheduler. Volatile command syntax and manifest details remain in their canonical owners rather than being copied across orientation documents.

Compatibility And Operational Effects

• Existing unbounded run, run-stage, analyze, and publish commands keep their current meanings except for the accepted render/extract order change and more precise analysis reuse.
• --from and --through are additive CLI options; configuration does not gain stage-order or range fields.
• regenerate-artifacts adds no semantics beyond its documented run alias.
• Bounded forced runs cannot mutate transcript stages outside their range.
• Full runs attempt deterministic render before invoking Notarius.
• Forcing render no longer regenerates an otherwise valid Notarius bundle, and forcing extraction no longer regenerates Markdown.
• Artifact-level validation adds filesystem hashing and fingerprint work before some analyze skips, trading modest local inspection cost for fewer Scriptorium invocations.
• The first analysis evaluation after upgrade may require regeneration because legacy aggregate results do not contain sufficient freshness evidence.

Out Of Scope

• Persistent transcript seals, transcript freeze state, or another lock type.
• Run-history listing, run-to-run artifact comparison, or draft promotion.
• Parallel execution of render and extract or of analysis artifacts.
• Non-contiguous stage selection.
• User-configurable stage order or invalidation dependencies.
• A generic DAG, phase, job, workflow, or manifest-subtask framework.
• New Seriatim, Notarius, or Scriptorium CLI capabilities.
• Changes to Notarius lane contracts or Scriptorium output schemas.
• Automatic observation of arbitrary transitive Scriptorium files or executable contents.
• New render retry, caching, resumability, or fingerprint behavior.
• Automatic publish or notification as part of regenerate-artifacts.

Target End State

Narratio has one comprehensible pipeline in which transcript production ends at render and post-transcript generation begins at extract. Operators can run any contiguous canonical range without accidentally executing stages outside it, and force applies only within the requested range.

The common development command:

narratio regenerate-artifacts SESSION

is transparently identical to a forced bounded run from extract through analyze. It preserves transcript state, regenerates the complete Notarius bundle, rebuilds the requested Scriptorium targets, and leaves delivery as a separate explicit action.

Analysis artifacts have independent, manifest-authoritative freshness within the fixed analyze stage. Narratio reuses valid unselected work, rebuilds stale dependencies and selected targets deterministically, withholds stale outputs from downstream consumers, and recognizes content-identical upstream results without tying reuse to ephemeral run paths or IDs.

Together, canonical ordering, dependency-aware invalidation, bounded execution, the transparent alias, and artifact-level analysis reuse provide an ergonomic development loop without turning Narratio into a general workflow engine.