narratio/docs/internal/stage-analyze.md

Stage: analyze

Purpose

Reconcile configured Scriptorium artifacts, execute only required work in dependency order, and safely materialize validated outputs.

Inputs

• ordinary configured artifacts from pipeline.scriptorium.artifacts; canonical party artifact families have already expanded into this map during configuration resolution
• optional selected artifact keys supplied through the stage environment
• built-in, configured, extraction, and previous-session source references in artifact inputs

Supported source families:

• built-ins: narratio.transcript.*, narratio.bounds.session
• prepared stable inputs: narratio.input.players, narratio.input.party, narratio.input.glossary, narratio.input.spell_catalog
• configured artifacts: narratio.artifact.<key>
• extraction lanes: narratio.extraction.<key>
• previous-session cache: narratio.previous_session.artifact.<key>

Outputs

• one current per-artifact manifest record per validated materialized output
• stage metadata describing selected/generated/reused artifacts

Key Behavior

• when pipeline.scriptorium is absent or no configured artifact is executable, completes successfully with no outputs and records explanatory metadata. This is not an explicit self-skip: both manifests record success, satisfy publish's prerequisite, and an ordinary later run reuses the result while the effective set remains empty. Enabling or selecting an artifact later makes missing versioned evidence non-resumable and schedules it without requiring force.
• builds a runtime artifact catalog containing built-ins, configured artifacts, and configured extraction lanes. Extraction availability is hydrated only from compatible successful extraction evidence.
• uses enabled configured artifacts by default. An explicit --artifacts selection is a one-invocation override that makes exactly the named configured artifacts explicit targets even when disabled. The work planner adds required configured prerequisites, reuses current ones, and schedules stale, missing, or otherwise non-current prerequisites before dependents.
• makes a non-executable configured artifact reusable only when its current manifest record and durable output pass the configured-artifact evidence contract; an incidental or stale canonical file is unavailable.
• validates selected artifact dependency order (cycle-safe topo ordering).
• resolves required/optional inputs per artifact source definition into an ordered semantic identity. Each identity records the configured input name, canonical source ID, required policy, explicit presence, source contract, checksum, size, and a source-based logical identity. Workspace paths and producer run IDs are excluded.
• orders input identities by configured input name independently of Go map iteration. Runtime adapter paths remain a separate execution-only map.
• omits an unavailable optional input from the adapter request while retaining explicit absence in its semantic identity; an unavailable required input fails.
• resolves prepared stable input sources through the shared manifest-authoritative identity resolver; it does not accept incidental files or fall back to campaign/session source paths.
• reuses checksums and sizes from validated prepared, extraction, and current configured-artifact evidence. Other resolved inputs are hashed as confined regular files with streaming reads and the central resolved-artifact size limit.
• owns a versioned SHA-256 fingerprint contract with one fixed-field canonical JSON payload and no map serialization. Configured artifacts are fingerprinted in deterministic dependency order.
• fingerprints the normalized artifact key, prompt and profile identifiers, effective render-debug behavior, session-relative output identity, sorted dependency keys, ordered input declarations and semantic identities, validated current dependency-output identities, and sorted effective Scriptorium variables (including Narratio's sticky session variable).
• provides read-only reconciliation that classifies each configured record as current, stale, missing, failed, legacy, or otherwise non-resumable, and separately identifies manifest records removed from current configuration. A record is current only when its fingerprint version and value match and its configured output still passes manifest-authoritative evidence validation.
• owns a read-only typed work planner. Its explicit targets are enabled artifacts by default or the exact normalized --artifacts selection when supplied. It closes targets over configured prerequisites, orders the closure topologically, reuses current members, and schedules every non-current member before its dependents.
• force applies only to explicit targets. A current prerequisite is reused unless it is itself an explicit forced target; disabled prerequisites may be rebuilt when required, while unrelated disabled artifacts are excluded.
• the work plan carries explicit targets, prerequisite-only work, deterministic execution and reuse lists, invalidated and removed records, and a cloned projected record collection. Valid unrelated configured records survive the projection, removed records are omitted, and legacy files never become current without regeneration.
• implements aggregate resume validation by running the same read-only catalog, fingerprint reconciliation, and work planner used by execution. A succeeded aggregate record is reusable exactly when the selected closure schedules no artifact work; stale unrelated records do not block a partial selection.
• exposes the typed artifact decision to session plan. Planning applies it to a cloned manifest after modeling earlier selected stage transitions, so aggregate run/skip and artifact execute/reuse decisions match the ordinary runner without creating durable state or invoking Scriptorium.
• executes only the work plan's scheduled entries. Manifest-validated current prerequisites remain available through the runtime catalog without invoking Scriptorium; newly produced prerequisites enter that catalog with the same contract, checksum, and size identity used for persisted current evidence.
• keeps adapter output in the invocation's run-local analyze directory until it is a safe, non-empty, bounded regular file with a calculated checksum and complete output contract. Canonical replacement uses the shared atomic file operation boundary and verifies that the installed checksum matches the validated run-local bytes.
• records each successful artifact's freshly computed fingerprint, canonical relative output path, contract, checksum, size, producer run ID, bounded Scriptorium provenance, logs, and generated configuration references in the analyze-owned projection.
• preserves valid unrelated current records during partial execution. If a rebuilt output's bytes and contract are unchanged, unselected dependents may remain current. If that semantic identity changes, unselected transitive dependents become stale without being executed; dependents included in the invocation are evaluated in dependency order instead.
• reports all evaluated targets and prerequisites in invocation state. The runner reconstructs aggregate session outputs from every current session record and invocation outputs from only records produced by the current run. Unrelated stale records do not make an otherwise successful partial invocation fail.
• resolves previous-session sources from local previous/ cache only.
• runs optional render-debug, then artifact execution.
• validates non-empty output files and materializes canonical outputs.

Failure Semantics

• required missing configured/previous-session inputs fail.
• missing required prepared stable input source includes prepare rerun guidance.
• missing required previous-session source includes prepare rerun guidance.
• missing required narratio.transcript.final_markdown or narratio.transcript.final_trimmed_markdown inputs includes render rerun guidance.
• dependency cycles or unavailable required dependencies fail.
• adapter validation failures fail stage.
• a scheduled artifact failure returns the restricted analyze-state projection with the active artifact marked failed, a bounded error, and no output authority. Current transitive dependents become stale without execution.
• earlier artifacts from the invocation remain current only after their run-local output passed validation and canonical materialization. They remain in invocation history; unattempted later artifacts do not appear there.
• unrelated current records survive a partial failure. Old canonical bytes for the failed artifact and newly materialized bytes whose projection cannot be persisted are incidental, not current evidence.
• the runner persists a valid partial projection before it marks aggregate analyze failed and invalidates publish and notify through the application dependency relation. Projection-persistence errors retain the last durable per-artifact authority and are joined with the original failure context.

Invariants

• analyze performs no remote storage calls for previous-session source resolution.
• input-identity resolution is read-only: it does not invoke adapters, materialize outputs, update status, or create run records.
• fingerprints exclude timeouts, retries, timestamps, producer and Narratio run IDs, executable and config paths, workspace roots, diagnostic locations, and executable or private transitive configuration contents. A change that is visible only inside Scriptorium—such as a file privately loaded by its config path—requires an explicit forced regeneration.
• output provenance and metadata are deterministic per execution.
• a canonical file without current per-artifact manifest evidence is never promoted to current state.

Related Contracts And Tests

• Configuration owns artifact fields and source-selection rules, including artifact families.
• CLI owns user-visible artifact selection.
• Scriptorium owns the subprocess contract.
• Implementation and tests: internal/stage/analyze.go, internal/stage/analyze_input_identity.go, internal/stage/analyze_test.go, internal/stage/analyze_input_identity_test.go, internal/stage/analyze_fingerprint.go, internal/stage/analyze_fingerprint_test.go, internal/stage/analyze_reconciliation.go, and internal/stage/analyze_reconciliation_test.go, internal/stage/analyze_plan.go, internal/stage/analyze_plan_test.go, and internal/stage/analyze_incremental_execution_test.go, and internal/stage/analyze_failure_test.go, internal/stage/analyze_resume.go, and internal/stage/analyze_resume_test.go