narratio/docs/roadmap/runtime-artifacts.md

Roadmap: Runtime-Defined Scriptorium Artifacts

Status

Implementation roadmap for a pre-release hard cutover.

Purpose

Narratio currently treats artifact generation as a narrow analyze stage that supports a hard-coded session_recap artifact. This roadmap describes how to generalize artifact generation so operators can define Scriptorium-backed output artifacts at runtime through pipeline.yml.

The goal is to keep Narratio as a fixed pipeline orchestrator while making the artifact generation step configurable, composable, deterministic, and easy to regenerate selectively.

Desired Outcome

Operators should be able to define artifacts such as session recaps, player handouts, NPC summaries, quest logs, entity maps, or other campaign-specific outputs without changing Narratio code.

A configured artifact is declared under:

pipeline.scriptorium.artifacts.<name>

Each configured artifact becomes a canonical runtime artifact source ID:

narratio.artifact.<name>

For example:

scriptorium:
  artifacts:
    session_recap:
      enabled: true
      prompt_id: dnd_session.session_recap
      output_path: artifacts/session_recap.md
      inputs:
        transcript:
          source: narratio.transcript.trimmed
          required: true

This artifact is addressable by later artifacts as:

narratio.artifact.session_recap

A dependent artifact can then consume it explicitly:

scriptorium:
  artifacts:
    player_handout:
      enabled: true
      depends_on:
        - session_recap
      prompt_id: dnd_session.player_handout
      output_path: artifacts/player_handout.md
      inputs:
        recap:
          source: narratio.artifact.session_recap
          required: true
        transcript:
          source: narratio.transcript.trimmed
          required: true

Resolved Design Decisions

The following decisions are settled for the initial implementation:

1. Configured artifact outputs must live under Narratio's internal artifact output directory, initially artifacts/.
2. The artifact output directory should be defined as an internal default in internal/config/defaults.go, but no public configuration knob should be exposed yet.
3. Artifact output_path should remain explicit in the initial implementation to avoid guessing file extensions or output formats.
4. A disabled artifact may still be referenced as an input if its declared output already exists on disk and passes basic validation.
5. A disabled artifact is not executable during the current analyze run.
6. Artifact-to-artifact references require an explicit depends_on entry. Narratio should fail fast if the dependency declaration is missing.
7. The manifest remains stage-oriented: analyze succeeds or fails as a full stage.
8. Analyze-stage metadata may record per-artifact output details for provenance and later resolution, but not for intra-stage resume semantics.
9. --artifacts should be added as a CLI filter for selective artifact generation.
10. --artifacts does not imply --force; it only changes which configured artifacts are treated as executable when analyze actually runs.
11. Because Narratio is still pre-release, the hard-coded session_recap behavior should be removed immediately rather than deprecated gradually.

Scope

This roadmap covers:

• introducing a runtime artifact catalog;
• generalizing configured Scriptorium artifact execution;
• supporting narratio.artifact.<name> source IDs;
• adding explicit artifact dependencies;
• supporting disabled-but-resolvable artifact inputs;
• adding selective artifact execution via --artifacts;
• recording generated artifacts in analyze-stage metadata and/or manifest outputs;
• removing hard-coded session_recap behavior;
• updating tests and documentation.

Non-Goals

This feature should not turn Narratio into a general workflow engine.

The initial implementation should not add:

• arbitrary shell-command artifacts;
• arbitrary user-defined stages;
• loops or conditional branching;
• automatic archive promotion of generated artifacts;
• semantic knowledge of particular artifact types;
• per-artifact resume semantics within a successful or failed analyze stage;
• automatic dependency inference without depends_on.

Narratio should continue to orchestrate a fixed pipeline. The configurable part is the set of Scriptorium artifact invocations performed during the analyze stage.

Current State

Narratio already has several relevant pieces in place:

• pipeline.scriptorium.artifacts is modeled as a map of artifact definitions.
• The Scriptorium adapter already accepts generic run/render requests.
• The artifact resolver already understands canonical artifact source IDs.
• The analyze stage already resolves inputs, optionally runs render-debug, invokes Scriptorium, verifies output, and records metadata.

The main limitation is that analyze currently treats session_recap as the only executable artifact and rejects other enabled artifact definitions.

Target Architecture

Runtime Artifact Catalog

Introduce a per-run artifact catalog that tracks built-in artifacts and configured artifacts.

Conceptually:

ArtifactCatalog
├── built-in artifacts
│   ├── narratio.transcript.merged
│   ├── narratio.transcript.polished
│   ├── narratio.transcript.full
│   ├── narratio.transcript.trimmed
│   └── narratio.bounds.session
│
└── configured artifacts
    ├── narratio.artifact.session_recap
    ├── narratio.artifact.player_handout
    └── narratio.artifact.npc_summary

The catalog should distinguish between three states:

planned       valid configured or built-in artifact known to Narratio
available     artifact has been produced or otherwise resolved
executable    configured artifact selected for execution in this analyze run

Configured artifacts can be planned without being executable. This distinction is important for disabled artifacts and for --artifacts filtering.

Configured Artifact Source IDs

Configured artifact keys map directly to source IDs:

pipeline.scriptorium.artifacts.<name>
→ narratio.artifact.<name>

session_recap should no longer be a special built-in analyze artifact. Instead, it is just a conventional configured artifact key:

scriptorium:
  artifacts:
    session_recap:
      enabled: true
      prompt_id: dnd_session.session_recap
      output_path: artifacts/session_recap.md

narratio.artifact.session_recap remains valid only because session_recap is configured.

Artifact Output Directory

Add an internal default artifact output directory, initially:

artifacts

This default should live in internal/config/defaults.go or the existing equivalent defaults location.

For the initial implementation:

• expose no public config knob for the artifact output directory;
• require each configured artifact to provide an explicit output_path;
• validate that each configured artifact output_path is run-relative;
• validate that each configured artifact output_path is under the internal artifact output directory;
• reject output paths that escape the run workspace or use path traversal.

This preserves future configurability without forcing Narratio to guess output extensions or formats now.

Enabled, Disabled, and Selected Artifacts

Configured artifacts should have three distinct execution states:

enabled by config       artifact has enabled: true
selected for execution  artifact remains executable after --artifacts filtering
disabled for execution  artifact is not executable, but may be resolvable from disk

Without --artifacts, all configured artifacts with enabled: true are selected for execution.

With --artifacts, only the named artifacts are selected for execution. All other configured artifacts are treated as disabled for the current analyze invocation, regardless of their configured enabled value.

Disabled artifacts may still be resolved as inputs if their configured output_path exists on disk and passes validation.

Disabled Artifact Resolution

If artifact B references artifact A, and A is disabled for execution, Narratio should attempt to resolve A from disk.

This should succeed only when:

1. A is defined in pipeline.scriptorium.artifacts;
2. A has a valid output_path;
3. the output path exists in the current run workspace;
4. the output is non-empty, or otherwise passes any available artifact-specific validation.

The resolved provenance should make the source clear, for example:

filesystem.disabled_artifact_output

If the file does not exist or fails validation, the dependent artifact should fail before invoking Scriptorium.

Example error wording:

artifact player_handout requires narratio.artifact.session_recap, but session_recap is disabled for execution and artifacts/session_recap.md does not exist

Explicit Dependencies

Artifact-to-artifact references require explicit depends_on entries.

If artifact B has an input source of narratio.artifact.A, then B.depends_on must include A.

This should fail:

scriptorium:
  artifacts:
    player_handout:
      enabled: true
      prompt_id: dnd_session.player_handout
      output_path: artifacts/player_handout.md
      inputs:
        recap:
          source: narratio.artifact.session_recap
          required: true

This should pass:

scriptorium:
  artifacts:
    player_handout:
      enabled: true
      depends_on:
        - session_recap
      prompt_id: dnd_session.player_handout
      output_path: artifacts/player_handout.md
      inputs:
        recap:
          source: narratio.artifact.session_recap
          required: true

depends_on values refer to configured artifact keys, not full source IDs.

Dependency validation should fail on:

• references to unknown artifact keys;
• missing depends_on entries for artifact-to-artifact input references;
• self-dependencies;
• dependency cycles among executable artifacts.

Dependencies on disabled artifacts are permitted, but the disabled dependency must resolve from disk before the dependent artifact runs.

Execution Order

The analyze stage should execute selected artifacts in dependency order.

Rules:

• selected artifacts are executable;
• disabled artifacts are never executed;
• selected artifacts may depend on other selected artifacts;
• selected artifacts may depend on disabled artifacts if those disabled artifacts resolve from disk;
• independent selected artifacts run in deterministic sorted-name order.

Use topological sorting over selected artifacts, while validating dependency references across the full configured artifact set.

Input Resolution

Input resolution should use the artifact catalog and existing artifact resolver behavior.

For each configured artifact input:

• built-in sources resolve through existing resolver behavior;
• previous_session_artifact preserves existing behavior;
• narratio.artifact.<name> resolves through the runtime artifact catalog;
• selected dependencies resolve after being produced earlier in the same analyze execution;
• disabled dependencies resolve from their configured output path on disk;
• optional missing inputs are omitted;
• required missing inputs fail before Scriptorium is invoked.

Analyze Stage Generalization

The analyze stage should become the generic Scriptorium artifact stage.

High-level flow:

1. Load configured Scriptorium artifacts.
2. Apply the --artifacts filter, if present.
3. If no artifacts are selected for execution, return success metadata with skipped=true.
4. Build the runtime artifact catalog.
5. Validate artifact names, output paths, source IDs, dependencies, selected artifacts, and required fields.
6. Resolve any disabled dependencies that are required by selected artifacts.
7. Sort selected artifacts by dependency order.
8. For each selected artifact:
   • resolve configured inputs;
   • build the Scriptorium run request;
   • optionally run Scriptorium render-debug;
   • run Scriptorium;
   • fail on validation-failed result;
   • verify the output exists and is non-empty;
   • record artifact output metadata;
   • register narratio.artifact.<name> as available in the catalog.
9. Return aggregate analyze-stage metadata containing all generated and reused artifacts relevant to the run.

The Scriptorium adapter should remain generic. It should not decide which artifacts run, how dependencies work, or how artifacts are registered.

Manifest and Metadata

The manifest should remain stage-oriented.

This means:

• analyze succeeds or fails as a full stage;
• if analyze has already succeeded and the user does not force it, the runner skips it as a full stage;
• Narratio should not implement per-artifact resume in the first version.

However, analyze-stage metadata should still record artifact outputs for provenance and future resolution.

Recommended metadata shape:

{
  "skipped": false,
  "artifacts": [
    {
      "name": "session_recap",
      "source_id": "narratio.artifact.session_recap",
      "output_kind": "scriptorium_artifact",
      "path": "artifacts/session_recap.md",
      "prompt_id": "dnd_session.session_recap",
      "profile_id": "local-gemma-31b",
      "provenance": "generated.current_analyze_run"
    },
    {
      "name": "player_handout",
      "source_id": "narratio.artifact.player_handout",
      "output_kind": "scriptorium_artifact",
      "path": "artifacts/player_handout.md",
      "prompt_id": "dnd_session.player_handout",
      "profile_id": "local-gemma-31b",
      "provenance": "generated.current_analyze_run"
    }
  ],
  "reused_artifacts": [
    {
      "name": "session_recap",
      "source_id": "narratio.artifact.session_recap",
      "path": "artifacts/session_recap.md",
      "provenance": "filesystem.disabled_artifact_output"
    }
  ]
}

The exact struct can differ from this example, but it should preserve:

• artifact name;
• canonical source ID;
• output path;
• prompt/profile provenance for generated artifacts;
• reused-vs-generated provenance.

Resume and Force Behavior

Keep resume behavior stage-level.

Recommended semantics:

No --force, analyze already succeeded:
  runner skips analyze, regardless of --artifacts.

--force, no --artifacts:
  analyze regenerates all configured artifacts with enabled: true.

--force --artifacts player_handout:
  analyze treats only player_handout as executable.
  all other configured artifacts are disabled for execution.
  disabled dependencies may be reused from disk.

--artifacts player_handout on a not-yet-completed analyze stage:
  analyze runs only player_handout.
  disabled dependencies may be reused from disk.

--artifacts should not imply --force. It is an execution filter, not a resume override.

--artifacts CLI Flag

Add an --artifacts flag to commands that can execute or resume the analyze stage.

The flag should accept one or more configured artifact names. Internally, normalize values to a set of artifact keys.

Recommended behavior:

• validate all requested artifact names against pipeline.scriptorium.artifacts;
• reject unknown artifact names before running stages;
• treat requested artifacts as the only executable artifacts for the analyze stage;
• treat all other configured artifacts as disabled for execution;
• allow disabled artifacts to satisfy dependencies from disk as described above;
• if --artifacts is used while executing a stage other than analyze, either reject it or ignore it with a clear validation error. Prefer rejection.

The exact CLI parsing style can follow Narratio's existing conventions. Both comma-separated and repeatable values are acceptable if the CLI package supports them cleanly, but the internal representation should be a set of artifact keys.

Archive Behavior

Do not automatically archive every generated artifact.

Artifact generation and archive promotion should remain separate concerns. Operators should continue to use archive.promote_artifacts to decide which generated files should be promoted or uploaded.

Example:

archive:
  promote_artifacts:
    - from: artifacts/session_recap.md
      to: artifacts/session_recap.md
      required: true
    - from: artifacts/player_handout.md
      to: artifacts/player_handout.md
      required: false

A later enhancement may add opt-in automatic promotion of configured artifacts, but explicit promotion should remain the default.

Implementation Plan

Phase 1: Config Model and Defaults

Add or update the configured artifact model to include:

• enabled;
• depends_on;
• prompt_id;
• profile_id;
• output_path;
• timeout;
• render_debug;
• inputs;
• vars.

Add an internal default artifact output directory in internal/config/defaults.go, initially set to artifacts.

Validation rules:

• artifact names must match a conservative identifier pattern such as ^[a-z][a-z0-9_]*$;
• selected/executable artifacts require prompt_id and output_path;
• configured artifacts that may be referenced while disabled require output_path;
• configured artifact output paths must be run-relative;
• configured artifact output paths must live under the internal artifact output directory;
• configured artifact output paths must not escape the run workspace;
• narratio.artifact.<name> input sources must refer to configured artifact keys;
• any narratio.artifact.<name> input source must have a matching depends_on entry;
• depends_on entries must refer to configured artifact keys;
• dependencies must not contain self-references or executable cycles;
• input names and var names must remain compatible with the Scriptorium adapter's validation rules;
• unknown YAML fields must continue to fail strict decode.

Tests:

• valid single configured artifact;
• valid multiple independent artifacts;
• valid artifact-to-artifact dependency;
• valid dependency on disabled artifact with output path;
• invalid artifact name;
• missing required fields;
• output path outside artifacts/;
• dependency on missing artifact;
• missing depends_on for artifact input source;
• self-dependency;
• cycle detection;
• typo in narratio.artifact.<name> source;
• unknown YAML fields still fail strict decode.

Phase 2: CLI Filtering

Add the --artifacts flag and carry the selected artifact set into the run execution options.

Implementation notes:

• parse values according to existing CLI conventions;
• normalize to artifact key strings;
• validate against configured artifact definitions after config load;
• make the selected set available to the analyze stage;
• reject use with commands or stages where analyze cannot run.

Tests:

• no --artifacts means all enabled artifacts are selected;
• one requested artifact is selected;
• multiple requested artifacts are selected;
• unknown requested artifact fails;
• --artifacts does not imply --force;
• --artifacts with already-succeeded analyze stage is skipped unless forced;
• --artifacts on unsupported stage command fails clearly.

Phase 3: Runtime Artifact Catalog

Introduce an internal artifact catalog abstraction.

Responsibilities:

• register built-in artifact definitions;
• register configured artifact definitions;
• map configured artifact keys to narratio.artifact.<name> IDs;
• track planned, available, and executable artifact states;
• expose lookup by canonical source ID;
• record generated provenance;
• record disabled-from-disk provenance.

Keep the catalog narrow. It should not execute Scriptorium and should not understand prompt semantics.

Tests:

• built-in source lookup;
• configured source registration;
• duplicate/conflicting source handling;
• planned but unavailable artifact lookup;
• selected artifact state;
• disabled artifact state;
• registering an artifact as available after generation;
• registering a disabled artifact as available from disk;
• resolving a configured artifact from analyze metadata if that behavior is implemented.

Phase 4: Resolver Integration

Update artifact resolution so configured artifact IDs are resolved through the runtime catalog.

Resolution behavior:

• built-in sources continue using existing resolver behavior;
• configured artifact sources resolve from catalog availability/provenance;
• selected configured artifacts become available after generation;
• disabled configured artifacts may become available from disk;
• missing optional configured artifact inputs are omitted;
• missing required configured artifact inputs fail clearly.

Tests:

• configured artifact consumes a built-in transcript source;
• configured artifact consumes another configured artifact produced earlier in the same analyze run;
• configured artifact consumes a disabled artifact resolved from disk;
• required disabled artifact missing on disk fails;
• required configured artifact missing fails;
• optional missing configured artifact is omitted;
• reused artifact provenance is recorded distinctly from generated artifact provenance.

Phase 5: Analyze Stage Generalization

Refactor analyze to execute selected configured artifacts.

Implementation notes:

• remove the hard-coded session_recap selection path;
• remove the hard-coded rejection of non-session_recap artifacts;
• preserve skip behavior when Scriptorium config is absent or no artifacts are selected;
• build the runtime artifact catalog;
• apply --artifacts filtering;
• validate selected artifacts and their dependencies;
• pre-resolve disabled dependencies from disk where required;
• compute deterministic dependency order;
• execute selected artifacts one at a time in dependency order;
• keep render-debug behavior at global and artifact levels;
• keep Scriptorium adapter invocation generic;
• after each successful run, register the artifact as available in the catalog;
• aggregate generated and reused artifact metadata.

Tests:

• no Scriptorium config skips;
• empty artifact map skips;
• no selected artifacts skips;
• disabled artifacts do not run;
• one selected artifact runs;
• multiple independent artifacts run in deterministic order;
• dependent selected artifact receives prior selected artifact as input;
• dependent selected artifact receives disabled-from-disk artifact as input;
• render-debug works for configured artifacts;
• Scriptorium validation failure fails the stage;
• missing required input fails the stage;
• successful outputs are non-empty and recorded;
• artifact filter executes only requested artifacts.

Phase 6: Manifest and Stage Metadata

Update analyze-stage metadata and manifest output recording to support dynamic configured artifacts.

Recommended behavior:

• every generated configured artifact gets source_id: narratio.artifact.<name>;
• every generated configured artifact gets a generic output kind such as scriptorium_artifact;
• reused disabled artifacts are recorded separately from generated artifacts;
• metadata is sufficient for debugging, provenance, and future resolver support;
• metadata does not create per-artifact resume semantics.

Because this is a pre-release hard cutover, do not preserve a special legacy session_recap output kind unless a current internal test or archive path still requires it temporarily. Prefer updating tests and examples to treat session_recap as an ordinary configured artifact.

Tests:

• metadata records one generated configured artifact;
• metadata records multiple generated configured artifacts;
• metadata records reused disabled artifact provenance;
• session_recap is recorded as a normal configured artifact;
• manifest still treats analyze as a single succeeded or failed stage;
• runner skip behavior remains stage-level.

Phase 7: Archive and Promotion Review

Review archive behavior after dynamic artifacts are recorded.

Implementation notes:

• do not automatically promote every configured artifact;
• keep archive.promote_artifacts explicit;
• update default or example promotion rules to use configured session_recap output path;
• ensure required promotion rules fail clearly when selected artifact generation did not produce a required file.

Tests:

• generated artifact can be promoted by explicit archive rule;
• required archive promotion fails if selected artifact was not generated and no file exists;
• optional archive promotion skips cleanly if file is absent;
• hard cutover does not rely on hard-coded session_recap generation.

Phase 8: Documentation and Examples

Update documentation after the implementation is complete.

Recommended documentation changes:

• update docs/config.md with the generalized artifact configuration model;
• update docs/internal/artifacts.md to describe the runtime artifact catalog;
• update docs/stages/analyze.md to describe generic Scriptorium artifact generation;
• update Scriptorium integration docs only if the adapter contract changes;
• update full annotated pipeline examples;
• add at least one example with multiple artifacts and one dependency;
• document --artifacts behavior and its relationship to --force;
• remove documentation stating that only session_recap is supported.

Documentation should make clear that:

• configured artifact source IDs use narratio.artifact.<name>;
• depends_on uses artifact keys, not full source IDs;
• artifact-to-artifact source references require explicit depends_on;
• disabled artifacts can be reused from disk when required by selected artifacts;
• --artifacts filters execution but does not imply --force;
• archive promotion remains explicit;
• per-artifact resume is not part of the initial implementation.

Migration Strategy

Because Narratio is pre-release, perform a hard cutover.

Required changes:

1. Remove the hard-coded session_recap analyze behavior.
2. Require session_recap to be declared under pipeline.scriptorium.artifacts.session_recap if the operator wants a session recap.
3. Treat narratio.artifact.session_recap as valid only when session_recap is a configured artifact key.
4. Update config examples to show session_recap as a normal configured artifact.
5. Update tests to stop assuming that session_recap is a built-in analyze artifact.
6. Keep archive promotion explicit and path-based.

Example replacement config:

scriptorium:
  binary: scriptorium
  config_path: /etc/scriptorium/config.yml
  timeout: 10m
  render_debug: false
  artifacts:
    session_recap:
      enabled: true
      prompt_id: dnd_session.session_recap
      profile_id: local-gemma-31b
      output_path: artifacts/session_recap.md
      timeout: 20m
      inputs:
        transcript:
          source: narratio.transcript.trimmed
          required: true
        prior_recap:
          source: previous_session_artifact
          artifact: artifacts/session_recap.md
          required: false
      vars:
        artifact_title: Session Recap

Acceptance Criteria

The feature is complete when:

• operators can define more than one enabled Scriptorium artifact in pipeline.yml;
• Narratio runs selected artifacts in deterministic dependency order;
• configured artifacts are addressable as narratio.artifact.<name>;
• one configured artifact can consume another configured artifact as an input;
• artifact-to-artifact input references require explicit depends_on;
• disabled artifacts can satisfy dependencies from existing on-disk outputs;
• missing required disabled artifacts fail clearly;
• optional missing inputs are omitted;
• --artifacts can selectively execute valid configured artifact names;
• --artifacts does not imply --force;
• render-debug behavior works for all configured artifacts;
• generated and reused artifacts are recorded in analyze-stage metadata;
• session_recap is no longer hard-coded and works as a normal configured artifact;
• archive promotion remains explicit;
• tests cover config validation, dependency sorting, disabled artifact resolution, resolver behavior, CLI filtering, analyze execution, archive interactions, and metadata.

Suggested Implementation Order

1. Config model, defaults, and validation.
2. CLI parsing and propagation of --artifacts selection.
3. Runtime artifact catalog.
4. Resolver integration for configured artifacts.
5. Analyze stage generalization.
6. Stage metadata and manifest output recording.
7. Archive behavior review.
8. Documentation and examples.

This order keeps the most static pieces first, then moves into execution behavior once the configuration contract is explicit and well tested.