Updated the roadmap for configurable artifact generation
This commit is contained in:
@@ -2,33 +2,31 @@
|
||||
|
||||
## Status
|
||||
|
||||
Proposed implementation roadmap.
|
||||
Implementation roadmap for a pre-release hard cutover.
|
||||
|
||||
## Purpose
|
||||
|
||||
Narratio currently treats artifact generation as a narrow `analyze` stage that supports a hard-coded `session_recap` artifact. This roadmap describes how to generalize artifact generation so operators can define multiple Scriptorium-backed output artifacts at runtime through `pipeline.yml`.
|
||||
Narratio currently treats artifact generation as a narrow `analyze` stage that supports a hard-coded `session_recap` artifact. This roadmap describes how to generalize artifact generation so operators can define Scriptorium-backed output artifacts at runtime through `pipeline.yml`.
|
||||
|
||||
The goal is to let Narratio continue acting as an orchestrator while making session artifacts configurable, composable, resumable, and visible through a unified artifact model.
|
||||
The goal is to keep Narratio as a fixed pipeline orchestrator while making the artifact generation step configurable, composable, deterministic, and easy to regenerate selectively.
|
||||
|
||||
## Desired Outcome
|
||||
|
||||
Operators should be able to define artifacts such as session recaps, player handouts, NPC summaries, quest logs, entity maps, or other campaign-specific outputs without changing Narratio code.
|
||||
|
||||
A configured artifact should be declared under `pipeline.scriptorium.artifacts.<name>` and should define, at minimum:
|
||||
|
||||
- whether it is enabled;
|
||||
- which Scriptorium prompt to run;
|
||||
- where the output should be written;
|
||||
- which Narratio artifacts should be passed as Scriptorium inputs;
|
||||
- which static vars should be passed to Scriptorium.
|
||||
|
||||
Configured artifacts should become canonical runtime artifact IDs using this form:
|
||||
A configured artifact is declared under:
|
||||
|
||||
```text
|
||||
narratio.artifact.<artifact_name>
|
||||
pipeline.scriptorium.artifacts.<name>
|
||||
```
|
||||
|
||||
For example, an artifact declared as:
|
||||
Each configured artifact becomes a canonical runtime artifact source ID:
|
||||
|
||||
```text
|
||||
narratio.artifact.<name>
|
||||
```
|
||||
|
||||
For example:
|
||||
|
||||
```yaml
|
||||
scriptorium:
|
||||
@@ -37,15 +35,19 @@ scriptorium:
|
||||
enabled: true
|
||||
prompt_id: dnd_session.session_recap
|
||||
output_path: artifacts/session_recap.md
|
||||
inputs:
|
||||
transcript:
|
||||
source: narratio.transcript.trimmed
|
||||
required: true
|
||||
```
|
||||
|
||||
should be registered as:
|
||||
This artifact is addressable by later artifacts as:
|
||||
|
||||
```text
|
||||
narratio.artifact.session_recap
|
||||
```
|
||||
|
||||
Other configured artifacts should then be able to use it as an input:
|
||||
A dependent artifact can then consume it explicitly:
|
||||
|
||||
```yaml
|
||||
scriptorium:
|
||||
@@ -60,8 +62,27 @@ scriptorium:
|
||||
recap:
|
||||
source: narratio.artifact.session_recap
|
||||
required: true
|
||||
transcript:
|
||||
source: narratio.transcript.trimmed
|
||||
required: true
|
||||
```
|
||||
|
||||
## Resolved Design Decisions
|
||||
|
||||
The following decisions are settled for the initial implementation:
|
||||
|
||||
1. Configured artifact outputs must live under Narratio's internal artifact output directory, initially `artifacts/`.
|
||||
2. The artifact output directory should be defined as an internal default in `internal/config/defaults.go`, but no public configuration knob should be exposed yet.
|
||||
3. Artifact `output_path` should remain explicit in the initial implementation to avoid guessing file extensions or output formats.
|
||||
4. A disabled artifact may still be referenced as an input if its declared output already exists on disk and passes basic validation.
|
||||
5. A disabled artifact is not executable during the current analyze run.
|
||||
6. Artifact-to-artifact references require an explicit `depends_on` entry. Narratio should fail fast if the dependency declaration is missing.
|
||||
7. The manifest remains stage-oriented: `analyze` succeeds or fails as a full stage.
|
||||
8. Analyze-stage metadata may record per-artifact output details for provenance and later resolution, but not for intra-stage resume semantics.
|
||||
9. `--artifacts` should be added as a CLI filter for selective artifact generation.
|
||||
10. `--artifacts` does not imply `--force`; it only changes which configured artifacts are treated as executable when `analyze` actually runs.
|
||||
11. Because Narratio is still pre-release, the hard-coded `session_recap` behavior should be removed immediately rather than deprecated gradually.
|
||||
|
||||
## Scope
|
||||
|
||||
This roadmap covers:
|
||||
@@ -70,22 +91,25 @@ This roadmap covers:
|
||||
- generalizing configured Scriptorium artifact execution;
|
||||
- supporting `narratio.artifact.<name>` source IDs;
|
||||
- adding explicit artifact dependencies;
|
||||
- recording dynamic artifacts in stage metadata and the session manifest;
|
||||
- preserving compatibility for the existing `session_recap` behavior;
|
||||
- supporting disabled-but-resolvable artifact inputs;
|
||||
- adding selective artifact execution via `--artifacts`;
|
||||
- recording generated artifacts in analyze-stage metadata and/or manifest outputs;
|
||||
- removing hard-coded `session_recap` behavior;
|
||||
- updating tests and documentation.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
This roadmap does not attempt to turn Narratio into a general workflow engine.
|
||||
This feature should not turn Narratio into a general workflow engine.
|
||||
|
||||
Specifically, this feature should not add:
|
||||
The initial implementation should not add:
|
||||
|
||||
- arbitrary shell-command artifacts;
|
||||
- multi-stage user-defined workflows;
|
||||
- conditional branching;
|
||||
- loops;
|
||||
- remote artifact discovery beyond existing archive/session behavior;
|
||||
- semantic understanding of each configured artifact type.
|
||||
- arbitrary user-defined stages;
|
||||
- loops or conditional branching;
|
||||
- automatic archive promotion of generated artifacts;
|
||||
- semantic knowledge of particular artifact types;
|
||||
- per-artifact resume semantics within a successful or failed analyze stage;
|
||||
- automatic dependency inference without `depends_on`.
|
||||
|
||||
Narratio should continue to orchestrate a fixed pipeline. The configurable part is the set of Scriptorium artifact invocations performed during the `analyze` stage.
|
||||
|
||||
@@ -104,13 +128,7 @@ The main limitation is that `analyze` currently treats `session_recap` as the on
|
||||
|
||||
### Runtime Artifact Catalog
|
||||
|
||||
Introduce a per-run artifact catalog that tracks both built-in and configured artifacts.
|
||||
|
||||
The catalog should include:
|
||||
|
||||
1. Built-in artifacts produced by fixed pipeline stages.
|
||||
2. Configured Scriptorium artifacts declared under `pipeline.scriptorium.artifacts`.
|
||||
3. Availability/provenance state for artifacts that have been produced or resolved from the manifest.
|
||||
Introduce a per-run artifact catalog that tracks built-in artifacts and configured artifacts.
|
||||
|
||||
Conceptually:
|
||||
|
||||
@@ -129,34 +147,26 @@ ArtifactCatalog
|
||||
└── narratio.artifact.npc_summary
|
||||
```
|
||||
|
||||
The catalog should distinguish between planned and available artifacts:
|
||||
The catalog should distinguish between three states:
|
||||
|
||||
- A planned artifact is validly declared and may be produced during the current run.
|
||||
- An available artifact has been produced successfully in the current run or resolved from prior successful manifest state.
|
||||
```text
|
||||
planned valid configured or built-in artifact known to Narratio
|
||||
available artifact has been produced or otherwise resolved
|
||||
executable configured artifact selected for execution in this analyze run
|
||||
```
|
||||
|
||||
### Artifact Source IDs
|
||||
Configured artifacts can be planned without being executable. This distinction is important for disabled artifacts and for `--artifacts` filtering.
|
||||
|
||||
Configured artifact keys should map directly to canonical source IDs:
|
||||
### Configured Artifact Source IDs
|
||||
|
||||
Configured artifact keys map directly to source IDs:
|
||||
|
||||
```text
|
||||
pipeline.scriptorium.artifacts.<name>
|
||||
→ narratio.artifact.<name>
|
||||
```
|
||||
|
||||
Example:
|
||||
|
||||
```text
|
||||
pipeline.scriptorium.artifacts.session_recap
|
||||
→ narratio.artifact.session_recap
|
||||
```
|
||||
|
||||
The existing hard-coded `narratio.artifact.session_recap` source should become a normal configured-artifact source, while retaining compatibility behavior where needed.
|
||||
|
||||
### Configured Artifact Dependencies
|
||||
|
||||
Add optional `depends_on` support to configured artifacts.
|
||||
|
||||
Example:
|
||||
`session_recap` should no longer be a special built-in analyze artifact. Instead, it is just a conventional configured artifact key:
|
||||
|
||||
```yaml
|
||||
scriptorium:
|
||||
@@ -165,11 +175,97 @@ scriptorium:
|
||||
enabled: true
|
||||
prompt_id: dnd_session.session_recap
|
||||
output_path: artifacts/session_recap.md
|
||||
inputs:
|
||||
transcript:
|
||||
source: narratio.transcript.trimmed
|
||||
required: true
|
||||
```
|
||||
|
||||
`narratio.artifact.session_recap` remains valid only because `session_recap` is configured.
|
||||
|
||||
### Artifact Output Directory
|
||||
|
||||
Add an internal default artifact output directory, initially:
|
||||
|
||||
```text
|
||||
artifacts
|
||||
```
|
||||
|
||||
This default should live in `internal/config/defaults.go` or the existing equivalent defaults location.
|
||||
|
||||
For the initial implementation:
|
||||
|
||||
- expose no public config knob for the artifact output directory;
|
||||
- require each configured artifact to provide an explicit `output_path`;
|
||||
- validate that each configured artifact `output_path` is run-relative;
|
||||
- validate that each configured artifact `output_path` is under the internal artifact output directory;
|
||||
- reject output paths that escape the run workspace or use path traversal.
|
||||
|
||||
This preserves future configurability without forcing Narratio to guess output extensions or formats now.
|
||||
|
||||
### Enabled, Disabled, and Selected Artifacts
|
||||
|
||||
Configured artifacts should have three distinct execution states:
|
||||
|
||||
```text
|
||||
enabled by config artifact has enabled: true
|
||||
selected for execution artifact remains executable after --artifacts filtering
|
||||
disabled for execution artifact is not executable, but may be resolvable from disk
|
||||
```
|
||||
|
||||
Without `--artifacts`, all configured artifacts with `enabled: true` are selected for execution.
|
||||
|
||||
With `--artifacts`, only the named artifacts are selected for execution. All other configured artifacts are treated as disabled for the current analyze invocation, regardless of their configured `enabled` value.
|
||||
|
||||
Disabled artifacts may still be resolved as inputs if their configured `output_path` exists on disk and passes validation.
|
||||
|
||||
### Disabled Artifact Resolution
|
||||
|
||||
If artifact `B` references artifact `A`, and `A` is disabled for execution, Narratio should attempt to resolve `A` from disk.
|
||||
|
||||
This should succeed only when:
|
||||
|
||||
1. `A` is defined in `pipeline.scriptorium.artifacts`;
|
||||
2. `A` has a valid `output_path`;
|
||||
3. the output path exists in the current run workspace;
|
||||
4. the output is non-empty, or otherwise passes any available artifact-specific validation.
|
||||
|
||||
The resolved provenance should make the source clear, for example:
|
||||
|
||||
```text
|
||||
filesystem.disabled_artifact_output
|
||||
```
|
||||
|
||||
If the file does not exist or fails validation, the dependent artifact should fail before invoking Scriptorium.
|
||||
|
||||
Example error wording:
|
||||
|
||||
```text
|
||||
artifact player_handout requires narratio.artifact.session_recap, but session_recap is disabled for execution and artifacts/session_recap.md does not exist
|
||||
```
|
||||
|
||||
### Explicit Dependencies
|
||||
|
||||
Artifact-to-artifact references require explicit `depends_on` entries.
|
||||
|
||||
If artifact `B` has an input source of `narratio.artifact.A`, then `B.depends_on` must include `A`.
|
||||
|
||||
This should fail:
|
||||
|
||||
```yaml
|
||||
scriptorium:
|
||||
artifacts:
|
||||
player_handout:
|
||||
enabled: true
|
||||
prompt_id: dnd_session.player_handout
|
||||
output_path: artifacts/player_handout.md
|
||||
inputs:
|
||||
recap:
|
||||
source: narratio.artifact.session_recap
|
||||
required: true
|
||||
```
|
||||
|
||||
This should pass:
|
||||
|
||||
```yaml
|
||||
scriptorium:
|
||||
artifacts:
|
||||
player_handout:
|
||||
enabled: true
|
||||
depends_on:
|
||||
@@ -182,15 +278,30 @@ scriptorium:
|
||||
required: true
|
||||
```
|
||||
|
||||
`depends_on` values should refer to configured artifact keys, not full source IDs.
|
||||
`depends_on` values refer to configured artifact keys, not full source IDs.
|
||||
|
||||
Use topological sorting to determine execution order. Fail validation on:
|
||||
Dependency validation should fail on:
|
||||
|
||||
- dependency references to missing or disabled artifacts;
|
||||
- references to unknown artifact keys;
|
||||
- missing `depends_on` entries for artifact-to-artifact input references;
|
||||
- self-dependencies;
|
||||
- dependency cycles.
|
||||
- dependency cycles among executable artifacts.
|
||||
|
||||
If two artifacts are independent, execute them in deterministic sorted-name order.
|
||||
Dependencies on disabled artifacts are permitted, but the disabled dependency must resolve from disk before the dependent artifact runs.
|
||||
|
||||
### Execution Order
|
||||
|
||||
The analyze stage should execute selected artifacts in dependency order.
|
||||
|
||||
Rules:
|
||||
|
||||
- selected artifacts are executable;
|
||||
- disabled artifacts are never executed;
|
||||
- selected artifacts may depend on other selected artifacts;
|
||||
- selected artifacts may depend on disabled artifacts if those disabled artifacts resolve from disk;
|
||||
- independent selected artifacts run in deterministic sorted-name order.
|
||||
|
||||
Use topological sorting over selected artifacts, while validating dependency references across the full configured artifact set.
|
||||
|
||||
### Input Resolution
|
||||
|
||||
@@ -198,48 +309,51 @@ Input resolution should use the artifact catalog and existing artifact resolver
|
||||
|
||||
For each configured artifact input:
|
||||
|
||||
- built-in sources should resolve through the existing resolver;
|
||||
- `previous_session_artifact` should preserve existing behavior;
|
||||
- `narratio.artifact.<name>` should resolve only if the named configured artifact is available;
|
||||
- optional missing inputs should be omitted;
|
||||
- required missing inputs should fail the artifact run.
|
||||
|
||||
Runtime-configured artifacts should not be considered available merely because their output path exists on disk. They should be available only when:
|
||||
|
||||
1. they were produced successfully earlier in the current `analyze` execution; or
|
||||
2. they are recorded as successful outputs in prior manifest state being used for resume; or
|
||||
3. Narratio intentionally supports a documented canonical fallback for that artifact.
|
||||
|
||||
For the initial implementation, prefer options 1 and 2 only.
|
||||
- built-in sources resolve through existing resolver behavior;
|
||||
- `previous_session_artifact` preserves existing behavior;
|
||||
- `narratio.artifact.<name>` resolves through the runtime artifact catalog;
|
||||
- selected dependencies resolve after being produced earlier in the same analyze execution;
|
||||
- disabled dependencies resolve from their configured output path on disk;
|
||||
- optional missing inputs are omitted;
|
||||
- required missing inputs fail before Scriptorium is invoked.
|
||||
|
||||
### Analyze Stage Generalization
|
||||
|
||||
The `analyze` stage should become the generic Scriptorium artifact stage.
|
||||
|
||||
Its high-level flow should be:
|
||||
High-level flow:
|
||||
|
||||
1. Load configured Scriptorium artifacts.
|
||||
2. Filter to enabled artifacts.
|
||||
3. If no artifacts are enabled, return success metadata with `skipped=true`.
|
||||
2. Apply the `--artifacts` filter, if present.
|
||||
3. If no artifacts are selected for execution, return success metadata with `skipped=true`.
|
||||
4. Build the runtime artifact catalog.
|
||||
5. Validate configured artifact names, source IDs, paths, dependencies, and required fields.
|
||||
6. Sort enabled artifacts by dependency order.
|
||||
7. For each artifact:
|
||||
5. Validate artifact names, output paths, source IDs, dependencies, selected artifacts, and required fields.
|
||||
6. Resolve any disabled dependencies that are required by selected artifacts.
|
||||
7. Sort selected artifacts by dependency order.
|
||||
8. For each selected artifact:
|
||||
- resolve configured inputs;
|
||||
- build the Scriptorium run request;
|
||||
- optionally run Scriptorium render-debug;
|
||||
- run Scriptorium;
|
||||
- fail on validation-failed result;
|
||||
- verify the output exists and is non-empty;
|
||||
- record artifact metadata;
|
||||
- record artifact output metadata;
|
||||
- register `narratio.artifact.<name>` as available in the catalog.
|
||||
8. Return aggregate stage metadata containing all generated artifacts.
|
||||
9. Return aggregate analyze-stage metadata containing all generated and reused artifacts relevant to the run.
|
||||
|
||||
The Scriptorium adapter should remain generic. It should not decide which artifacts run, how dependencies work, or how artifacts are registered.
|
||||
|
||||
### Manifest and Metadata
|
||||
|
||||
The analyze stage should record all generated configured artifacts in manifest/stage metadata.
|
||||
The manifest should remain stage-oriented.
|
||||
|
||||
This means:
|
||||
|
||||
- `analyze` succeeds or fails as a full stage;
|
||||
- if `analyze` has already succeeded and the user does not force it, the runner skips it as a full stage;
|
||||
- Narratio should not implement per-artifact resume in the first version.
|
||||
|
||||
However, analyze-stage metadata should still record artifact outputs for provenance and future resolution.
|
||||
|
||||
Recommended metadata shape:
|
||||
|
||||
@@ -254,7 +368,7 @@ Recommended metadata shape:
|
||||
"path": "artifacts/session_recap.md",
|
||||
"prompt_id": "dnd_session.session_recap",
|
||||
"profile_id": "local-gemma-31b",
|
||||
"provenance": "manifest.analyze.outputs"
|
||||
"provenance": "generated.current_analyze_run"
|
||||
},
|
||||
{
|
||||
"name": "player_handout",
|
||||
@@ -263,26 +377,69 @@ Recommended metadata shape:
|
||||
"path": "artifacts/player_handout.md",
|
||||
"prompt_id": "dnd_session.player_handout",
|
||||
"profile_id": "local-gemma-31b",
|
||||
"provenance": "manifest.analyze.outputs"
|
||||
"provenance": "generated.current_analyze_run"
|
||||
}
|
||||
],
|
||||
"reused_artifacts": [
|
||||
{
|
||||
"name": "session_recap",
|
||||
"source_id": "narratio.artifact.session_recap",
|
||||
"path": "artifacts/session_recap.md",
|
||||
"provenance": "filesystem.disabled_artifact_output"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
For backward compatibility, `session_recap` may continue to emit any legacy output kind or metadata expected by existing tests and archive behavior.
|
||||
The exact struct can differ from this example, but it should preserve:
|
||||
|
||||
### Resume Behavior
|
||||
- artifact name;
|
||||
- canonical source ID;
|
||||
- output path;
|
||||
- prompt/profile provenance for generated artifacts;
|
||||
- reused-vs-generated provenance.
|
||||
|
||||
The initial implementation can keep stage-level resume behavior.
|
||||
### Resume and Force Behavior
|
||||
|
||||
That means:
|
||||
Keep resume behavior stage-level.
|
||||
|
||||
- if `analyze` has already succeeded and is not forced, the runner can skip it as before;
|
||||
- if `analyze` is forced, all enabled configured artifacts should be regenerated;
|
||||
- if one artifact fails, the stage fails;
|
||||
- a later rerun can re-execute the analyze stage as a whole.
|
||||
Recommended semantics:
|
||||
|
||||
Per-artifact resume can be considered later, but it is not necessary for the first version.
|
||||
```text
|
||||
No --force, analyze already succeeded:
|
||||
runner skips analyze, regardless of --artifacts.
|
||||
|
||||
--force, no --artifacts:
|
||||
analyze regenerates all configured artifacts with enabled: true.
|
||||
|
||||
--force --artifacts player_handout:
|
||||
analyze treats only player_handout as executable.
|
||||
all other configured artifacts are disabled for execution.
|
||||
disabled dependencies may be reused from disk.
|
||||
|
||||
--artifacts player_handout on a not-yet-completed analyze stage:
|
||||
analyze runs only player_handout.
|
||||
disabled dependencies may be reused from disk.
|
||||
```
|
||||
|
||||
`--artifacts` should not imply `--force`. It is an execution filter, not a resume override.
|
||||
|
||||
### `--artifacts` CLI Flag
|
||||
|
||||
Add an `--artifacts` flag to commands that can execute or resume the analyze stage.
|
||||
|
||||
The flag should accept one or more configured artifact names. Internally, normalize values to a set of artifact keys.
|
||||
|
||||
Recommended behavior:
|
||||
|
||||
- validate all requested artifact names against `pipeline.scriptorium.artifacts`;
|
||||
- reject unknown artifact names before running stages;
|
||||
- treat requested artifacts as the only executable artifacts for the analyze stage;
|
||||
- treat all other configured artifacts as disabled for execution;
|
||||
- allow disabled artifacts to satisfy dependencies from disk as described above;
|
||||
- if `--artifacts` is used while executing a stage other than `analyze`, either reject it or ignore it with a clear validation error. Prefer rejection.
|
||||
|
||||
The exact CLI parsing style can follow Narratio's existing conventions. Both comma-separated and repeatable values are acceptable if the CLI package supports them cleanly, but the internal representation should be a set of artifact keys.
|
||||
|
||||
### Archive Behavior
|
||||
|
||||
@@ -303,11 +460,11 @@ archive:
|
||||
required: false
|
||||
```
|
||||
|
||||
A later enhancement may add opt-in automatic promotion of generated artifacts, but explicit promotion should remain the default.
|
||||
A later enhancement may add opt-in automatic promotion of configured artifacts, but explicit promotion should remain the default.
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Phase 1: Config Model and Validation
|
||||
### Phase 1: Config Model and Defaults
|
||||
|
||||
Add or update the configured artifact model to include:
|
||||
|
||||
@@ -321,31 +478,62 @@ Add or update the configured artifact model to include:
|
||||
- `inputs`;
|
||||
- `vars`.
|
||||
|
||||
Add an internal default artifact output directory in `internal/config/defaults.go`, initially set to `artifacts`.
|
||||
|
||||
Validation rules:
|
||||
|
||||
- artifact names must match a conservative identifier pattern such as `^[a-z][a-z0-9_]*$`;
|
||||
- enabled artifacts require `prompt_id` and `output_path`;
|
||||
- enabled artifact output paths must be run-relative and must not escape the run workspace;
|
||||
- dependency references must point to enabled configured artifacts;
|
||||
- dependencies must not contain cycles;
|
||||
- `narratio.artifact.<name>` input sources must refer to known configured artifacts;
|
||||
- selected/executable artifacts require `prompt_id` and `output_path`;
|
||||
- configured artifacts that may be referenced while disabled require `output_path`;
|
||||
- configured artifact output paths must be run-relative;
|
||||
- configured artifact output paths must live under the internal artifact output directory;
|
||||
- configured artifact output paths must not escape the run workspace;
|
||||
- `narratio.artifact.<name>` input sources must refer to configured artifact keys;
|
||||
- any `narratio.artifact.<name>` input source must have a matching `depends_on` entry;
|
||||
- `depends_on` entries must refer to configured artifact keys;
|
||||
- dependencies must not contain self-references or executable cycles;
|
||||
- input names and var names must remain compatible with the Scriptorium adapter's validation rules;
|
||||
- disabled artifacts should not be executable or dependency targets.
|
||||
- unknown YAML fields must continue to fail strict decode.
|
||||
|
||||
Tests:
|
||||
|
||||
- valid single configured artifact;
|
||||
- valid multiple independent artifacts;
|
||||
- valid artifact-to-artifact dependency;
|
||||
- valid dependency on disabled artifact with output path;
|
||||
- invalid artifact name;
|
||||
- missing required fields;
|
||||
- output path outside `artifacts/`;
|
||||
- dependency on missing artifact;
|
||||
- dependency on disabled artifact;
|
||||
- missing `depends_on` for artifact input source;
|
||||
- self-dependency;
|
||||
- cycle detection;
|
||||
- typo in `narratio.artifact.<name>` source;
|
||||
- unknown YAML fields still fail strict decode.
|
||||
|
||||
### Phase 2: Runtime Artifact Catalog
|
||||
### Phase 2: CLI Filtering
|
||||
|
||||
Add the `--artifacts` flag and carry the selected artifact set into the run execution options.
|
||||
|
||||
Implementation notes:
|
||||
|
||||
- parse values according to existing CLI conventions;
|
||||
- normalize to artifact key strings;
|
||||
- validate against configured artifact definitions after config load;
|
||||
- make the selected set available to the analyze stage;
|
||||
- reject use with commands or stages where analyze cannot run.
|
||||
|
||||
Tests:
|
||||
|
||||
- no `--artifacts` means all enabled artifacts are selected;
|
||||
- one requested artifact is selected;
|
||||
- multiple requested artifacts are selected;
|
||||
- unknown requested artifact fails;
|
||||
- `--artifacts` does not imply `--force`;
|
||||
- `--artifacts` with already-succeeded analyze stage is skipped unless forced;
|
||||
- `--artifacts` on unsupported stage command fails clearly.
|
||||
|
||||
### Phase 3: Runtime Artifact Catalog
|
||||
|
||||
Introduce an internal artifact catalog abstraction.
|
||||
|
||||
@@ -354,11 +542,12 @@ Responsibilities:
|
||||
- register built-in artifact definitions;
|
||||
- register configured artifact definitions;
|
||||
- map configured artifact keys to `narratio.artifact.<name>` IDs;
|
||||
- track planned versus available artifacts;
|
||||
- track planned, available, and executable artifact states;
|
||||
- expose lookup by canonical source ID;
|
||||
- record provenance when an artifact becomes available.
|
||||
- record generated provenance;
|
||||
- record disabled-from-disk provenance.
|
||||
|
||||
Keep the catalog narrow. It should not execute anything and should not know about Scriptorium prompts.
|
||||
Keep the catalog narrow. It should not execute Scriptorium and should not understand prompt semantics.
|
||||
|
||||
Tests:
|
||||
|
||||
@@ -366,17 +555,22 @@ Tests:
|
||||
- configured source registration;
|
||||
- duplicate/conflicting source handling;
|
||||
- planned but unavailable artifact lookup;
|
||||
- selected artifact state;
|
||||
- disabled artifact state;
|
||||
- registering an artifact as available after generation;
|
||||
- resolving a configured artifact from prior manifest metadata.
|
||||
- registering a disabled artifact as available from disk;
|
||||
- resolving a configured artifact from analyze metadata if that behavior is implemented.
|
||||
|
||||
### Phase 3: Resolver Integration
|
||||
### Phase 4: Resolver Integration
|
||||
|
||||
Update artifact resolution so configured artifact IDs are resolved through the runtime catalog.
|
||||
|
||||
Resolution behavior:
|
||||
|
||||
- built-in sources continue using existing manifest-preferred, canonical-fallback behavior;
|
||||
- built-in sources continue using existing resolver behavior;
|
||||
- configured artifact sources resolve from catalog availability/provenance;
|
||||
- selected configured artifacts become available after generation;
|
||||
- disabled configured artifacts may become available from disk;
|
||||
- missing optional configured artifact inputs are omitted;
|
||||
- missing required configured artifact inputs fail clearly.
|
||||
|
||||
@@ -384,57 +578,90 @@ Tests:
|
||||
|
||||
- configured artifact consumes a built-in transcript source;
|
||||
- configured artifact consumes another configured artifact produced earlier in the same analyze run;
|
||||
- configured artifact consumes another configured artifact from prior manifest state;
|
||||
- required missing configured artifact fails;
|
||||
- optional missing configured artifact is omitted.
|
||||
- configured artifact consumes a disabled artifact resolved from disk;
|
||||
- required disabled artifact missing on disk fails;
|
||||
- required configured artifact missing fails;
|
||||
- optional missing configured artifact is omitted;
|
||||
- reused artifact provenance is recorded distinctly from generated artifact provenance.
|
||||
|
||||
### Phase 4: Analyze Stage Generalization
|
||||
### Phase 5: Analyze Stage Generalization
|
||||
|
||||
Refactor `analyze` to execute all enabled configured artifacts.
|
||||
Refactor `analyze` to execute selected configured artifacts.
|
||||
|
||||
Implementation notes:
|
||||
|
||||
- preserve the existing skip behavior when Scriptorium config is absent or no artifacts are enabled;
|
||||
- remove the hard-coded `session_recap` selection path;
|
||||
- remove the hard-coded rejection of non-`session_recap` artifacts;
|
||||
- compute deterministic dependency order before execution;
|
||||
- execute artifacts one at a time in dependency order;
|
||||
- preserve skip behavior when Scriptorium config is absent or no artifacts are selected;
|
||||
- build the runtime artifact catalog;
|
||||
- apply `--artifacts` filtering;
|
||||
- validate selected artifacts and their dependencies;
|
||||
- pre-resolve disabled dependencies from disk where required;
|
||||
- compute deterministic dependency order;
|
||||
- execute selected artifacts one at a time in dependency order;
|
||||
- keep render-debug behavior at global and artifact levels;
|
||||
- keep Scriptorium adapter invocation logic generic;
|
||||
- keep Scriptorium adapter invocation generic;
|
||||
- after each successful run, register the artifact as available in the catalog;
|
||||
- aggregate metadata across all artifacts.
|
||||
- aggregate generated and reused artifact metadata.
|
||||
|
||||
Tests:
|
||||
|
||||
- no Scriptorium config skips;
|
||||
- empty artifact map skips;
|
||||
- no selected artifacts skips;
|
||||
- disabled artifacts do not run;
|
||||
- one enabled artifact runs;
|
||||
- one selected artifact runs;
|
||||
- multiple independent artifacts run in deterministic order;
|
||||
- dependent artifact receives prior artifact as input;
|
||||
- dependent selected artifact receives prior selected artifact as input;
|
||||
- dependent selected artifact receives disabled-from-disk artifact as input;
|
||||
- render-debug works for configured artifacts;
|
||||
- Scriptorium validation failure fails the stage;
|
||||
- missing required input fails the stage;
|
||||
- successful outputs are non-empty and recorded.
|
||||
- successful outputs are non-empty and recorded;
|
||||
- artifact filter executes only requested artifacts.
|
||||
|
||||
### Phase 5: Manifest Compatibility and Output Kinds
|
||||
### Phase 6: Manifest and Stage Metadata
|
||||
|
||||
Update manifest/stage output recording to support dynamic configured artifacts.
|
||||
Update analyze-stage metadata and manifest output recording to support dynamic configured artifacts.
|
||||
|
||||
Recommended behavior:
|
||||
|
||||
- every configured artifact gets `source_id: narratio.artifact.<name>`;
|
||||
- every configured artifact gets a generic output kind such as `scriptorium_artifact`;
|
||||
- `session_recap` may also retain legacy metadata/output kind for compatibility;
|
||||
- manifest provenance should be sufficient for later resolution during resume.
|
||||
- every generated configured artifact gets `source_id: narratio.artifact.<name>`;
|
||||
- every generated configured artifact gets a generic output kind such as `scriptorium_artifact`;
|
||||
- reused disabled artifacts are recorded separately from generated artifacts;
|
||||
- metadata is sufficient for debugging, provenance, and future resolver support;
|
||||
- metadata does not create per-artifact resume semantics.
|
||||
|
||||
Because this is a pre-release hard cutover, do not preserve a special legacy `session_recap` output kind unless a current internal test or archive path still requires it temporarily. Prefer updating tests and examples to treat `session_recap` as an ordinary configured artifact.
|
||||
|
||||
Tests:
|
||||
|
||||
- manifest records one configured artifact;
|
||||
- manifest records multiple configured artifacts;
|
||||
- `session_recap` remains compatible with existing expectations;
|
||||
- configured artifact can be resolved from manifest metadata on later run/resume.
|
||||
- metadata records one generated configured artifact;
|
||||
- metadata records multiple generated configured artifacts;
|
||||
- metadata records reused disabled artifact provenance;
|
||||
- `session_recap` is recorded as a normal configured artifact;
|
||||
- manifest still treats `analyze` as a single succeeded or failed stage;
|
||||
- runner skip behavior remains stage-level.
|
||||
|
||||
### Phase 6: Documentation and Examples
|
||||
### Phase 7: Archive and Promotion Review
|
||||
|
||||
Review archive behavior after dynamic artifacts are recorded.
|
||||
|
||||
Implementation notes:
|
||||
|
||||
- do not automatically promote every configured artifact;
|
||||
- keep `archive.promote_artifacts` explicit;
|
||||
- update default or example promotion rules to use configured `session_recap` output path;
|
||||
- ensure required promotion rules fail clearly when selected artifact generation did not produce a required file.
|
||||
|
||||
Tests:
|
||||
|
||||
- generated artifact can be promoted by explicit archive rule;
|
||||
- required archive promotion fails if selected artifact was not generated and no file exists;
|
||||
- optional archive promotion skips cleanly if file is absent;
|
||||
- hard cutover does not rely on hard-coded `session_recap` generation.
|
||||
|
||||
### Phase 8: Documentation and Examples
|
||||
|
||||
Update documentation after the implementation is complete.
|
||||
|
||||
@@ -445,58 +672,89 @@ Recommended documentation changes:
|
||||
- update `docs/stages/analyze.md` to describe generic Scriptorium artifact generation;
|
||||
- update Scriptorium integration docs only if the adapter contract changes;
|
||||
- update full annotated pipeline examples;
|
||||
- add at least one example with multiple artifacts and one dependency.
|
||||
- add at least one example with multiple artifacts and one dependency;
|
||||
- document `--artifacts` behavior and its relationship to `--force`;
|
||||
- remove documentation stating that only `session_recap` is supported.
|
||||
|
||||
Documentation should make clear that:
|
||||
|
||||
- configured artifact source IDs use `narratio.artifact.<name>`;
|
||||
- `depends_on` uses artifact keys, not full source IDs;
|
||||
- artifact-to-artifact source references require explicit `depends_on`;
|
||||
- disabled artifacts can be reused from disk when required by selected artifacts;
|
||||
- `--artifacts` filters execution but does not imply `--force`;
|
||||
- archive promotion remains explicit;
|
||||
- per-artifact resume is not part of the initial implementation.
|
||||
|
||||
## Migration Strategy
|
||||
|
||||
Existing configurations using `session_recap` should continue working.
|
||||
Because Narratio is pre-release, perform a hard cutover.
|
||||
|
||||
Recommended migration path:
|
||||
Required changes:
|
||||
|
||||
1. Treat `pipeline.scriptorium.artifacts.session_recap` as a normal configured artifact.
|
||||
2. Keep `narratio.artifact.session_recap` as a supported source ID.
|
||||
3. Preserve existing default archive promotion for `artifacts/session_recap.md` where applicable.
|
||||
4. Preserve existing tests for session recap behavior while adding new generic artifact tests.
|
||||
5. Remove or update documentation that says only `session_recap` is supported.
|
||||
1. Remove the hard-coded `session_recap` analyze behavior.
|
||||
2. Require `session_recap` to be declared under `pipeline.scriptorium.artifacts.session_recap` if the operator wants a session recap.
|
||||
3. Treat `narratio.artifact.session_recap` as valid only when `session_recap` is a configured artifact key.
|
||||
4. Update config examples to show `session_recap` as a normal configured artifact.
|
||||
5. Update tests to stop assuming that `session_recap` is a built-in analyze artifact.
|
||||
6. Keep archive promotion explicit and path-based.
|
||||
|
||||
## Open Decisions
|
||||
Example replacement config:
|
||||
|
||||
Before implementation, decide the following:
|
||||
|
||||
1. Should configured artifact output paths be required to live under `artifacts/`?
|
||||
2. Should disabled artifacts be valid references for `narratio.artifact.<name>` sources, or should validation fail immediately?
|
||||
3. Should `depends_on` be required whenever an artifact input references another configured artifact, or should Narratio infer dependencies from input source IDs?
|
||||
4. Should the first implementation support configured artifact resolution from prior manifest state, or only from artifacts produced earlier in the same analyze execution?
|
||||
5. Should `session_recap` keep a legacy output kind forever, or only through a compatibility window?
|
||||
|
||||
Recommended answers:
|
||||
|
||||
1. Prefer requiring configured artifact outputs under `artifacts/` unless there is a strong reason not to.
|
||||
2. Fail references to disabled artifacts.
|
||||
3. Require `depends_on` for clarity, and validate that it matches artifact input references.
|
||||
4. Support prior manifest resolution if the existing manifest model makes this straightforward; otherwise defer.
|
||||
5. Keep legacy `session_recap` compatibility until the next major release boundary.
|
||||
```yaml
|
||||
scriptorium:
|
||||
binary: scriptorium
|
||||
config_path: /etc/scriptorium/config.yml
|
||||
timeout: 10m
|
||||
render_debug: false
|
||||
artifacts:
|
||||
session_recap:
|
||||
enabled: true
|
||||
prompt_id: dnd_session.session_recap
|
||||
profile_id: local-gemma-31b
|
||||
output_path: artifacts/session_recap.md
|
||||
timeout: 20m
|
||||
inputs:
|
||||
transcript:
|
||||
source: narratio.transcript.trimmed
|
||||
required: true
|
||||
prior_recap:
|
||||
source: previous_session_artifact
|
||||
artifact: artifacts/session_recap.md
|
||||
required: false
|
||||
vars:
|
||||
artifact_title: Session Recap
|
||||
```
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
The feature should be considered complete when:
|
||||
The feature is complete when:
|
||||
|
||||
- operators can define more than one enabled Scriptorium artifact in `pipeline.yml`;
|
||||
- Narratio runs all enabled artifacts in deterministic dependency order;
|
||||
- Narratio runs selected artifacts in deterministic dependency order;
|
||||
- configured artifacts are addressable as `narratio.artifact.<name>`;
|
||||
- one configured artifact can consume another configured artifact as an input;
|
||||
- missing required inputs fail clearly;
|
||||
- artifact-to-artifact input references require explicit `depends_on`;
|
||||
- disabled artifacts can satisfy dependencies from existing on-disk outputs;
|
||||
- missing required disabled artifacts fail clearly;
|
||||
- optional missing inputs are omitted;
|
||||
- `--artifacts` can selectively execute valid configured artifact names;
|
||||
- `--artifacts` does not imply `--force`;
|
||||
- render-debug behavior works for all configured artifacts;
|
||||
- generated artifacts are recorded in manifest/stage metadata;
|
||||
- existing `session_recap` behavior remains compatible;
|
||||
- generated and reused artifacts are recorded in analyze-stage metadata;
|
||||
- `session_recap` is no longer hard-coded and works as a normal configured artifact;
|
||||
- archive promotion remains explicit;
|
||||
- tests cover config validation, dependency sorting, resolver behavior, analyze execution, and manifest metadata.
|
||||
- tests cover config validation, dependency sorting, disabled artifact resolution, resolver behavior, CLI filtering, analyze execution, archive interactions, and metadata.
|
||||
|
||||
## Suggested Implementation Order
|
||||
|
||||
1. Config model, defaults, and validation.
|
||||
2. CLI parsing and propagation of `--artifacts` selection.
|
||||
3. Runtime artifact catalog.
|
||||
4. Resolver integration for configured artifacts.
|
||||
5. Analyze stage generalization.
|
||||
6. Stage metadata and manifest output recording.
|
||||
7. Archive behavior review.
|
||||
8. Documentation and examples.
|
||||
|
||||
This order keeps the most static pieces first, then moves into execution behavior once the configuration contract is explicit and well tested.
|
||||
|
||||
Reference in New Issue
Block a user