Update feature roadmap to include a named pipeline profile configuration model

This commit is contained in:
2026-07-03 10:09:02 -05:00
parent b4ee4c64f0
commit c4da7bea1a
8 changed files with 333 additions and 66 deletions

View File

@@ -250,12 +250,61 @@ The goal is to make configuration discoverable and avoid implicit or hidden
operational values. User-visible defaults and cross-package operational defaults
should be defined in config code.
Unless documented otherwise, precedence is:
Configuration should be organized around named pipeline profiles. A pipeline is
a fixed-shape template for the application workflow, not a free-form DAG or
general workflow program. The six-stage flow remains fixed:
1. CLI flags
2. environment variables
3. configuration file
4. built-in defaults
```text
input -> chunk -> extract -> merge -> normalize -> output
```
A pipeline profile should bind registered modules to those stage slots:
- one shared input module;
- one shared chunk module by default;
- one or more artifact lanes, each with extract, merge, normalize, and
validator behavior;
- one output module.
The MVP should use one shared chunk module per pipeline. Per-lane chunk
overrides are a future extension and should be added only if a real artifact
lane needs different chunking.
The CLI should select a named pipeline by ID, such as
`notarius run dnd-session --input session.json`. Structural module selection
should come from configuration, not ad hoc CLI flags. CLI flags may select a
subset of configured artifact lanes, such as `--only spells,npcs`, and may
override operational settings such as model, concurrency, output directory, or
diagnostics directory.
Pipeline definitions should support compact defaults:
- `chunk`: `generic`;
- lane `merge`: `appendorder`;
- lane `normalize`: `noop`;
- `output`: `json`;
- `llm_profile`: `default` where an LLM profile is needed.
Module bindings should support both string shorthand and object form. For
example, `extract: dnd/spells` and
`extract: {module: dnd/spells, llm_profile: fast}` should normalize to the same
internal binding type.
Module registries should expose module metadata, including flat string
capabilities, without requiring module construction. Config validation should
fail fast on unknown module keys, unknown pipeline IDs, missing required slots,
missing capabilities, unknown LLM profiles, empty artifact-lane sets, or
`--only` lane names that do not exist in the selected pipeline.
Keep capabilities as a flat string set. Do not evolve capabilities into a type
system unless real module interactions prove the need.
Unless documented otherwise, precedence from lowest to highest is:
1. built-in defaults
2. configuration file
3. environment variables
4. CLI flags
Prefer YAML configuration unless the project has a strong reason to use another
format. Config files should be discoverable at
@@ -265,8 +314,15 @@ Configuration files should not contain raw secrets unless the application is
explicitly designed for that. Prefer environment variables or secret files for
secrets.
Stage-module-specific configuration should remain grouped by the module that
owns it.
Stage-module-specific configuration should remain inline with the pipeline slot
that owns it. Do not add named module instances until repeated inline settings
create real drift or duplication. LLM profiles are the justified top-level
exception because model settings are cross-cutting.
The run manifest should record the selected `pipeline_id` and a digest of the
resolved pipeline definition after defaults and lane selection are applied.
`pipeline_id` alone is not sufficient provenance because a named pipeline can
change over time.
## Embedded Assets