Update feature roadmap to include a named pipeline profile configuration model
This commit is contained in:
@@ -250,12 +250,61 @@ The goal is to make configuration discoverable and avoid implicit or hidden
|
||||
operational values. User-visible defaults and cross-package operational defaults
|
||||
should be defined in config code.
|
||||
|
||||
Unless documented otherwise, precedence is:
|
||||
Configuration should be organized around named pipeline profiles. A pipeline is
|
||||
a fixed-shape template for the application workflow, not a free-form DAG or
|
||||
general workflow program. The six-stage flow remains fixed:
|
||||
|
||||
1. CLI flags
|
||||
2. environment variables
|
||||
3. configuration file
|
||||
4. built-in defaults
|
||||
```text
|
||||
input -> chunk -> extract -> merge -> normalize -> output
|
||||
```
|
||||
|
||||
A pipeline profile should bind registered modules to those stage slots:
|
||||
|
||||
- one shared input module;
|
||||
- one shared chunk module by default;
|
||||
- one or more artifact lanes, each with extract, merge, normalize, and
|
||||
validator behavior;
|
||||
- one output module.
|
||||
|
||||
The MVP should use one shared chunk module per pipeline. Per-lane chunk
|
||||
overrides are a future extension and should be added only if a real artifact
|
||||
lane needs different chunking.
|
||||
|
||||
The CLI should select a named pipeline by ID, such as
|
||||
`notarius run dnd-session --input session.json`. Structural module selection
|
||||
should come from configuration, not ad hoc CLI flags. CLI flags may select a
|
||||
subset of configured artifact lanes, such as `--only spells,npcs`, and may
|
||||
override operational settings such as model, concurrency, output directory, or
|
||||
diagnostics directory.
|
||||
|
||||
Pipeline definitions should support compact defaults:
|
||||
|
||||
- `chunk`: `generic`;
|
||||
- lane `merge`: `appendorder`;
|
||||
- lane `normalize`: `noop`;
|
||||
- `output`: `json`;
|
||||
- `llm_profile`: `default` where an LLM profile is needed.
|
||||
|
||||
Module bindings should support both string shorthand and object form. For
|
||||
example, `extract: dnd/spells` and
|
||||
`extract: {module: dnd/spells, llm_profile: fast}` should normalize to the same
|
||||
internal binding type.
|
||||
|
||||
Module registries should expose module metadata, including flat string
|
||||
capabilities, without requiring module construction. Config validation should
|
||||
fail fast on unknown module keys, unknown pipeline IDs, missing required slots,
|
||||
missing capabilities, unknown LLM profiles, empty artifact-lane sets, or
|
||||
`--only` lane names that do not exist in the selected pipeline.
|
||||
|
||||
Keep capabilities as a flat string set. Do not evolve capabilities into a type
|
||||
system unless real module interactions prove the need.
|
||||
|
||||
Unless documented otherwise, precedence from lowest to highest is:
|
||||
|
||||
1. built-in defaults
|
||||
2. configuration file
|
||||
3. environment variables
|
||||
4. CLI flags
|
||||
|
||||
Prefer YAML configuration unless the project has a strong reason to use another
|
||||
format. Config files should be discoverable at
|
||||
@@ -265,8 +314,15 @@ Configuration files should not contain raw secrets unless the application is
|
||||
explicitly designed for that. Prefer environment variables or secret files for
|
||||
secrets.
|
||||
|
||||
Stage-module-specific configuration should remain grouped by the module that
|
||||
owns it.
|
||||
Stage-module-specific configuration should remain inline with the pipeline slot
|
||||
that owns it. Do not add named module instances until repeated inline settings
|
||||
create real drift or duplication. LLM profiles are the justified top-level
|
||||
exception because model settings are cross-cutting.
|
||||
|
||||
The run manifest should record the selected `pipeline_id` and a digest of the
|
||||
resolved pipeline definition after defaults and lane selection are applied.
|
||||
`pipeline_id` alone is not sufficient provenance because a named pipeline can
|
||||
change over time.
|
||||
|
||||
## Embedded Assets
|
||||
|
||||
|
||||
Reference in New Issue
Block a user