Update feature roadmap to include a named pipeline profile configuration model

This commit is contained in:
2026-07-03 10:09:02 -05:00
parent b4ee4c64f0
commit c4da7bea1a
8 changed files with 333 additions and 66 deletions

View File

@@ -21,7 +21,8 @@ In scope:
- prompt registry pattern;
- response-schema registry pattern;
- diagnostics run directory pattern;
- minimal config structs and defaults for implemented runtime pieces.
- config loading and validation for named pipeline profiles and implemented
runtime pieces.
Out of scope:
@@ -31,6 +32,7 @@ Out of scope:
- correction ledger terminology;
- Audita module or validator behavior;
- real D&D prompts or schemas unless needed as inert registry tests.
- embedded built-in pipeline profiles.
## Proposed Stages
@@ -82,28 +84,55 @@ Initial diagnostics should cover:
- invocation metadata;
- redacted effective config;
- selected pipeline ID;
- resolved pipeline definition and digest;
- source document artifact;
- run report placeholder;
- error log on failure.
Avoid Audita-specific artifact names such as correction ledger.
### Stage 5: Minimal Runtime Config
### Stage 5: Pipeline Profile Config
Add config structs and defaults only for infrastructure that now exists.
Add config structs, loading, defaults, and validation for named pipeline
profiles.
Initial config areas:
- input module key;
- extractor keys;
- primary LLM settings;
- validation LLM settings if needed;
- `llm_profiles`;
- `pipelines.<pipeline_id>.input`;
- `pipelines.<pipeline_id>.chunk`;
- `pipelines.<pipeline_id>.artifacts.<lane>.extract`;
- lane `merge`, `normalize`, and validator settings;
- output module selection;
- inline module-binding object form and string shorthand;
- default `chunk`, `merge`, `normalize`, `output`, and `llm_profile`;
- selected pipeline ID and lane filtering for runtime use;
- concurrency;
- work directory;
- diagnostics retention.
Config loading can remain minimal unless the implementation needs full file/env
precedence at this checkpoint.
Config loading should support the standard precedence model:
1. built-in defaults
2. configuration file
3. environment variables
4. CLI flags
Structural module selection should come from pipeline config. CLI flags may
override operational settings and artifact lane selection, but should not offer
ad hoc `--extractor` or `--chunker` wiring.
Add validation for unknown pipeline IDs, unknown module keys, missing required
slots, missing capabilities, unknown LLM profiles, empty artifact-lane sets, and
invalid lane selections.
If the CLI shell is ready, add:
```sh
notarius config validate
notarius pipelines list
```
## Done Criteria
@@ -113,10 +142,13 @@ precedence at this checkpoint.
ledger code has been copied.
- Runtime tests cover secret redaction, schema registry lookup, prompt metadata,
and scheduler behavior where applicable.
- Config tests cover named pipeline profiles, defaults, lane selection,
capability validation, and resolved pipeline digesting.
## Review Questions
- Did we copy only reusable infrastructure?
- Do provider-specific types stay behind adapter/runtime boundaries?
- Are diagnostics names and report concepts extraction-oriented?
- Is config limited to implemented behavior?
- Is config limited to named pipeline profiles and implemented behavior?
- Are structural pipeline changes kept out of ad hoc CLI flags?