Update feature roadmap to include a named pipeline profile configuration model

This commit is contained in:
2026-07-03 10:09:02 -05:00
parent b4ee4c64f0
commit c4da7bea1a
8 changed files with 333 additions and 66 deletions

View File

@@ -303,6 +303,8 @@ Per-run provenance record.
```go
type RunManifest struct {
EnvelopeVersion string `json:"envelope_version"`
PipelineID string `json:"pipeline_id"`
PipelineDigest string `json:"pipeline_digest"`
InputModule string `json:"input_module"`
Chunker string `json:"chunker"`
SourceDigests []string `json:"source_digests"`
@@ -316,8 +318,8 @@ type RunManifest struct {
```
The manifest should eventually include model names, prompt IDs, prompt hashes,
response schema versions, config source, started/completed timestamps, and
diagnostics paths.
response schema versions, config source, redacted resolved config digest,
started/completed timestamps, and diagnostics paths.
## Initial Extractor Targets
@@ -393,6 +395,107 @@ The architecture should support extractors outside the D&D domain. Examples:
These should be addable as extract modules without changing runner,
validator, source-reference, or LLM framework contracts.
## Configuration Model
Notarius should use named pipeline profiles selected by ID at the CLI. A
pipeline is a fixed-shape template for the known application workflow, not a
free-form list of steps:
```text
input -> chunk -> extract -> merge -> normalize -> output
```
A pipeline profile should define one shared front end and one or more artifact
lanes:
- shared input module;
- shared chunk module by default;
- artifact lanes containing extract, merge, normalize, and validator behavior;
- shared output module.
The MVP should use one shared chunk module per pipeline. Per-lane chunk
overrides can be added later if an artifact lane, such as combat, proves it
needs a different chunking strategy.
Example shape:
```yaml
llm_profiles:
default:
model: example-model
max_concurrency: 4
pipelines:
dnd-session:
input: seriatim
chunk: dnd/transcript
artifacts:
spells:
extract: dnd/spells
normalize: dnd/spells
npcs:
extract: dnd/npcs
items:
extract: dnd/items
```
The CLI should run named pipelines:
```sh
notarius run dnd-session --input session-014.json
notarius run dnd-session --input session-014.json --only spells,npcs
```
`--only` should select configured artifact lanes. It should not create an
ad hoc pipeline. Structural module selection should come from config, while CLI
flags may override operational knobs such as model, concurrency, output
directory, and diagnostics directory.
Initial defaults:
- `chunk`: `generic`;
- lane `merge`: `appendorder`;
- lane `normalize`: `noop`;
- `output`: `json`;
- `llm_profile`: `default` where an LLM profile is needed.
Module bindings should support both string shorthand and object form:
```yaml
extract: dnd/spells
```
```yaml
extract:
module: dnd/spells
llm_profile: fast
prompt_version: v1
```
Both forms should normalize into a single internal `ModuleBinding` shape before
validation and manifest hashing.
Pipeline validation should use module metadata declared through registries.
Modules should expose flat string capability metadata, such as `speaker` or
`timestamps`, without requiring module construction. Config validation should
fail fast for:
- unknown pipeline IDs;
- unknown module keys;
- missing required slots;
- missing required capabilities;
- unknown LLM profiles;
- empty artifact-lane sets;
- `--only` lane names that do not exist in the selected pipeline.
The MVP should keep pipelines config-file-only. Built-in pipeline profiles can
be added later if the project needs embedded defaults, but that introduces
merge/override semantics that the MVP does not need.
The resolved pipeline definition should be hashed after defaults and lane
selection are applied. The run manifest should record both `pipeline_id` and
`pipeline_digest`; a pipeline ID alone is not stable provenance.
## Proposed Pipeline Flow
The application workflow should be first-class:
@@ -404,21 +507,22 @@ input -> chunk -> extract -> merge -> normalize -> output
Proposed runner flow:
1. Load effective config.
2. Create diagnostics run directory.
3. Resolve the configured input module through the input adapter registry.
4. Read source input.
5. Parse source input into a `SourceDocument`.
6. Validate source-document invariants.
7. Resolve the configured chunker.
8. Chunk source units into deterministic source chunks.
9. Resolve configured extractor instances through a registry.
10. Extract from chunks in extractor-defined mode.
11. Merge per-chunk artifact candidates deterministically.
12. Normalize merged artifact candidates.
13. Run deterministic validators before LLM-backed validators.
14. Retain approved artifacts and rejected-artifact diagnostics.
15. Serialize final output JSON.
16. Write run manifest, diagnostics, and optional report JSON.
2. Resolve the selected pipeline profile by ID.
3. Apply defaults and `--only` lane selection.
4. Validate module keys, lane definitions, LLM profiles, and capabilities.
5. Hash the resolved pipeline definition.
6. Create diagnostics run directory.
7. Resolve the configured input module through the input adapter registry.
8. Read source input.
9. Parse source input into a `SourceDocument`.
10. Validate source-document invariants.
11. Resolve the configured chunker.
12. Chunk source units into deterministic source chunks.
13. Resolve configured artifact lanes through registries.
14. Extract, merge, normalize, and validate each selected artifact lane.
15. Retain approved artifacts and rejected-artifact diagnostics.
16. Serialize output files and run-level manifest/index.
17. Write diagnostics and optional report JSON.
The runner should operate on source documents and source chunks only. Any
transcript-specific behavior should happen before the runner, inside the input
@@ -504,6 +608,17 @@ yet.
- Core source metadata should remain `map[string]any`. Well-known metadata keys
should be documented as conventions, and input modules may expose typed
accessor helpers for their own metadata.
- Configuration should use named pipeline profiles selected by ID at the CLI.
- A pipeline profile should be a fixed template, not a free-form DAG: shared
input and chunk stages, one or more artifact lanes, and shared output.
- `--only` should select configured artifact lanes without creating ad hoc
pipelines.
- Module bindings should support string shorthand and inline object settings,
normalized into one internal binding shape.
- Registries should expose flat capability metadata so config can fail fast on
invalid module combinations.
- The run manifest should record both `pipeline_id` and a digest of the resolved
pipeline definition after defaults and lane selection.
## Open Design Questions
@@ -511,8 +626,8 @@ yet.
should use domain-specific merge?
- Which artifact types need domain-specific normalization for deduplication,
identity resolution, or consistency?
- What should the configuration model look like for selecting input, chunk,
extract, merge, normalize, output, and validator modules?
- Which operational settings should be allowed as CLI/environment overrides
without weakening pipeline provenance?
## Near-Term Documentation Tasks