Update feature roadmap to include a named pipeline profile configuration model
This commit is contained in:
@@ -303,6 +303,8 @@ Per-run provenance record.
|
||||
```go
|
||||
type RunManifest struct {
|
||||
EnvelopeVersion string `json:"envelope_version"`
|
||||
PipelineID string `json:"pipeline_id"`
|
||||
PipelineDigest string `json:"pipeline_digest"`
|
||||
InputModule string `json:"input_module"`
|
||||
Chunker string `json:"chunker"`
|
||||
SourceDigests []string `json:"source_digests"`
|
||||
@@ -316,8 +318,8 @@ type RunManifest struct {
|
||||
```
|
||||
|
||||
The manifest should eventually include model names, prompt IDs, prompt hashes,
|
||||
response schema versions, config source, started/completed timestamps, and
|
||||
diagnostics paths.
|
||||
response schema versions, config source, redacted resolved config digest,
|
||||
started/completed timestamps, and diagnostics paths.
|
||||
|
||||
## Initial Extractor Targets
|
||||
|
||||
@@ -393,6 +395,107 @@ The architecture should support extractors outside the D&D domain. Examples:
|
||||
These should be addable as extract modules without changing runner,
|
||||
validator, source-reference, or LLM framework contracts.
|
||||
|
||||
## Configuration Model
|
||||
|
||||
Notarius should use named pipeline profiles selected by ID at the CLI. A
|
||||
pipeline is a fixed-shape template for the known application workflow, not a
|
||||
free-form list of steps:
|
||||
|
||||
```text
|
||||
input -> chunk -> extract -> merge -> normalize -> output
|
||||
```
|
||||
|
||||
A pipeline profile should define one shared front end and one or more artifact
|
||||
lanes:
|
||||
|
||||
- shared input module;
|
||||
- shared chunk module by default;
|
||||
- artifact lanes containing extract, merge, normalize, and validator behavior;
|
||||
- shared output module.
|
||||
|
||||
The MVP should use one shared chunk module per pipeline. Per-lane chunk
|
||||
overrides can be added later if an artifact lane, such as combat, proves it
|
||||
needs a different chunking strategy.
|
||||
|
||||
Example shape:
|
||||
|
||||
```yaml
|
||||
llm_profiles:
|
||||
default:
|
||||
model: example-model
|
||||
max_concurrency: 4
|
||||
|
||||
pipelines:
|
||||
dnd-session:
|
||||
input: seriatim
|
||||
chunk: dnd/transcript
|
||||
artifacts:
|
||||
spells:
|
||||
extract: dnd/spells
|
||||
normalize: dnd/spells
|
||||
npcs:
|
||||
extract: dnd/npcs
|
||||
items:
|
||||
extract: dnd/items
|
||||
```
|
||||
|
||||
The CLI should run named pipelines:
|
||||
|
||||
```sh
|
||||
notarius run dnd-session --input session-014.json
|
||||
notarius run dnd-session --input session-014.json --only spells,npcs
|
||||
```
|
||||
|
||||
`--only` should select configured artifact lanes. It should not create an
|
||||
ad hoc pipeline. Structural module selection should come from config, while CLI
|
||||
flags may override operational knobs such as model, concurrency, output
|
||||
directory, and diagnostics directory.
|
||||
|
||||
Initial defaults:
|
||||
|
||||
- `chunk`: `generic`;
|
||||
- lane `merge`: `appendorder`;
|
||||
- lane `normalize`: `noop`;
|
||||
- `output`: `json`;
|
||||
- `llm_profile`: `default` where an LLM profile is needed.
|
||||
|
||||
Module bindings should support both string shorthand and object form:
|
||||
|
||||
```yaml
|
||||
extract: dnd/spells
|
||||
```
|
||||
|
||||
```yaml
|
||||
extract:
|
||||
module: dnd/spells
|
||||
llm_profile: fast
|
||||
prompt_version: v1
|
||||
```
|
||||
|
||||
Both forms should normalize into a single internal `ModuleBinding` shape before
|
||||
validation and manifest hashing.
|
||||
|
||||
Pipeline validation should use module metadata declared through registries.
|
||||
Modules should expose flat string capability metadata, such as `speaker` or
|
||||
`timestamps`, without requiring module construction. Config validation should
|
||||
fail fast for:
|
||||
|
||||
- unknown pipeline IDs;
|
||||
- unknown module keys;
|
||||
- missing required slots;
|
||||
- missing required capabilities;
|
||||
- unknown LLM profiles;
|
||||
- empty artifact-lane sets;
|
||||
- `--only` lane names that do not exist in the selected pipeline.
|
||||
|
||||
The MVP should keep pipelines config-file-only. Built-in pipeline profiles can
|
||||
be added later if the project needs embedded defaults, but that introduces
|
||||
merge/override semantics that the MVP does not need.
|
||||
|
||||
The resolved pipeline definition should be hashed after defaults and lane
|
||||
selection are applied. The run manifest should record both `pipeline_id` and
|
||||
`pipeline_digest`; a pipeline ID alone is not stable provenance.
|
||||
|
||||
## Proposed Pipeline Flow
|
||||
|
||||
The application workflow should be first-class:
|
||||
@@ -404,21 +507,22 @@ input -> chunk -> extract -> merge -> normalize -> output
|
||||
Proposed runner flow:
|
||||
|
||||
1. Load effective config.
|
||||
2. Create diagnostics run directory.
|
||||
3. Resolve the configured input module through the input adapter registry.
|
||||
4. Read source input.
|
||||
5. Parse source input into a `SourceDocument`.
|
||||
6. Validate source-document invariants.
|
||||
7. Resolve the configured chunker.
|
||||
8. Chunk source units into deterministic source chunks.
|
||||
9. Resolve configured extractor instances through a registry.
|
||||
10. Extract from chunks in extractor-defined mode.
|
||||
11. Merge per-chunk artifact candidates deterministically.
|
||||
12. Normalize merged artifact candidates.
|
||||
13. Run deterministic validators before LLM-backed validators.
|
||||
14. Retain approved artifacts and rejected-artifact diagnostics.
|
||||
15. Serialize final output JSON.
|
||||
16. Write run manifest, diagnostics, and optional report JSON.
|
||||
2. Resolve the selected pipeline profile by ID.
|
||||
3. Apply defaults and `--only` lane selection.
|
||||
4. Validate module keys, lane definitions, LLM profiles, and capabilities.
|
||||
5. Hash the resolved pipeline definition.
|
||||
6. Create diagnostics run directory.
|
||||
7. Resolve the configured input module through the input adapter registry.
|
||||
8. Read source input.
|
||||
9. Parse source input into a `SourceDocument`.
|
||||
10. Validate source-document invariants.
|
||||
11. Resolve the configured chunker.
|
||||
12. Chunk source units into deterministic source chunks.
|
||||
13. Resolve configured artifact lanes through registries.
|
||||
14. Extract, merge, normalize, and validate each selected artifact lane.
|
||||
15. Retain approved artifacts and rejected-artifact diagnostics.
|
||||
16. Serialize output files and run-level manifest/index.
|
||||
17. Write diagnostics and optional report JSON.
|
||||
|
||||
The runner should operate on source documents and source chunks only. Any
|
||||
transcript-specific behavior should happen before the runner, inside the input
|
||||
@@ -504,6 +608,17 @@ yet.
|
||||
- Core source metadata should remain `map[string]any`. Well-known metadata keys
|
||||
should be documented as conventions, and input modules may expose typed
|
||||
accessor helpers for their own metadata.
|
||||
- Configuration should use named pipeline profiles selected by ID at the CLI.
|
||||
- A pipeline profile should be a fixed template, not a free-form DAG: shared
|
||||
input and chunk stages, one or more artifact lanes, and shared output.
|
||||
- `--only` should select configured artifact lanes without creating ad hoc
|
||||
pipelines.
|
||||
- Module bindings should support string shorthand and inline object settings,
|
||||
normalized into one internal binding shape.
|
||||
- Registries should expose flat capability metadata so config can fail fast on
|
||||
invalid module combinations.
|
||||
- The run manifest should record both `pipeline_id` and a digest of the resolved
|
||||
pipeline definition after defaults and lane selection.
|
||||
|
||||
## Open Design Questions
|
||||
|
||||
@@ -511,8 +626,8 @@ yet.
|
||||
should use domain-specific merge?
|
||||
- Which artifact types need domain-specific normalization for deduplication,
|
||||
identity resolution, or consistency?
|
||||
- What should the configuration model look like for selecting input, chunk,
|
||||
extract, merge, normalize, output, and validator modules?
|
||||
- Which operational settings should be allowed as CLI/environment overrides
|
||||
without weakening pipeline provenance?
|
||||
|
||||
## Near-Term Documentation Tasks
|
||||
|
||||
|
||||
Reference in New Issue
Block a user