2.1 KiB
ADR-0002: Linear pipes-and-filters pipeline, not a general DAG
Status: Accepted Date: 2026-07-13
Context
Notarius processes source material through one known workflow:
input -> chunk -> extract -> merge -> normalize -> output
Input and chunking apply to the source as a whole. Each selected artifact lane then performs extract, merge, and normalize, after which output aggregates the lane outcomes. Chunk extraction has a natural scatter-gather shape, but no current use case requires arbitrary branches, joins, or user-defined stage topology.
Decision
Notarius implements a fixed six-stage pipes-and-filters pipeline. Configuration selects implementations for these stages but cannot add stages, reorder them, or define an arbitrary graph.
The framework owns stage sequencing and the scatter-gather boundary between chunk, extract, and merge. Extract results are handed to merge in deterministic source-chunk order regardless of execution strategy. Each artifact lane remains logically linear. Output runs after every selected lane has either produced an accepted normalized artifact or reached a recorded rejection. A framework execution failure aborts the pipeline.
The runner's concrete internal representation and stage-specific scheduling policies are implementation details. Concurrency must preserve the pipeline's deterministic handoffs, validation behavior, and provenance, and all execution strategies must continue to honor context cancellation.
Alternatives considered
- Build a general DAG engine now. This would support hypothetical branching topologies, but would add scheduling, topology validation, configuration, and state-management complexity without a current consumer. Revisit this choice only when a concrete workflow requires a topology the fixed pipeline cannot express.
Consequences
The runner, configuration model, and operator mental model remain small. Stage ownership stays visible, and general chunking, merging, or normalization cannot be hidden inside extractors. A future DAG requirement will require an explicit architectural change rather than incremental exceptions to the fixed pipeline.