Revise the architecture plan to reflect an input -> chunk -> process -> merge -> normalize -> output workflow

docs/roadmap/initial-architecture.md

@@ -11,21 +11,21 @@ Notarius should extract structured JSON artifacts from primary source inputs
 using modular, LLM-backed extractors.
 
 The first MVP should target audio transcripts generated by Seriatim. That
-choice should be implemented as an input adapter, not as a transcript-specific
-assumption in the application core. Later input sources, such as unstructured
-Markdown notes or Obsidian documents, should be addable through new adapters and
-extractors without reshaping the framework.
+choice should be implemented as an input-stage module, not as a
+transcript-specific assumption in the application core. Later input sources,
+such as unstructured Markdown notes or Obsidian documents, should be addable
+through new input and process modules without reshaping the framework.
 
 The first extraction domain should be D&D session analysis, starting with spell
-casts. That domain should live in extractor packages and related schemas, not in
-core framework packages.
+casts. That domain should live in process-stage modules and related schemas, not
+in core framework packages.
 
 The application should follow the same broad architecture as Audita:
 
 - deterministic core packages for config, source documents, artifacts, diagnostics, and reporting;
-- input adapters that translate external source formats into a small internal source model;
+- input-stage modules that translate external source formats into a small internal source model;
 - reusable framework packages for contracts, orchestration, LLM runtime, structured output, and validation;
-- independent extractor packages that own domain-specific behavior;
+- independent process-stage modules that own domain-specific behavior;
 - independent validator packages;
 - embedded prompt and JSON schema assets;
 - CLI orchestration that wires the pieces together without owning domain logic.
@@ -36,8 +36,8 @@ artifacts rather than proposing and applying transcript corrections.
 ## Architectural Principles
 
 - Keep the core input model generic: ordered text units plus metadata.
-- Keep source-format details in hexagonal input adapters.
-- Keep extraction-domain details in extractor packages.
+- Keep source-format details in hexagonal input modules.
+- Keep extraction-domain details in process modules.
 - Treat evidence as source references, not transcript references.
 - Prefer narrow, useful abstractions over a universal document model.
 - Preserve enough provenance for validation, replay, and downstream inspection.
@@ -57,12 +57,13 @@ internal/core/reporting
 internal/core/extractorcatalog
 internal/core/inputcatalog
 
-internal/adapters/input/seriatim
-internal/adapters/input/markdown
-
 internal/framework/contracts
 internal/framework/extraction
 internal/framework/runner
+internal/framework/pipeline
+internal/framework/merge
+internal/framework/normalize
+internal/framework/output
 internal/framework/validators
 internal/framework/llm
 internal/framework/responseschema
@@ -70,10 +71,24 @@ internal/framework/structuredoutput
 internal/framework/promptcontext
 internal/framework/warnings
 
-internal/extractors/dnd/spells
-internal/extractors/dnd/items
-internal/extractors/dnd/npcs
-internal/extractors/dnd/combat
+internal/modules/input/seriatim
+internal/modules/input/markdown
+
+internal/modules/chunk/generic
+internal/modules/chunk/dndtranscript
+
+internal/modules/process/dnd/spells
+internal/modules/process/dnd/items
+internal/modules/process/dnd/npcs
+internal/modules/process/dnd/combat
+
+internal/modules/merge/appendorder
+internal/modules/merge/dnd/spells
+
+internal/modules/normalize/noop
+internal/modules/normalize/dnd/spells
+
+internal/modules/output/json
 
 internal/validators/source_refs
 internal/validators/schema_validity
@@ -85,9 +100,9 @@ examples
 docs/internal
 ```
 
-The `markdown` adapter is listed as a likely future package. The MVP should only
-implement the Seriatim adapter unless a second adapter is needed to test the
-boundary.
+The `markdown` input module and D&D-specific chunk, merge, normalize, and
+output modules are listed as likely future packages. The MVP should implement
+only the stage modules needed by the checkpoint sequence.
 
 ## Core Concepts
 
@@ -123,7 +138,7 @@ Initial source-unit assumptions:
 - adapter-specific metadata may carry speaker, timestamps, heading paths, page
   numbers, or other source details.
 
-### Input Adapter
+### Input Module / Adapter Contract
 
 Hexagonal boundary for external source formats.
 
@@ -134,7 +149,7 @@ type InputAdapter interface {
 }
 ```
 
-The MVP adapter should target Seriatim minimal transcript JSON. Seriatim segment
+The MVP input module should target Seriatim minimal transcript JSON. Seriatim segment
 fields should map as follows:
 
 - `id` becomes `SourceUnit.ID`;
@@ -178,17 +193,44 @@ type Extractor interface {
     ArtifactType() string
     SchemaVersion() string
     Validators() []Validator
-    Extract(ctx context.Context, req ExtractionRequest) (ExtractionResult, error)
+    Process(ctx context.Context, req ProcessRequest) (ProcessResult, error)
 }
 ```
 
-An extractor should receive either a whole source document or a source slice,
-depending on runner configuration. It should return typed artifact candidates
-plus warnings. It should not mutate the source document.
+An extractor should receive either a whole source document or a source chunk,
+depending on processing mode. It should return typed artifact candidates plus
+warnings. It should not mutate the source document.
 
-Extractor packages own domain concepts. For example, D&D spell extraction should
-live under `internal/extractors/dnd/spells`; a future to-do extractor for notes
-should live under a different domain path and use the same framework contract.
+Process modules own domain concepts. For example, D&D spell extraction should
+live under `internal/modules/process/dnd/spells`; a future to-do extractor for
+notes should live under a different process-module path and use the same
+framework contract.
+
+### Chunker
+
+Reusable stage contract for splitting a source document into ordered source
+chunks.
+
+Chunking is a first-class pipeline concern because source documents may exceed a
+single LLM extraction pass. Chunkers should preserve source-unit order and
+produce stable chunk metadata suitable for diagnostics and replay.
+
+### Merger
+
+Reusable stage contract for combining per-chunk artifact candidates into one
+merged candidate collection.
+
+Merge should combine outputs without doing semantic reconciliation. A generic
+append-in-chunk-order merger should be sufficient for many artifact streams,
+including the likely first D&D spell-cast extractor.
+
+### Normalizer
+
+Reusable stage contract for reconciling merged artifact candidates.
+
+Normalize is distinct from merge. Normalizers may deduplicate repeated facts,
+resolve aliases, reconcile conflicting fields, check cross-chunk consistency,
+or attach normalization warnings.
 
 ### Validator
 
@@ -303,27 +345,37 @@ The architecture should support extractors outside the D&D domain. Examples:
 - decisions and action items from meeting transcripts;
 - named people, places, and dates from research notes.
 
-These should be addable as extractor packages without changing runner,
+These should be addable as process modules without changing runner,
 validator, source-reference, or LLM framework contracts.
 
-## Proposed Runner Flow
+## Proposed Pipeline Flow
+
+The application workflow should be first-class:
+
+```text
+input -> chunk -> process -> merge -> normalize -> output
+```
+
+Proposed runner flow:
 
 1. Load effective config.
 2. Create diagnostics run directory.
-3. Resolve the configured input adapter.
+3. Resolve the configured input module through the input adapter registry.
 4. Read source input.
 5. Parse source input into a `SourceDocument`.
 6. Validate source-document invariants.
-7. Chunk source units into deterministic source slices.
-8. Resolve configured extractor instances through a registry.
-9. Execute extractor instances in configured order.
-10. Run deterministic validators before LLM-backed validators.
-11. Retain approved artifacts and rejected-artifact diagnostics.
-12. Merge approved slice artifacts deterministically.
-13. Serialize final output JSON.
-14. Write run manifest, diagnostics, and optional report JSON.
+7. Resolve the configured chunker.
+8. Chunk source units into deterministic source chunks.
+9. Resolve configured extractor instances through a registry.
+10. Process chunks in extractor-defined mode.
+11. Merge per-chunk artifact candidates deterministically.
+12. Normalize merged artifact candidates.
+13. Run deterministic validators before LLM-backed validators.
+14. Retain approved artifacts and rejected-artifact diagnostics.
+15. Serialize final output JSON.
+16. Write run manifest, diagnostics, and optional report JSON.
 
-The runner should operate on source documents and source slices only. Any
+The runner should operate on source documents and source chunks only. Any
 transcript-specific behavior should happen before the runner, inside the input
 adapter, or after the runner, inside output rendering that understands source
 metadata.
@@ -335,6 +387,7 @@ Reuse these architectural patterns:
 - deterministic parsing and schema validation style;
 - deterministic chunking of ordered source units;
 - explicit extractor registry;
+- explicit pipeline stage contracts;
 - `contracts` package for transport-neutral interfaces;
 - OpenAI-compatible structured LLM client;
 - scheduler for bounded LLM concurrency;
@@ -358,17 +411,18 @@ extraction-report concepts.
 
 ## Checkpoint Roadmap
 
-The initial implementation should proceed through five coherent checkpoints.
+The initial implementation should proceed through six coherent checkpoints.
 Each checkpoint should leave the repository in a reviewable state, with the code
 compiling and targeted tests covering the newly introduced contracts or behavior.
 
 1. [Core Contracts And Skeleton](1-core-contracts-and-skeleton.md)
 2. [Framework Composition](2-framework-composition.md)
-3. [Portable Audita Infrastructure](3-portable-audita-infrastructure.md)
-4. [Seriatim Input Adapter](4-seriatim-input-adapter.md)
-5. [D&D Spells Extractor](5-dnd-spells-extractor.md)
+3. [Pipeline Stages, Chunking, Merge, And Normalize](3-pipeline-stages-chunking-merge-normalize.md)
+4. [Portable Audita Infrastructure](4-portable-audita-infrastructure.md)
+5. [Seriatim Input Module](5-seriatim-input-module.md)
+6. [D&D Spells Extractor](6-dnd-spells-extractor.md)
 
-The first useful vertical slice should arrive at checkpoint 5: Seriatim
+The first useful vertical slice should arrive at checkpoint 6: Seriatim
 transcript input to validated D&D spell artifact output. Earlier checkpoints are
 intentionally contract-first and may not produce useful user output yet.
 
@@ -383,10 +437,11 @@ intentionally contract-first and may not produce useful user output yet.
   artifact metadata be allowed without source references?
 - Should overlapping source-reference ranges be merged, preserved exactly, or
   both?
-- Should extraction run independently per source slice only, or should some
+- Should extraction run independently per source chunk only, or should some
   extractors receive whole-document context?
-- Should a later reconciliation stage deduplicate entities and events across
-  source slices?
+- Which artifact types can use a generic append-in-chunk-order merger?
+- Which artifact types need domain-specific normalization for deduplication,
+  identity resolution, or consistency?
 - Should LLM review be part of each extractor's validator chain or a separate
   review phase?
 - Should the Seriatim adapter accept only its minimal schema initially or also