Add documentation and roadmap for a significant refactor around domain-focused module packages

docs/adr/0002-linear-pipes-and-filters-pipeline.md

@@ -0,0 +1,51 @@
+# ADR-0002: Linear pipes-and-filters pipeline, not a general DAG
+
+**Status:** Proposed
+**Date:** 2026-07-13
+
+## Context
+
+Notarius processes source material through one known workflow:
+
+```text
+input -> chunk -> extract -> merge -> normalize -> output
+```
+
+Input and chunking apply to the source as a whole. Each selected artifact lane
+then performs extract, merge, and normalize, after which output aggregates the
+lane outcomes. Chunk extraction has a natural scatter-gather shape, but no
+current use case requires arbitrary branches, joins, or user-defined stage
+topology.
+
+## Decision
+
+Notarius implements a fixed six-stage pipes-and-filters pipeline. Configuration
+selects implementations for these stages but cannot add stages, reorder them,
+or define an arbitrary graph.
+
+The framework owns stage sequencing and the scatter-gather boundary between
+chunk, extract, and merge. Extract results are handed to merge in deterministic
+source-chunk order regardless of execution strategy. Each artifact lane remains
+logically linear. Output runs after every selected lane has either produced an
+accepted normalized artifact or reached a recorded rejection. A framework
+execution failure aborts the pipeline.
+
+The runner's concrete internal representation and stage-specific scheduling
+policies are implementation details. Concurrency must preserve the pipeline's
+deterministic handoffs, validation behavior, and provenance, and all execution
+strategies must continue to honor context cancellation.
+
+## Alternatives considered
+
+- Build a general DAG engine now. This would support hypothetical branching
+  topologies, but would add scheduling, topology validation, configuration, and
+  state-management complexity without a current consumer. Revisit this choice
+  only when a concrete workflow requires a topology the fixed pipeline cannot
+  express.
+
+## Consequences
+
+The runner, configuration model, and operator mental model remain small. Stage
+ownership stays visible, and general chunking, merging, or normalization cannot
+be hidden inside extractors. A future DAG requirement will require an explicit
+architectural change rather than incremental exceptions to the fixed pipeline.

docs/adr/0003-typed-interfaces-with-two-zone-data-model.md

@@ -0,0 +1,119 @@
+# ADR-0003: Strongly typed stage interfaces with a two-zone data model
+
+**Status:** Proposed
+**Date:** 2026-07-13
+
+## Context
+
+Pipeline stages must exchange source data and extracted artifacts. Universal
+source data has one engine-wide meaning, while extracted artifacts have
+domain-specific shapes. Passing opaque bytes or `any` between all stages would
+make invalid wiring and merge behavior runtime concerns. Requiring JSON at
+every handoff would preserve interoperability but discard useful Go type safety
+while all modules are in-process.
+
+The framework must also support multiple configured artifact domains, durable
+checkpoints, diagnostics, and output encoders without making those consumers
+depend on every domain's Go types.
+
+## Decision
+
+Notarius uses two typed data zones followed by one serialized boundary.
+
+### Source zone
+
+Input and chunk stages use conservative, engine-owned document, segment, chunk,
+and source-reference types. Their exact Go names are implementation details.
+Every segment carries engine-owned source provenance identifying the source
+location from which it was produced. Chunks preserve the ordered provenance of
+their segments.
+
+Source-format-specific fields remain in input modules or explicitly namespaced
+metadata; they do not become framework contracts.
+
+### Domain artifact zone
+
+Each artifact lane has one domain-owned Go artifact type `T`. Its extract,
+merge, normalize, and domain-aware validation implementations use generic,
+strongly typed contracts over the same `T`. Raw JSON, opaque bytes, and `any`
+are not stage-handoff contracts within a lane.
+
+Each registered domain artifact type supplies a codec for `T`. The codec owns:
+
+- stable schema identity and an explicit schema version;
+- JSON serialization and deserialization;
+- the media type and schema metadata required at serialized boundaries; and
+- rejection of data that cannot be represented by the declared artifact
+  schema.
+
+An artifact type's JSON representation is a maintained domain contract.
+Changing it incompatibly requires a new schema version.
+
+Extract, merge, and normalize may change the contents of `T`, but they do not
+change the lane's canonical Go artifact type or artifact schema identity. An
+extractor maps any provider- or prompt-specific response type into `T` before
+returning. A future lane that requires different artifact types at different
+stages requires a new architectural decision.
+
+### Serialized boundary
+
+After normalization, each typed artifact is converted into an engine-owned
+serialized artifact containing bytes, media type, and schema metadata. Output
+aggregation and output encoders consume this type-erased form. Intermediate
+checkpoint and debug encodings do not become stage-handoff contracts.
+
+LLM transport, checkpoints, and opt-in debug recording are also explicit
+serialization boundaries. They may encode or decode a typed artifact through
+its domain codec, but they do not change the in-memory type used between
+extract, merge, normalize, and typed validators. Checkpoint reuse requires a
+compatible schema identity and version.
+
+An LLM structured-response schema is a module transport contract and may differ
+from the domain artifact schema. The calling module owns the response type and
+maps it into the canonical `T`; the artifact codec remains authoritative for
+artifact checkpoints and output serialization.
+
+The framework may use private type-erased adapters to store heterogeneous lane
+registrations and execute configured domains. Such an adapter must assemble a
+type-consistent lane before execution and must not expose `any` or raw payloads
+as module-facing handoffs inside the domain artifact zone.
+
+### Construction and dependencies
+
+Every module operation accepts `context.Context`. Modules receive stable runtime
+collaborators through an injected dependency set at construction time. In
+particular, LLM-using modules receive the application-provided structured LLM
+client and do not construct provider clients or bypass shared scheduling.
+
+The application boundary enforces one configurable global upper bound on
+in-flight LLM calls across all stages, lanes, retries, and validators.
+
+Configuration options are parsed and validated while a module is constructed,
+before that module executes. Per-run data such as source material, references,
+session identity, and lane identity remains operation input rather than a
+construction dependency.
+
+## Alternatives considered
+
+- Pass raw bytes between stages. This maximizes decoupling but moves wiring,
+  parsing, and merge errors to runtime and prevents domain types from being the
+  canonical in-process contract.
+- Require JSON plus schemas at every stage boundary. This is appropriate for an
+  out-of-process boundary, but adds serialization and parsing inside the current
+  in-process pipeline. The stable codec contract preserves this upgrade path if
+  remote plugins are introduced.
+- Use a uniform `Process(any) (any, error)` contract. This simplifies a fully
+  dynamic engine but turns incompatible module composition into type assertions
+  and runtime failures. The fixed topology does not require that tradeoff.
+
+## Consequences
+
+Domain pipelines gain compile-time handoff safety and explicit merge semantics.
+Serialization, schema compatibility, checkpoint decoding, and output erasure
+have named owners. Dynamic registration requires a small erased adapter around
+each typed lane, and generic stage implementations must be instantiated for a
+specific artifact type or behavior rather than manipulating arbitrary JSON.
+
+The engine-owned source model becomes a long-lived contract and must evolve
+conservatively. Domain authors must maintain a codec and versioned schema in
+addition to their Go artifact type.

docs/adr/0004-package-modules-by-domain.md

@@ -0,0 +1,80 @@
+# ADR-0004: Package modules by domain, not by stage
+
+**Status:** Proposed
+**Date:** 2026-07-13
+
+## Context
+
+Module packages can be grouped first by pipeline stage, such as
+`modules/chunk/dnd/scenes`, or first by domain, such as
+`modules/dnd/chunk/scenes`. A domain's extract, merge, normalize, validation,
+schema, prompt, and artifact-codec implementations collaborate around the same
+artifact types and are likely to evolve together.
+
+Go package dependencies also constrain registration. If shared types live in a
+domain root package, that package cannot import child implementation packages
+to register them because the children already import the root types.
+
+## Decision
+
+Production extensions are grouped by domain under:
+
+```text
+internal/modules/<domain>/<stage>/<name>
+```
+
+Shared artifact types live at the domain root, for example
+`internal/modules/dnd/types.go`. Domain-specific validators, prompt fragments,
+schemas, reference helpers, and codecs also live within that domain tree.
+
+Each domain exposes one production registration entry point from a sibling
+registrar package, for example `internal/modules/dnd/register`. The registrar
+may import the domain root and its child implementations; the domain root does
+not import its registrar or child packages. This keeps shared types available
+as `dnd.SpellList` without creating a Go import cycle.
+
+The `generic` tree is a peer extension family for reusable implementations that
+contain no concrete source-format or artifact-domain knowledge. Source-format
+and output-format families, such as Seriatim and JSON output, follow the same
+domain-first organization even when they do not define a Zone-B artifact type.
+
+Concrete domain implementation packages do not import another concrete domain.
+Generic extension packages never import concrete domains. A domain registrar
+may import domain-neutral generic extension packages to instantiate a reusable
+strategy for that domain's artifact type; the generic implementation remains
+unaware of the concrete type's domain semantics. Reuse needed directly by a
+domain implementation lives in a domain-neutral framework or helper package,
+not in a peer extension package.
+
+The application composition root may import multiple registrar packages, and
+black-box integration tests may compose multiple domains. Other cross-domain
+reuse occurs through engine contracts and composition-time registration rather
+than concrete peer-domain imports.
+
+A domain registrar owns registration of that domain's modules, validators,
+default validator chains, artifact codecs, schemas, and prompt assets. It does
+not take ownership of application execution or process behavior.
+
+## Alternatives considered
+
+- Group modules by stage. This keeps interchangeable strategies side by side,
+  but scatters a domain's shared artifact model and collaborating extensions
+  across the repository. It is preferable when generic strategy libraries
+  dominate or when the project is primarily a stage-extension framework rather
+  than an application composed from domain suites.
+- Put both shared types and `Register` in the domain root. This gives the
+  shortest import path but creates an import cycle once child implementations
+  import the root artifact types.
+
+## Consequences
+
+The repository layout makes supported domains immediately visible, and adding
+or extracting a domain affects one cohesive subtree. The CLI composition root
+depends on a small set of domain registrars instead of every leaf package.
+
+Package moves must preserve user-visible module and validator keys unless a
+separate compatibility decision changes them. Shared behavior that cannot be
+expressed through framework contracts may need to move into a domain-neutral
+framework package rather than creating a concrete peer-domain import. Registrar
+packages become explicit composition points for instantiating generic typed
+strategies, in addition to registering domain-owned implementations.