notarius/docs/internal/modules.md

Module And Validator Internals

Production module and validator implementations live under their domain-first trees in internal/modules. The selectable keys, configuration options, reference slots, and default validator chain are canonical in the module and validator catalogs in Configuration.

Extension Pattern

A stage module package provides a stable key, constructor, contract implementation, ModuleSpec, Register, and focused behavior and registration tests. A validator package follows the same pattern with ValidatorSpec and the validator registry. Package-family registrars compose those leaf registrations into the production catalog and own family-level policy such as default validator chains and prompt asset collection.

Production extract, merge, normalize, and validator packages currently use the explicit legacy raw registration APIs. Typed registration is framework-ready, but no production artifact kind or codec is registered yet.

Specs expose capability and execution metadata without constructing an implementation. Chunk, extract, merge, and normalize modules that accept auxiliary material declare identical reference slots from both ReferenceSlots() and ModuleSpec().ReferenceSlots; registration tests enforce that agreement. Runtime delivery uses the corresponding stage request's References field.

LLM-backed extensions own their prompt definitions and response schemas under package-local embedded assets. Shared filesystem composition belongs in internal/framework/promptfs; reusable D&D prompt fragments, reference declarations, prompt-input assembly, and source-unit helpers belong in internal/modules/dnd/shared. Stage contracts expose only Notarius structured- completion types, not Scriptorium public types.

Reference material may inform a module or prompt but must not become source evidence. The resolver and materializer behavior is described in Pipeline Internals.

Input Adapter

internal/modules/seriatim/input/transcript

The adapter decodes the supported transcript JSON, selects the source identity, computes canonical source provenance, validates segments, and maps each segment into a generic source unit with a self-reference plus speaker and timestamp metadata. Its spec advertises the transcript capabilities consumed by D&D modules.

Parsing is strict about required values and duplicate unit IDs but deliberately ignores unrelated Seriatim fields. The external format and derived-identity rules are defined in the Seriatim contract.

Chunkers

internal/modules/generic/chunk/units

The generic chunker validates the source document, walks units in configured windows, clones each selected unit, and emits deterministic ordered chunk IDs. Overlap changes the next window start but never reorders units. It records the first and last unit and unit count in chunk metadata, and derives the chunk's canonical source reference from those unit references.

The accepted options and defaults are defined in Configuration. Generic framework validation canonicalizes the returned unit slices before extraction.

internal/modules/dnd/chunk/scenes

The scene chunker prepares a structured Scriptorium request from the full transcript, session, and optional D&D reference inputs. It validates the model's scene boundaries against source-unit IDs and converts them into deterministic chunks with canonical source references spanning each scene's units.

Scene validation requires sequential, contiguous, non-overlapping coverage from the first source unit through the last. Each chunk contains JSON scene content and module-owned metadata for the scene description, boundaries, confidence, participants, and unit count. Boundary caveats become warnings. Malformed structured output is returned as an error; there is no fallback chunker.

The package embeds its prompt and response schema and reports their non-secret identity and hashes through singleton module metadata. Shared D&D assets supply reference declarations and prompt inputs; their user-facing keys and accepted file types remain canonical in Configuration.

Extractor

internal/modules/dnd/extract/spells

The spell extractor prepares a structured request from one chunk, the chunk-scoped source input, the session, and optional D&D reference inputs. It decodes the model response, assigns the generic source identity to every source reference, canonicalizes duplicate references, orders spell casts by their earliest cited unit, and returns raw JSON plus response-schema provenance.

The package owns its embedded prompt, response schemas, and prompt/schema manifest metadata. Shared D&D helpers keep prompt input names and source-unit reference conversion consistent with the scene chunker. The extractor produces raw output; production validators own approval policy.

The durable payload and manifest metadata shapes are defined in the D&D spell artifact contract.

Merger And Normalizer

internal/modules/generic/merge/appendorder

The merger preserves extract-result order. It passes through one JSON result, concatenates a common top-level array field across multiple JSON objects, and otherwise emits an array of the decoded values. It rejects invalid JSON and non-JSON media types, and it preserves compatible schema provenance.

internal/modules/generic/normalize/noop

The normalizer defensively clones the accepted merge result, including payload bytes, metadata, warnings, and schema provenance, without changing its logical content.

Output Encoder

internal/modules/generic/output/json

The JSON encoder sorts normalized results by lane, derives collision-checked safe logical names, pretty-prints JSON payloads, and assembles the logical index, manifest, rejected-result, warning, and lane files. Invalid JSON, unsupported media types, unsafe names, and sanitized-name collisions are errors.

The encoder returns logical files only. The CLI places them on disk, and the JSON output contract defines their external paths and schemas.

Generic Validators

The generic validator implementations live under internal/modules/generic/validate.

The unconditional accept and reject validators provide deterministic production registrations used primarily for controlled composition and tests.

The JSON syntax validator uses encoding/json to reject malformed payloads. The JSON Schema validator requires schema bytes on the validation request, parses the instance and schema with jsonschema, and distinguishes payload rejection from schema loading or compilation errors. Neither validator calls the LLM.

D&D Spell Validators

internal/modules/dnd/validate/spells/spellpayload provides strict decoding, shape checks, source-reference candidates, and cited-text lookup shared by the three validators.

The shape validator rejects malformed JSON, unknown fields, missing or empty spell fields, and empty reference lists. The source-reference validator applies generic source-reference validation to every cited range. The relatedness validator approves structurally valid payloads but warns when a case-insensitive spell name is absent from all cited source text. It leaves malformed payloads to the earlier validators in the configured chain.

These validators are deterministic. Their selectable keys and production order are defined in Configuration; their durable payload rules are defined in the artifact contract.

Production Registration

The CLI allocates one complete framework registry set and one LLM asset registry. It invokes internal/modules/generic/register, internal/modules/seriatim/register, and internal/modules/dnd/register in that order, then exposes the matching catalog for resolution. The generic and Seriatim registrars own their production leaf registrations. The D&D registrar owns D&D leaf registrations, the spell default-validator chain, and D&D prompt/schema asset collection.

Framework packages must not import production extensions. Tests may compose registries and catalogs directly with fakes.

Adding An Extension

When adding a production module or validator:

1. implement the stage or validator contract and package-local key;
2. expose and test its spec, constructor, and registration function;
3. keep format or domain parsing inside the concrete package;
4. add package-owned prompt/schema assets when the extension is LLM-backed;
5. register it through its package-family registrar and add a default chain there only when production policy requires one;
6. add resolution and composition coverage for capabilities, options, references, and validation behavior;
7. update the selectable-key catalog in Configuration, the relevant external contract, this inventory, and maintained examples when user-visible behavior changes.

Do not add the extension to docs/development.md; that file routes by task and does not inventory implementations.

Tests To Inspect

• Package-local *_test.go files under the module or validator being changed.
• internal/framework/pipeline/registry_integration_test.go: registry and spec composition.
• internal/framework/pipeline/default_modules_test.go: framework binding defaults.
• internal/cli/run_test.go: production catalog, config resolution, and end-to-end CLI composition.
• internal/framework/promptfs/*_test.go and internal/modules/dnd/shared/*_test.go: shared prompt and reference assembly.
• internal/modules/integration/*_test.go: black-box composition across production extension domains.