notarius/docs/internal/modules.md

Module Internals

This guide owns the mechanics for implementing and registering production modules. Configuration owns selectable keys, binding syntax, reference configuration, and default validator chains. Durable input and output shapes belong in integration contracts.

The D&D family has additional shared conventions and domain-specific exceptions. See D&D Module Internals rather than adding them here.

Module Boundary

A module is a typed implementation registered for one pipeline stage. Its ModuleSpec is the public-to-the-framework declaration of its stable key, stage, execution class, required and provided capabilities, artifact kind, and accepted reference slots. The execution class states whether a module is deterministic or llm_backed; registries retain it for catalog inspection and resolved-pipeline debug data without constructing the module. The framework uses the declaration to resolve a configured binding before it builds the implementation. After selection, the resolver applies profile inheritance only to bindings whose declared execution class is llm_backed and rejects a binding-specific profile on a deterministic module. The user-facing precedence contract belongs in Configuration.

Implementations that accept options must provide both an option validator and a builder. The validator is used while resolving configuration; the builder decodes the same options and constructs the implementation from the prepared BuildRequest. Reject unknown options in both paths. A builder receives only the dependencies and materialized references that the framework prepared for that operation, so it must not re-read configuration or files.

Registry helpers register the typed builder for a stage-specific registry. They are preferable to hand-written untyped registration because they retain the artifact type at the framework boundary. Registrars validate the registries they need, register each leaf implementation, and add any family-owned assets or default validator chains. They return contextual errors so production composition fails at startup rather than at the first run.

An artifact family can register an optional typed evidence projector alongside its codec. The projector returns defensive copies of the artifact's direct generic source references and must use the codec's exact Go type. It does not interpret surrounding context or publish files; the pipeline validates the capability during preparation and the output boundary owns publication. See the Published Evidence Context contract for the durable source-unit excerpt. Lane artifacts retain citation and lane provenance; the framework does not add either to that published excerpt.

An artifact family is broader than a module: it owns the cohesive domain feature across its artifact type, codec, stage modules, validators, prompt policy, schemas, identity helpers, and reference projections. An extractor and normalizer in one artifact family remain independently registered modules in their respective pipeline stages. This ownership vocabulary does not create a new registry or change the fixed pipeline.

Production Composition

Production composition is intentionally split by family:

• The generic registrar provides the unit chunker, generic JSON validators, JSON output encoder, and shared semantic-reconciliation prompt and response schema assets.
• The Seriatim registrar provides the transcript input adapter. Its external input behavior is defined by the Seriatim contract.
• The D&D registrar provides its codecs, extractors, mergers, normalizers, validators, prompt assets, fallback profile asset, and default chains. Its behavioral conventions are documented in D&D Module Internals.

The CLI owns the composition that invokes these registrars. A module package may register its own family but must not assemble the CLI or make framework packages depend on production extensions.

Semantic Reconciliation

internal/framework/semanticreconcile is a domain-neutral strategy used by a typed normalize module; it is not itself a selectable stage module. A source-backed artifact-family normalizer projects its deterministic records into contextual candidates and owned typed record envelopes, supplies its chosen prompt identity and resolved LLM profile, and constructs an engine with explicit limits. The core filters invalid evidence, assigns contiguous request-local integer handles, renders bounded candidate and transcript materials, invokes the structured-completion boundary, and assesses the returned duplicate groups into a stable non-overlapping plan.

The normalizer then applies that plan through a typed ApplicationPolicy. The core preserves ungrouped records, contribution order, and provenance while the artifact family owns group guards, field and evidence consolidation, durable ID derivation, retry and fallback presentation, warnings, and postconditions. Request-local handles do not enter the typed value or durable artifact. Fewer than two eligible candidates skips model invocation; exceeding a candidate or combined-material bound preserves the deterministic result under the family's fallback policy. Provider, transport, cancellation, and context-construction failures remain execution errors.

The core supplies a conservative generic prompt and the single private response schema. A domain prompt may substitute its semantic instructions but mounts the core-owned protocol and candidate/transcript presentation assets. Prompt, schema, policy, and limit identities participate in manifest metadata and checkpoint fingerprints. The generic registrar owns production registration of those shared assets; a consuming domain registrar owns only its domain prompt.

Adding Or Changing A Module

1. Choose the pipeline stage and the typed artifact boundary. Put external input or durable artifact formats in the relevant integration contract, not in this guide or in a private LLM response type.
2. Define a stable ModuleSpec with an explicit execution class, the exact capabilities, and reference slots needed for the operation. Model a producer/consumer handoff as an artifact-compatible slot; configuration then chooses an external file or a generated binding.
3. Implement strict option decoding, construction, and the typed stage interface. Preserve caller ownership: do not retain mutable request data and return defensive copies where an implementation exposes stored data.
4. Register the module through its typed registry helper and add it to the owning family registrar. Add a default validator chain only when that family owns the behavior; otherwise require an explicit compatible chain.
5. Update the selectable-key and chain reference in Configuration, the applicable integration contract, and focused tests. Keep the configuration document as the sole list of production keys and validator order.

Validation And References

Validators operate on the value produced at their configured stage. A default chain is ordered behavior, not a set: JSON parsing, structural checks, domain-specific checks, durable-schema checks, and advisory checks may have different responsibilities and failure handling. The active default chains and override rules are maintained in Configuration.

Reference slots are part of the module specification. They describe the accepted artifact kind, media type, size, and whether a binding is required; the framework validates those constraints before construction. An external reference is materialized during preparation. A generated reference is a compatible normalized artifact handed from an earlier pipeline step at operation time. The configuration reference rules, including precedence and ordered-handoff requirements, are maintained in Configuration.

Focused Verification

Exercise the leaf implementation and its registration path when changing a module. Registry and registrar tests cover duplicate keys, required registries, and typed construction; pipeline resolution tests cover capabilities, options, and reference compatibility. Domain packages should additionally test their codecs, validators, normalizers, and any integration handoffs they own.

Run the affected package tests while iterating. The complete module suite is:

go test ./internal/modules/...