Files
notarius/docs/internal/modules.md

631 lines
35 KiB
Markdown

# Module And Validator Internals
Production module and validator implementations live under their domain-first
trees in `internal/modules`.
The selectable keys, configuration options, reference slots, and default
validator chain are canonical in the
[module](../config.md#implemented-production-modules) and
[validator](../config.md#implemented-production-validators) catalogs in
Configuration.
## Extension Pattern
A stage module package provides a stable key, constructor, contract
implementation, `ModuleSpec`, `Register`, and focused behavior and registration
tests. A validator package follows the same pattern with `ValidatorSpec` and the
validator registry. Package-family registrars compose those leaf registrations
into the production catalog and own family-level policy such as default
validator chains and prompt asset collection.
Production input, chunk, output, and D&D spell-, NPC-, combat-, interaction-, and scene-description-extract packages
register strict option decoders and run-local builders. Preparation decodes their options into
implementation-owned values and injects dependencies plus the materialized
reference set for the selected target. Each builder receives an isolated clone
of that set; input and output builders receive no references. The spell, NPC,
combat, interaction, and scene-description extractors are typed over the canonical D&D model. D&D validators, merge,
and normalize use typed variants; JSON representation validators use serialized
requests; and unconditional validators expose separate chunk and typed
variants. The D&D production registrar registers the canonical typed spell,
NPC, combat, interaction, and scene-description implementations, including their kind-specific merge and
normalize behavior.
For D&D artifact defaults, generic JSON syntax validation runs first. Rejecting
domain validators then own semantic diagnostics before generic JSON Schema
validation provides the final rejecting representation backstop; warning-only
relatedness validators run last. This default composition does not reorder an
explicitly configured validator chain.
Prepared extractors, extract validators, and codecs may be reused concurrently
by the run-wide extract pool. Production implementations are immutable after
construction: they retain only typed options, immutable assets, or the shared
concurrency-safe LLM client. Implementations that introduce mutable state must
synchronize that state without creating a separate provider scheduler.
Specs expose capability and execution metadata without constructing an
implementation. Registry entries separately expose option validation and
run-local construction. Chunk, extract, merge, and normalize modules that accept
auxiliary material declare identical reference slots from both
`ReferenceSlots()` and `ModuleSpec().ReferenceSlots`; registration tests enforce
that agreement. Runtime delivery uses the corresponding stage request's
`References` field.
LLM-backed extensions own their prompt definitions and response schemas under
package-local embedded assets. Shared filesystem composition belongs in
`internal/framework/promptfs`; reusable D&D prompt fragments, reference
declarations, prompt-input assembly, and source-unit/citation helpers belong in
`internal/modules/dnd/shared`, which owns operation-scoped indexed
source-reference validation, citation traversal, ordering and canonicalization,
plus bounded D&D diagnostics. The
D&D scene chunker and spell, NPC, combat-turn, NPC-interaction, and scene-description extractors use ordered
package-local prompt manifests for both rendering and prompt fingerprinting, so
only the shared fragments each prompt actually renders participate in either
operation. Extraction prompts place stable shared and lane-specific context
before the variable transcript and use shared assets for wording common across
lanes. The canonical ordering and cache-boundary policy is documented in
[LLM Runtime](llm.md#dd-extraction-prompt-ordering-and-cache-boundaries). Stage
contracts expose only Notarius structured-completion types, not Scriptorium
public types.
The shared `PrepareChunkExtraction` helper owns common extraction preflight and
transcript material preparation for the spell, NPC, combat-turn,
NPC-interaction, and scene-description extractors. It validates common request
state, clones supplied source metadata, falls back to the materialized chunk
when content is absent, checks that content remains chunk-identical, and fills
only the common default fields. Extractors retain receiver, dependency, and
lane-specific checks locally and wrap helper errors with their module context.
Reference material may inform a module or prompt but must not become source
evidence. The resolver and materializer behavior is described in
[Pipeline Internals](pipeline.md#reference-materialization).
## Domain Reference Data
### `internal/modules/dnd/spells/catalog`
The spell catalog package owns the embedded, versioned D&D 5e 2014 SRD spell
reference data. Its strict JSON asset contains one canonical record per spell,
including spell level and all applicable class memberships. `LoadSRD5E2014`
validates catalog identity, provenance metadata, ordering, uniqueness, levels,
classes, aliases, and lookup-key collisions before exposing immutable copies.
Lookup is case-insensitive and normalizes whitespace and common apostrophe
variants while preserving source punctuation in canonical display names. The
catalog contains 319 unique spells and 779 class memberships. Source and
license details live beside the asset in `SOURCES.md`. This domain-owned data is
separate from `internal/modules/dnd/shared`, which is reserved for reusable
prompt and source-reference machinery.
`ResolveEffectiveCatalog` builds the immutable recognition view used by the
spell extractor and catalog validator. It starts with the embedded SRD catalog
and optionally applies one strict JSON overlay from the `spell_catalog` item in
a materialized reference set. Overlay catalogs are ordered by ID, may add names
and aliases, and may augment an existing canonical spell without replacing its
display name. Cross-spell lookup collisions are errors. The effective view
exposes sorted canonical names, normalized lookup, overlay identities, and a
semantic digest; overlay content remains contextual reference material rather
than source evidence. Its external JSON contract is defined in the
[spell-catalog overlay contract](../integrations/dnd-spell-catalog-overlays.md).
### `internal/modules/dnd/npcs/identity`, `internal/modules/dnd/npcs/registry`, and `internal/modules/dnd/codec/npcs`
The NPC identity package owns Unicode comparison keys, deterministic
`npc:sha256:` IDs, display normalization, and whole-registry collision issues.
The registry package resolves one optional normalized artifact through the
strict codec, validates whole-registry identity, canonicalizes its JSON, and
provides immutable records, a names-only prompt projection, distinct durable
and projection digests, count, and exact canonical-name lookup. External files cross this boundary during
preparation; generated artifacts cross it at the ordered step handoff. It owns
the `npcs` slot and its bounded, content-safe validation failures. NPC source
references are durable provenance and are not treated as evidence for a
consuming pipeline. The NPC codec owns the strict durable `dnd/npc-list` JSON
boundary and exposes candidate versus approved encode/decode operations. The
shared `internal/modules/dnd/codec/candidatejson` package supplies strict typed
candidate JSON mechanics; each artifact codec retains its own durable schema
and approved-value policy.
The `internal/modules/dnd/codec/combatturns` package owns the durable
`dnd/combat-turn-list` schema and candidate versus approved JSON boundary. It
is registered by the production D&D family registrar for the selectable combat
lane.
The `internal/modules/dnd/codec/npcinteractions` package owns the durable
`dnd/npc-interaction-list` schema and candidate versus approved JSON boundary.
It is registered by the production D&D family registrar for the selectable
interaction lane. Its external contract is documented in the
[D&D NPC interaction artifact contract](../integrations/dnd-npc-interaction-artifacts.md).
The `internal/modules/dnd/codec/scenedescriptions` package owns the durable
`dnd/scene-description-list` schema and candidate versus approved JSON boundary.
It is registered by the production D&D family registrar. Its external contract
is documented in the
[D&D scene-description artifact contract](../integrations/dnd-scene-description-artifacts.md).
## Input Adapter
### `internal/modules/seriatim/input/transcript`
The adapter decodes the supported transcript JSON, selects the source identity,
computes canonical source provenance, validates segments, and maps each segment
into a generic source unit with a self-reference plus speaker and timestamp
metadata. It accepts no module options. Its spec advertises the transcript
capabilities consumed by D&D modules.
Parsing is strict about required values and duplicate unit IDs but deliberately
ignores unrelated Seriatim fields. The external format and derived-identity
rules are defined in the
[Seriatim contract](../integrations/seriatim.md).
## Chunkers
Chunkers implement `contracts.Chunker.Plan`. A plan identifies ordered source
unit ranges and may carry optional namespaced JSON annotations; it does not
contain materialized chunk content. The framework canonicalizes annotations,
validates ranges against the current source, and materializes chunk IDs,
indexes, references, content, units, and generic metadata. Materialized source
unit metadata is independently owned. Annotation
namespaces remain optional data: generic framework code and downstream modules
must not require D&D scene annotations or import `dnd/scenes`.
### `internal/modules/generic/chunk/units`
The generic chunker validates the source document and returns ranges over units
in configured windows. Overlap changes the next window start but never reorders
units. Framework materialization derives the resulting chunk identity and
generic metadata from those ranges.
The accepted options and defaults are defined in
[Configuration](../config.md#implemented-production-modules). Generic
framework validation canonicalizes the returned unit slices before extraction.
The chunker decodes its options during construction and retains only the typed
window settings used by `Plan`.
### `internal/modules/dnd/chunk/scenes`
The scene chunker prepares a structured Scriptorium request from the full
transcript, session, and optional D&D reference inputs. It validates the model's
inclusive source-unit endpoints against document position and converts them
into deterministic plan ranges. Preparation injects the shared structured LLM
client into the chunker; `Plan` supplies only the run-specific profile, session,
source, references, and metadata.
Scene validation requires sequential, contiguous, non-overlapping coverage from
the first source unit through the last. Its private response contains only the
boundary endpoints; the accepted plan has no D&D-specific annotations and
produces no boundary warnings. Malformed structured output is returned as an
error; there is no fallback chunker.
The package embeds its prompt and response schema and reports their non-secret
identity and hashes through singleton module metadata. Shared D&D assets supply
reference declarations and prompt inputs; their user-facing keys and accepted
file types remain canonical in [Configuration](../config.md).
## Extractor
### `internal/modules/dnd/extract/spells`
The spell extractor prepares a structured request from one chunk, the
chunk-scoped source input, the session, and optional D&D reference inputs. It
decodes the model response, assigns the generic source identity to every source
reference, canonicalizes duplicate references, orders spell casts by their
earliest valid source-document position, and returns `dnd.SpellList`.
Its private response schema admits only the structural transport envelope:
required fields, JSON types, array and object shapes, and unknown-field
rejection. It maps integer source-unit candidates directly without repairing
semantic values, so the deterministic shape, catalog, and source-reference
validators own blank values, empty evidence, and invalid or unresolved ranges.
The extractor owns its private model-response DTO, embedded prompt, LLM response
schema, strict option decoder, injected shared LLM client, and prompt/schema
manifest metadata. During preparation it resolves the optional `spell_catalog`
reference into an immutable effective catalog and adds a generated
canonical-name-only JSON input to every structured completion request. Overlay
failures therefore stop construction before source parsing or an LLM call;
campaign references remain separate disambiguation inputs and never become
source evidence.
The prompt includes only actual casting events and unambiguous declared casting
attempts. Spell mentions, plans, rules discussion, and catalog matches without
a casting event are excluded. Shared extraction-evidence and identity rules
require transcript-supported caster and spell facts, while the catalog,
campaign references, and NPC names only disambiguate source text. Structural
source validation remains deterministic; semantic evidence sufficiency is
enforced through extraction policy and evaluation.
Both the extractor and deterministic catalog validator expose
the effective base-plus-overlay semantic digest as scoped prepared-component
checkpoint identity. Raw overlay provenance independently covers file-byte
changes, while the semantic digest also invalidates reuse when the embedded
catalog or catalog composition changes. The extractor additionally fingerprints
its complete prompt assets and private response schema, so either semantic
contract changing invalidates previously recorded extraction checkpoints. The
separate `internal/modules/dnd/codec/spells` package
owns the durable schema and stable JSON representation for artifact kind
`dnd/spell-list`. The runner keeps the result typed through validators and later
stages, using the codec only for checkpoint, debug, and output boundaries.
Shared D&D helpers keep prompt input names and source-unit reference conversion
consistent with the scene chunker.
The extractor also declares the optional `npcs` registry slot and consumes the
immutable registry boundary from `internal/modules/dnd/npcs/registry`. An
external registry is prepared before execution; a generated registry is
validated and supplied at operation time. Bound external registries add only
the full `npc_registry_digest` and `npc_count` to module metadata. The local
`npc_registry` checkpoint fingerprint always covers the names-only projection,
including its exact unbound value. Generated bindings are represented by
framework handoff provenance and dependency fingerprints. The unbound prompt
input is exactly `{"npcs":[]}` and has no registry provenance.
The shared NPC grounding fragment is placed immediately after the common
campaign reference message and is included in the spell prompt fingerprint.
The durable payload and manifest metadata shapes are defined in the
[D&D spell artifact contract](../integrations/dnd-spell-artifacts.md).
### `internal/modules/dnd/extract/npcs`
The NPC extractor maps private model output to the canonical `dnd.NPCList`,
assigns source identity and deterministic NPC IDs, and preserves source
references for deterministic validation. It uses the shared campaign
references only for disambiguation and does not consume the optional NPC
registry slot. Its prompt and private response schema are package-owned. The
private response contains only a name and model-facing evidence ranges for each
record; anonymous groups, generic roles, invented labels, descriptions,
aliases, and relationships are outside its contract. The
prompt follows the shared D&D extraction ordering and cache policy documented
in [LLM Runtime](llm.md#dd-extraction-prompt-ordering-and-cache-boundaries).
The private response schema owns only structural transport validation and maps
integer source-unit candidates unchanged. Required semantic content, non-empty
evidence, and valid source ranges are rejected by the deterministic shape and
source-reference validators.
### `internal/modules/dnd/extract/scenedescriptions`
The scene-description extractor makes one structured completion for each
accepted chunk and maps its private `kind`, `title`, and `summary` response to
one `dnd.SceneDescription`. It assigns the current chunk ID and exact range,
preserves kind without repair, and trims only title and summary whitespace.
Optional players, party, and glossary references can disambiguate prompt terms
but do not supply evidence. The package owns its private schema, prompt assets,
and mapping fingerprint; deterministic validators own the durable semantic
checks. The durable contract is defined in the
[D&D scene-description artifact contract](../integrations/dnd-scene-description-artifacts.md).
### `internal/modules/dnd/extract/combatturns`
The combat extractor prepares one structured request per supplied chunk using
the shared extraction-evidence, identity, campaign-reference, NPC-grounding,
and transcript prompt inputs. It
maps the private response to `dnd.CombatTurnList`, assigns the current source
identity, removes exact duplicate source ranges, and orders turns by valid
source-document position while preserving malformed candidate fields for
deterministic validators. Its package-owned private response schema enforces
only the structural JSON envelope; semantic artifact constraints remain with
the validator chain. Its prepared metadata and checkpoint fingerprints contain
only prompt/schema/mapping identities plus an optional NPC registry digest.
The prompt follows the shared D&D extraction ordering and cache policy
documented in
[LLM Runtime](llm.md#dd-extraction-prompt-ordering-and-cache-boundaries). The
package exposes typed registration and is included in the production D&D
registrar with the default combat extraction chain.
The combat normalizer accepts only the optional structured NPC registry.
Campaign references remain extractor-only LLM context and are not materialized
for deterministic normalization.
### `internal/modules/dnd/extract/npcinteractions`
The NPC interaction extractor requires the structured `npcs` registry slot. It
uses the registry's names-only prompt projection with shared extraction
evidence, identity, and transcript material, then maps private model records to
`dnd.NPCInteractionList` with the current source identity. Registry source
references are never reused as interaction evidence. The private response
schema carries only name, bounded interaction kind, and source-unit ranges;
deterministic validators own registry membership, source validity, and
relatedness. Extract-stage source validation additionally requires every cited
range to be wholly contained in the current materialized chunk. Prompt, schema,
mapping, and the names-only registry projection
participate in checkpoint identity, while generated producer identity remains
framework provenance.
The shared D&D source-reference order defines canonical evidence ordering. The
domain-owned `internal/modules/dnd/npcinteractions` package defines occurrence
ordering, valid-evidence eligibility, and collision-safe exact identity. The
interaction normalizer and normalized invariants validator consume those
rules, so their production and checking paths cannot drift. Normalizer and
relatedness warning lists use the shared D&D diagnostic cap and emit a final
omission-summary warning when truncated.
### `internal/modules/dnd/normalize/npcs`
The NPC normalizer performs deterministic identity-aware consolidation in
merged input order. It consolidates only equal canonical-name comparison keys,
retains the first display record, and unions exact source references. It exposes
the identity policy as its local checkpoint fingerprint and emits bounded
normalization warnings.
## Merger And Normalizer
### `internal/modules/generic/merge/appendorder`
The merger passes typed values to an injected combine function in framework
source-chunk order. The D&D registrar specializes it for all five artifact
lists; each append merger preserves collection presence and order while giving
the result independently owned nested source-reference slices.
### `internal/modules/generic/normalize/noop`
The normalizer returns the merged domain value unchanged and is reusable for
any registered artifact type.
### `internal/modules/dnd/normalize/spells`
The typed spell normalizer resolves the optional `spell_catalog` reference into
the same immutable SRD-plus-overlay effective catalog used by spell extraction
and catalog validation. It performs no LLM calls. For each spell cast it
canonicalizes recognized names using the catalog's case, whitespace,
apostrophe, and alias rules; canonicalizes source references with the shared
document-aware order; removes only exact reference duplicates; and emits
bounded, scoped warnings for each mutation or unresolved name.
After those per-cast changes, it collapses only casts with the same canonical
spell, case-folded and whitespace-normalized caster, and complete non-empty
valid source-reference set. It retains the first occurrence and its caster,
source references, and stable order. Unknown names, empty or invalid evidence,
and adjacent or overlapping but different ranges remain unchanged for
validation.
The normalizer exposes the effective catalog digest as its independently scoped
`effective_catalog` checkpoint fingerprint and reports catalog base ID, digest,
and overlay IDs as manifest metadata. Catalog contents, reference paths, and
raw overlay bytes are not included in either surface. The normalize-stage
reference is stage-local, so an overlay-capable pipeline binds the catalog
independently for extraction and normalization.
### `internal/modules/dnd/normalize/combatturns`
The combat normalizer prepares an external NPC registry before execution or
receives a generated registry at the ordered step handoff, then uses the
immutable view during runtime. It display-normalizes actors,
rewrites canonical-name matches for actors, orders and deduplicates source
references, stable-sorts records by source-document position, and collapses
only exact duplicate identities with fully valid evidence. It deep-clones
output storage and emits bounded warnings scoped to merged input indexes. Its
metadata and fingerprints identify the normalization and NPC identity policies.
External bindings may contribute registry
digest/count metadata; generated identity is retained in framework provenance
and dependency fingerprints. The normalizer is included in the production D&D
registrar with the default combat normalization chain.
### `internal/modules/dnd/normalize/npcinteractions`
The interaction normalizer requires the same immutable NPC registry. It
canonicalizes exact registry-name matches, orders and de-duplicates source
references, stable-sorts occurrences by source-document position, and collapses
only exact interaction identities with valid evidence. It does not infer,
merge, or summarize distinct occurrences. Its metadata and fingerprints expose
the normalization and NPC identity policies; generated registry identity stays
in framework provenance and checkpoint dependencies.
### `internal/modules/dnd/normalize/scenedescriptions`
The scene-description normalizer has no options or references. It validates
each source range against the source document, trims title and summary
whitespace, orders records by source position then ID, removes only exactly
identical records, and rejects conflicting reused IDs or ranges. Its policy
fingerprint identifies this deterministic behavior; the matching invariant
validator checks the normalized result in the production chain.
## Output Encoder
### `internal/modules/generic/output/json`
The JSON encoder sorts normalized results by lane, derives collision-checked
safe logical names, pretty-prints JSON payloads, and assembles the logical index,
manifest, rejected-result, warning, and lane files. Invalid JSON, unsupported
media types, unsafe names, and sanitized-name collisions are errors.
Its strict `include_chunk_map` option is disabled by default. When enabled, it
validates the framework-supplied accepted chunk map through its codec and adds
the pipeline-wide `chunk-map.json` plus its index descriptor; it does not treat
the map as a lane payload. The external shape is owned by the
[Accepted Chunk Map contract](../integrations/chunk-map.md).
The encoder returns logical files only. The CLI places them on disk, and the
[JSON output contract](../integrations/json-output.md) defines their external
paths and schemas.
## Generic Validators
The generic validator implementations live under
`internal/modules/generic/validate`.
The unconditional accept and reject validators provide explicit chunk and
typed-artifact variants used primarily for controlled composition and tests.
The serialized JSON syntax validator uses `encoding/json` to reject malformed
representation bytes. The serialized JSON Schema validator requires schema
bytes, parses the instance and schema with `jsonschema`, and distinguishes
payload rejection from schema loading or compilation errors. The framework
serialized-validation request carries either canonical chunk bytes or artifact
codec bytes according to its target context. Neither validator calls the LLM.
## D&D Spell Validators
All four validators receive `dnd.SpellList` directly. The shape validator
rejects a missing list, blank caster or spell names, and empty reference lists.
The catalog validator defers when shape is invalid, then checks every non-empty
spell name against the immutable effective SRD and overlay catalog. It accepts
normalized canonical names and aliases without rewriting the artifact; unknown
names reject the complete result with bounded, stable index/name diagnostics. The
source-reference validator defers malformed shapes, validates every cited
range, and reports all range defects through a bounded aggregate while
preserving `invalid_source_refs`. The relatedness validator resolves all cited
ranges through the shared document-order traversal, then warns when a normalized
consecutive spell-name token sequence is absent from the cited source text.
Invalid shape
or cited ranges produce no relatedness warnings; the shape and source-reference
validators own those defects.
These validators are deterministic. Shape, source-reference, and relatedness
each expose a local semantic `policy` checkpoint fingerprint. The catalog
validator instead exposes its effective catalog digest as its semantic
checkpoint identity and does not add a separate policy fingerprint. Their
selectable keys and production order are defined in
[Configuration](../config.md#implemented-production-validators); their durable
payload rules are defined in the
[artifact contract](../integrations/dnd-spell-artifacts.md).
## D&D NPC Validators
NPC shape validation checks the required ID and name strings, list presence, and source-reference
shape. The source-reference validator defers malformed shapes, checks
current-document identity, unit existence, and range ordering, and reports all
defects through bounded aggregates. Source relatedness uses the shared
document-order traversal and normalized consecutive-token matching, emitting at
most one bounded warning per record when the canonical name does not occur near
its cited text. Invalid shape or cited ranges produce no relatedness warnings.
Normalize identity validation checks deterministic IDs, canonical names, and
duplicate canonical-name or ID ownership.
All are deterministic and expose the policy fingerprints used by the
production chains.
## D&D Combat Validators
Combat shape validation owns the required list, actor, supported turn kind, and
non-empty source-reference collection. Combat source-reference validation defers invalid
shape, checks source identity, unit existence, and range order, and reports all
defects through bounded aggregates. Combat source-relatedness defers invalid
shape or ranges, uses the shared traversal to combine overlapping cited units
in document order, and emits at most one bounded advisory warning per turn for
an unrelated actor. Actors use normalized consecutive-token matching. The
normalized-invariants validator owns actor display normalization, canonical
source-reference order, chronology, and exact duplicate identity; it defers
shape and source-reference failures. All four validators are deterministic and
expose local policy fingerprints. In the registered defaults, JSON syntax runs
first; combat shape, normalized invariants when applicable, and source-reference
validation precede JSON Schema validation; warning-only relatedness runs last.
## D&D NPC Interaction Validators
Interaction shape validation owns the required list, registry name, supported
kind, and non-empty source-reference collection. Registry validation checks
exact membership in the required immutable NPC registry. Source-reference and
relatedness validation use the current transcript only; malformed candidates
are deferred by later validators and produce no relatedness warning. The
normalized-invariants validator owns canonical registry names, source-reference
order, chronology, and exact duplicate identity. The production chains run
shape, registry, and source-reference checks before JSON Schema validation;
relatedness remains warning-only and last.
## D&D Scene Description Validators
Scene-description shape validation owns the non-empty list, trimmed ID and
prose, closed kind, and basic source-reference shape. Extract-stage source
validation additionally requires the one record to attach exactly to the
current accepted chunk; later source validation checks source membership.
Relatedness checks the title and summary independently against only their cited
transcript range and emits bounded advisory warnings. The normalized-invariants
validator owns ordering, exact duplicate elimination, and conflicting ID or
range detection. The production chains run shape and source-reference checks
before JSON Schema validation; the warning-only relatedness check is last.
## Production Registration
Production composition occurs through family registrars. The CLI allocates one
complete framework registry set and one LLM asset registry. It invokes
`internal/modules/generic/register`,
`internal/modules/seriatim/register`, and `internal/modules/dnd/register` in
that order, then exposes the matching catalog for resolution. The generic and
Seriatim registrars own their production leaf registrations. The D&D registrar
owns D&D leaf registrations, typed spell, NPC, combat, interaction, and scene-description default-validator
chains, typed append-order specializations, and D&D prompt/schema asset
collection. Its registration helpers group module, validator, prompt-asset, and
chain composition while retaining artifact-specific merge and clone behavior in
the registrar.
Concrete implementation packages do not import generic implementation
packages directly. A concrete family's `register` package is its composition
point for specializing reusable generic implementations, while the generic
registrar composes only generic children.
Core and framework production packages do not import production extensions.
CLI production code is the sole application composition root for extensions
and imports only exact family registrar packages. Other production packages,
including commands and newly introduced package trees, do not import module
packages directly. Compatibility tests in the CLI, core, and framework trees
may import roots and implementation leaves directly. Other non-module tests do
not receive that exemption. White-box tests within module families retain the
production family boundaries. `internal/modules/integration` is test
infrastructure: its black-box tests may compose multiple families, but it is
not a production module family or production dependency target.
## Adding An Extension
When adding a production module or validator:
1. implement the stage or validator contract and package-local key;
2. expose and test its spec, constructor, and registration function;
3. keep format or domain parsing inside the concrete package;
4. add package-owned prompt/schema assets when the extension is LLM-backed;
new LLM-backed D&D extraction modules must follow the stable-to-variable
prompt ordering, shared-asset ownership, and cache-boundary policy in
[LLM Runtime](llm.md#dd-extraction-prompt-ordering-and-cache-boundaries), or
document the implemented exception and its evidence there;
5. register it through its package-family registrar and add a default chain
there only when production policy requires one;
6. add resolution and composition coverage for capabilities, options,
references, and validation behavior;
7. update the selectable-key catalog in [Configuration](../config.md), the
relevant external contract, this inventory, and maintained examples when
user-visible behavior changes.
Do not add the extension to `docs/development.md`; that file routes by task and
does not inventory implementations.
### D&D Extractor Contract
New D&D extractors preserve these package-owned responsibilities:
- Reject unknown options unless an option namespace is intentionally
extensible, and use shared common preflight while retaining receiver,
dependency, and lane-specific checks locally.
- Return independently owned results and exposed metadata that callers may
safely mutate.
- Keep the private response DTO, structural response schema and its identity,
provider-response mapping, durable artifact conversion, and lane diagnostics
in the owning package.
- Include every stable semantic input that can change durable output in
checkpoint identity. Consider prompt, schema, mapping, canonicalization,
prepared reference projections, identity, normalization, and trimming where
applicable.
- Add focused behavioral coverage where the lane's risks warrant it, including
construction and registration, option rejection, preflight, provider
failures, structured decoding, mapping and ownership, prompt
role/input/cache order, and checkpoint invalidation.
Prompt ordering, shared-asset ownership, cache boundaries, and private-schema
rules are defined in [LLM Runtime](llm.md#dd-extraction-prompt-ordering-and-cache-boundaries).
[Pipeline Internals](pipeline.md#reference-materialization) owns reference
materialization, and its [checkpoint hooks](pipeline.md#checkpoint-and-debug-hooks)
define checkpoint behavior. Follow [Architecture](../policy/architecture.md#source-and-domain-boundaries)
for ownership boundaries and the [Testing Policy](../policy/testing.md) when
selecting durable coverage. This contract intentionally does not prescribe
prompt prose or length, hashes, test counts, filenames, fixture layouts, or
generic implementation builders.
## Tests To Inspect
- Package-local `*_test.go` files under the module or validator being changed.
- `internal/framework/pipeline/typed_resolution_test.go`: typed registry, spec,
and heterogeneous artifact composition.
- `internal/framework/pipeline/profile_test.go`: framework binding defaults and
profile resolution.
- `internal/cli/production_contract_test.go`: production catalog, config
resolution, and composition smoke coverage.
- `internal/cli/example_contract_test.go`: maintained example ownership.
- `internal/framework/promptfs/*_test.go` and
`internal/modules/dnd/shared/*_test.go`: shared prompt and reference assembly.
- `internal/modules/integration/*_test.go`: black-box composition across
production extension domains.