171 lines
4.7 KiB
Markdown
171 lines
4.7 KiB
Markdown
# Modules
|
|
|
|
Production modules live under `internal/modules`. Each module implements one
|
|
contract from `internal/framework/contracts`, exposes a `ModuleSpec`, and
|
|
registers itself with the matching pipeline registry.
|
|
|
|
The CLI production catalog currently registers only the modules listed here.
|
|
|
|
## Contract Pattern
|
|
|
|
A production module package should provide:
|
|
|
|
- a stable module key;
|
|
- a constructor such as `New`;
|
|
- the relevant contract implementation;
|
|
- `ModuleSpec`;
|
|
- `Register`;
|
|
- focused tests for registration, options, contract behavior, and errors.
|
|
|
|
Module specs should describe capabilities accurately. Resolution uses specs to
|
|
reject incompatible pipelines before execution.
|
|
|
|
Chunk modules receive the structured LLM client through `contracts.ChunkRequest`
|
|
when they need model-backed chunking. The pipeline runner validates generic
|
|
chunk result invariants before extraction; module-owned policies may be stricter
|
|
but must stay within the module package.
|
|
|
|
## `seriatim` Input
|
|
|
|
Package: `internal/modules/input/seriatim`
|
|
|
|
The `seriatim` adapter parses Seriatim transcript JSON into a generic source
|
|
document. It owns transcript JSON details, source ID selection, source digest
|
|
creation, transcript segment validation, and segment metadata mapping.
|
|
|
|
Provides:
|
|
|
|
- `source.transcript`
|
|
- `transcript.speaker`
|
|
- `transcript.timestamps`
|
|
|
|
External JSON shape belongs in the Seriatim integration doc.
|
|
|
|
## `generic` Chunker
|
|
|
|
Package: `internal/modules/chunk/generic`
|
|
|
|
The `generic` chunker splits source units into ordered chunks. It validates the
|
|
source document, clones source units, assigns chunk IDs such as `chunk-000001`,
|
|
and records chunk metadata for start unit, end unit, and unit count.
|
|
|
|
Options:
|
|
|
|
- `max_units`: positive integer, default `50`;
|
|
- `overlap_units`: non-negative integer, default `0`, and less than
|
|
`max_units`.
|
|
|
|
Provides:
|
|
|
|
- `chunks`
|
|
|
|
## `dnd/spells` Extractor
|
|
|
|
Package: `internal/modules/extract/dnd/spells`
|
|
|
|
The `dnd/spells` extractor owns D&D spell-cast artifact semantics. It renders
|
|
embedded prompts, loads the embedded structured response schema, calls the
|
|
structured LLM client, converts spell-cast responses into artifact candidates,
|
|
and supplies deterministic validators.
|
|
|
|
Requires:
|
|
|
|
- `chunks`
|
|
- `source.transcript`
|
|
|
|
Provides:
|
|
|
|
- `dnd.spell_casts`
|
|
|
|
Artifact type and schema version:
|
|
|
|
- artifact type: `dnd.spell_cast`
|
|
- schema version: `v1`
|
|
|
|
The extractor adds prompt and response-schema provenance to lane manifest
|
|
metadata. Durable artifact payload details belong in the
|
|
[D&D spell artifact contract](../integrations/dnd-spell-artifacts.md).
|
|
|
|
## D&D Spell Validators
|
|
|
|
The spell extractor returns two built-in validators:
|
|
|
|
- `dnd/spells/shape`: rejects malformed payloads and missing required fields.
|
|
- `dnd/spells/source_refs`: rejects candidates without valid source references.
|
|
|
|
Reason codes include:
|
|
|
|
- `invalid_payload`
|
|
- `missing_required_field`
|
|
- `missing_source_ref`
|
|
- `invalid_source_ref`
|
|
|
|
These validators are supplied by the extractor when no validators are configured
|
|
for the lane.
|
|
|
|
## `appendorder` Merger
|
|
|
|
Package: `internal/modules/merge/appendorder`
|
|
|
|
The `appendorder` merger clones and appends candidates in chunk order. It does
|
|
not deduplicate or reconcile candidates.
|
|
|
|
Provides:
|
|
|
|
- `merged`
|
|
|
|
## `noop` Normalizer
|
|
|
|
Package: `internal/modules/normalize/noop`
|
|
|
|
The `noop` normalizer clones merged candidates and returns them unchanged.
|
|
|
|
Requires:
|
|
|
|
- `merged`
|
|
|
|
Provides:
|
|
|
|
- `normalized`
|
|
|
|
## `json` Output
|
|
|
|
Package: `internal/modules/output/json`
|
|
|
|
The `json` output encoder converts approved artifacts, rejected artifacts,
|
|
warnings, and the run manifest into logical JSON output files. It groups
|
|
approved artifacts by artifact type and sanitizes artifact-type file names.
|
|
|
|
Requires:
|
|
|
|
- `normalized`
|
|
|
|
Provides:
|
|
|
|
- `encoded`
|
|
|
|
Durable output file shapes belong in the
|
|
[JSON output contract](../integrations/json-output.md). Operator behavior
|
|
belongs in [Operations](../operations.md).
|
|
|
|
## Production Registration
|
|
|
|
Production registration is centralized in `internal/cli/catalog.go`.
|
|
|
|
Do not make framework code import production modules. The CLI wires production
|
|
modules at the application boundary; tests may provide fake registries or fake
|
|
catalogs directly.
|
|
|
|
## Adding A Module
|
|
|
|
When adding a module, keep source-format and extraction-domain boundaries clear:
|
|
|
|
- input modules may know external source formats;
|
|
- extract modules may know artifact semantics and prompt/schema assets;
|
|
- merge and normalize modules own candidate combination and reconciliation;
|
|
- output modules own serialization, not diagnostics or CLI reporting.
|
|
|
|
Update [Development](../policy/development.md), [Configuration](../config.md),
|
|
internal docs, integration docs, and examples when the new module becomes
|
|
implemented production behavior.
|