seriatim/docs/internal/modules.md

Built-In Modules

Purpose

Describes implemented built-in module behavior and boundaries in internal/builtin.

Implemented module set

Input reader:

• json-files

Preprocessing:

• validate-raw
• normalize-speakers
• trim-text

Merger:

• chronological-merge

Postprocessing:

• detect-overlaps
• resolve-overlaps
• backchannel
• filler
• resolve-danglers
• coalesce
• autocorrect
• assign-ids
• validate-output

Output writer:

• json

Inputs, outputs, and side effects

• json-files: reads JSON files from cfg.InputFiles, parses supported segment/word fields, emits warnings for untimed words.
• validate-raw: validates raw source/timing invariants.
• normalize-speakers: converts raw transcripts to canonical segments, optionally resolving speakers from cfg.SpeakersFile.
• trim-text: trims canonical segment text whitespace.
• chronological-merge: flattens canonical segments and applies deterministic sort (model.SegmentLess).
• detect-overlaps: annotates overlap groups.
• resolve-overlaps: rewrites overlap groups using timed words and thresholds.
• backchannel/filler: classify short utterances using duration thresholds.
• resolve-danglers: merges dangling derived fragments.
• coalesce: merges adjacent same-speaker segments within configured gap.
• autocorrect: applies YAML replacement rules when configured.
• assign-ids: assigns final sequential IDs.
• validate-output: validates selected public artifact shape.
• json: writes artifact JSON to cfg.OutputFile.

Filesystem side effects are limited to:

• reading configured input/YAML files
• writing configured output artifact

Config fields used

Primary module inputs from config.Config:

• file paths: InputFiles, SpeakersFile, AutocorrectFile, OutputFile
• schema/modules: OutputSchema, OutputModules
• overlap/coalesce thresholds: OverlapWordRunGap, WordRunReorderWindow, CoalesceGap
• category thresholds: BackchannelMaxDuration, FillerMaxDuration

Ordering constraints

• Preprocessing must satisfy state contracts from raw to canonical.
• Order-sensitive transforms should run before assign-ids.
• validate-output should run after final ID assignment and output-shape mutations.
• Default configuration includes a second detect-overlaps pass after transformations.

Boundaries

• Modules implement behavior; CLI/config parsing remains outside modules.
• Modules communicate through explicit model contracts and report events.
• Output modules operate on final artifacts and do not re-run transform logic.

Failure behavior

Representative failures:

• invalid input JSON shape or typed field errors (json-files)
• invalid YAML or unmatched speaker map entries
• unknown module names during registry resolution
• invalid ordering/state transitions in preprocessing chain
• validation failure in validate-output
• output write failure in json writer

Tests to inspect before changes

• internal/builtin/preprocess_test.go
• internal/builtin/postprocess_test.go
• internal/overlap/resolve_test.go
• internal/overlap/detect_test.go
• internal/coalesce/coalesce_test.go
• internal/danglers/danglers_test.go
• internal/backchannel/backchannel_test.go
• internal/filler/filler_test.go
• internal/autocorrect/autocorrect_test.go
• internal/cli/merge_test.go

Invariants

• Modules are selected by canonical name through the registry.
• Execution is sequential and deterministic for a fixed configuration.
• assign-ids defines final public segment IDs.
• validate-output enforces public artifact contracts through schema.