Add development policy and internal implementation documentation

docs/internal/modules.md

@@ -0,0 +1,119 @@
+# Built-In Modules
+
+## Purpose
+
+Describes implemented built-in module behavior and boundaries in
+`internal/builtin`.
+
+## Implemented module set
+
+Input reader:
+
+- `json-files`
+
+Preprocessing:
+
+- `validate-raw`
+- `normalize-speakers`
+- `trim-text`
+
+Merger:
+
+- `chronological-merge`
+
+Postprocessing:
+
+- `detect-overlaps`
+- `resolve-overlaps`
+- `backchannel`
+- `filler`
+- `resolve-danglers`
+- `coalesce`
+- `autocorrect`
+- `assign-ids`
+- `validate-output`
+
+Output writer:
+
+- `json`
+
+## Inputs, outputs, and side effects
+
+- `json-files`: reads JSON files from `cfg.InputFiles`, parses supported
+  segment/word fields, emits warnings for untimed words.
+- `validate-raw`: validates raw source/timing invariants.
+- `normalize-speakers`: converts raw transcripts to canonical segments,
+  optionally resolving speakers from `cfg.SpeakersFile`.
+- `trim-text`: trims canonical segment text whitespace.
+- `chronological-merge`: flattens canonical segments and applies deterministic
+  sort (`model.SegmentLess`).
+- `detect-overlaps`: annotates overlap groups.
+- `resolve-overlaps`: rewrites overlap groups using timed words and thresholds.
+- `backchannel`/`filler`: classify short utterances using duration thresholds.
+- `resolve-danglers`: merges dangling derived fragments.
+- `coalesce`: merges adjacent same-speaker segments within configured gap.
+- `autocorrect`: applies YAML replacement rules when configured.
+- `assign-ids`: assigns final sequential IDs.
+- `validate-output`: validates selected public artifact shape.
+- `json`: writes artifact JSON to `cfg.OutputFile`.
+
+Filesystem side effects are limited to:
+
+- reading configured input/YAML files
+- writing configured output artifact
+
+## Config fields used
+
+Primary module inputs from `config.Config`:
+
+- file paths: `InputFiles`, `SpeakersFile`, `AutocorrectFile`, `OutputFile`
+- schema/modules: `OutputSchema`, `OutputModules`
+- overlap/coalesce thresholds: `OverlapWordRunGap`,
+  `WordRunReorderWindow`, `CoalesceGap`
+- category thresholds: `BackchannelMaxDuration`, `FillerMaxDuration`
+
+## Ordering constraints
+
+- Preprocessing must satisfy state contracts from `raw` to `canonical`.
+- Order-sensitive transforms should run before `assign-ids`.
+- `validate-output` should run after final ID assignment and output-shape
+  mutations.
+- Default configuration includes a second `detect-overlaps` pass after
+  transformations.
+
+## Boundaries
+
+- Modules implement behavior; CLI/config parsing remains outside modules.
+- Modules communicate through explicit model contracts and report events.
+- Output modules operate on final artifacts and do not re-run transform logic.
+
+## Failure behavior
+
+Representative failures:
+
+- invalid input JSON shape or typed field errors (`json-files`)
+- invalid YAML or unmatched speaker map entries
+- unknown module names during registry resolution
+- invalid ordering/state transitions in preprocessing chain
+- validation failure in `validate-output`
+- output write failure in `json` writer
+
+## Tests to inspect before changes
+
+- `internal/builtin/preprocess_test.go`
+- `internal/builtin/postprocess_test.go`
+- `internal/overlap/resolve_test.go`
+- `internal/overlap/detect_test.go`
+- `internal/coalesce/coalesce_test.go`
+- `internal/danglers/danglers_test.go`
+- `internal/backchannel/backchannel_test.go`
+- `internal/filler/filler_test.go`
+- `internal/autocorrect/autocorrect_test.go`
+- `internal/cli/merge_test.go`
+
+## Invariants
+
+- Modules are selected by canonical name through the registry.
+- Execution is sequential and deterministic for a fixed configuration.
+- `assign-ids` defines final public segment IDs.
+- `validate-output` enforces public artifact contracts through `schema`.