eric/seriatim
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 7 Wiki Activity

Create canonical CLI and config docs and simplify README

Browse Source
This commit is contained in:
Eric Rakestraw
2026-05-24 13:31:17 +00:00
parent 7d9bf33d18
commit 385c62a5b4
3 changed files with 361 additions and 502 deletions
Download Patch File Download Diff File Expand all files Collapse all files

525
README.md
View File

@@ -1,513 +1,34 @@
# seriatim
`seriatim` merges per-speaker WhisperX-style JSON transcripts into a single JSON transcript that preserves speaker identity and chronological order. It also trims existing seriatim output artifacts by segment ID and normalizes external transcript-like JSON into standard seriatim output schemas.
`seriatim` is a Go CLI for transcript artifact processing.
The current implementation supports the `merge`, `trim`, and `normalize` commands. `merge` reads one or more input JSON files, optionally maps each input file to a canonical speaker using `speakers.yml`, sorts all segments by timestamp, detects and resolves overlaps when word-level timing is available, assigns consecutive numeric `id` values, and writes a merged JSON artifact. `trim` reads an existing seriatim output artifact and projects it to a retained segment subset. `normalize` reads transcript-like JSON input, validates required segment fields, sorts deterministically, assigns fresh IDs, and emits a selected seriatim output schema.
It merges per-speaker WhisperX-style JSON into one deterministic transcript, trims existing seriatim artifacts by segment ID, and normalizes transcript-like JSON into standard seriatim output schemas.
## Usage
## Quickstart
Run from source:
Shortest useful merge command:
```sh
go run ./cmd/seriatim merge \
--input-file samples/raw/2026-04-19-Eric_Rakestraw.json \
--input-file samples/raw/2026-04-19-Mike_Brown.json \
--input-file speaker-a.json \
--input-file speaker-b.json \
--output-file merged.json
```
Optional report output:
```sh
go run ./cmd/seriatim merge \
--input-file eric.json \
--input-file mike.json \
--output-file merged.json \
--report-file report.json
```
Trim an existing seriatim artifact:
```sh
go run ./cmd/seriatim trim \
--input-file merged.json \
--output-file trimmed.json \
--keep "1-10, 15, 20-25"
```
Normalize external transcript-style JSON:
```sh
go run ./cmd/seriatim normalize \
--input-file transcript.json \
--output-file normalized.json
```
Normalize an Audita-style bare segment array to full schema with report output:
```sh
go run ./cmd/seriatim normalize \
--input-file audita-segments.json \
--output-file normalized-full.json \
--output-schema seriatim-full \
--report-file normalize-report.json
```
## CLI
```text
seriatim merge [flags]
seriatim trim [flags]
seriatim normalize [flags]
```
Global flags:
| Flag | Description |
| --- | --- |
| `--help` | Show command help. |
| `--version` | Show application version. Local builds default to `dev`; release builds inject the release version. |
`merge` flags:
| Flag | Required | Default | Description |
| --- | --- | --- | --- |
| `--input-file` | Yes | none | Input transcript JSON file. Repeat once per speaker/input file. |
| `--output-file` | Yes | none | Merged transcript JSON output path. |
| `--report-file` | No | none | Optional report JSON output path. |
| `--speakers` | No | none | Speaker map YAML file. When omitted, input file basenames are used as speaker labels. |
| `--autocorrect` | No | none | Autocorrect rules YAML file. When omitted, the default `autocorrect` module leaves text unchanged. |
| `--input-reader` | No | `json-files` | Input reader module. |
| `--output-modules` | No | `json` | Comma-separated output modules. |
| `--output-schema` | No | `seriatim-intermediate` | JSON output contract. Allowed values are `seriatim-minimal`, `seriatim-intermediate`, and `seriatim-full`. If omitted, the runtime default is used; consumers that depend on a specific shape should set this explicitly. |
| `--preprocessing-modules` | No | `validate-raw,normalize-speakers,trim-text` | Comma-separated preprocessing modules, evaluated in order. |
| `--postprocessing-modules` | No | `detect-overlaps,resolve-overlaps,backchannel,filler,resolve-danglers,coalesce,detect-overlaps,autocorrect,assign-ids,validate-output` | Comma-separated postprocessing modules, evaluated in order. |
| `--coalesce-gap` | No | `3.0` | Maximum same-speaker gap in seconds for `coalesce`; also used as the `resolve-overlaps` context window. Must be a non-negative float. |
`trim` flags:
| Flag | Required | Default | Description |
| --- | --- | --- | --- |
| `--input-file` | Yes | none | Input seriatim output artifact JSON file. |
| `--output-file` | Yes | none | Trimmed transcript JSON output path. |
| `--keep` | Exactly one of `--keep` or `--remove` is required | none | Segment ID selector to retain. |
| `--remove` | Exactly one of `--keep` or `--remove` is required | none | Segment ID selector to drop. |
| `--output-schema` | No | preserve input artifact schema | Optional output schema override: `seriatim-minimal`, `seriatim-intermediate`, or `seriatim-full`. |
| `--report-file` | No | none | Optional report JSON output path. |
| `--allow-empty` | No | `false` | Allow trimming to zero retained segments. |
`trim` selection rules:
- `--keep` and `--remove` are mutually exclusive.
- Exactly one of `--keep` or `--remove` is required.
- Selection is by segment ID only.
- Invalid selected segment IDs fail the command by default.
`trim` selector syntax:
- Segment IDs are positive 1-based integers.
- Inclusive ranges are supported: `1-10`.
- Comma-separated selectors are supported: `1-10,15,20-25`.
- Whitespace around numbers, commas, and hyphens is allowed: `1 - 10, 15, 20 - 25`.
- Duplicate and overlapping ranges are accepted and normalized as a union.
- Descending ranges (for example `10-1`) are rejected.
`trim` behavior:
- `trim` consumes existing seriatim JSON output artifacts only.
- `trim` does not accept raw WhisperX transcript JSON as input.
- Retained output segment IDs are renumbered sequentially from `1` to `N`.
- Transcript order is preserved from input transcript order; selector order does not reorder output.
- When output schema is `seriatim-full`, overlap groups are recomputed from retained segments.
- `--output-schema seriatim-full` is supported when trim has full-schema artifact data to emit; trim does not synthesize missing full-schema provenance from minimal/intermediate input artifacts.
- `trim` does not run merge postprocessors such as `resolve-overlaps`, `coalesce`, or `autocorrect`.
`trim` report output:
- When `--report-file` is provided, the report includes standard trim/validation/output events.
- The report includes a `trim-audit` event containing trim operation metadata, including selected IDs, retained/removed counts, removed IDs, and old-to-new segment ID mapping.
- Old-to-new ID mapping is emitted as a deterministic ordered array of `{old_id, new_id}` pairs.
`normalize` flags:
| Flag | Required | Default | Description |
| --- | --- | --- | --- |
| `--input-file` | Yes | none | Input transcript JSON file. |
| `--output-file` | Yes | none | Normalized transcript JSON output path. |
| `--output-schema` | No | `seriatim-intermediate` (resolved via `SERIATIM_OUTPUT_SCHEMA` when set) | Output JSON schema: `seriatim-minimal`, `seriatim-intermediate`, or `seriatim-full`. |
| `--output-modules` | No | `json` | Comma-separated output modules. Current normalize support is `json` only. |
| `--report-file` | No | none | Optional report JSON output path. |
`normalize` input shapes:
- Top-level object with a `segments` array.
- Bare top-level array of segment objects (for example, Audita-style output).
`normalize` behavior:
- Repairs missing timing fields deterministically:
if one of `start`/`end` is present, sets both to that value;
if both are missing, uses midpoint of previous `end` and next `start`,
with edge fallback to available neighbor and `0.0` for single-segment inputs.
- If `end < start`, swaps them.
- Fills missing/empty `speaker` with `Unknown_Speaker`.
- Drops segments with missing, empty, or whitespace-only `text`.
- Validates repaired timing with `start >= 0`.
- Accepts existing input `id` values as provenance only.
- Reassigns output segment IDs sequentially from `1` to `N`.
- Sorts deterministically by `(start, end, original_input_index, speaker)`.
- Uses original input order only as a tie-breaker.
- Does not run merge postprocessors such as overlap detection, overlap resolution, coalescing, or autocorrect.
- Useful for converting external transcript outputs into standard seriatim artifacts.
`normalize` report output:
- When `--report-file` is provided, normalize emits deterministic report events with input shape detection, segment counts, schema/module selections, sorting/ID diagnostics, and output write/validation summaries.
- A machine-readable `normalize-audit` event is included for downstream tooling.
Environment variables:
| Environment Variable | Default | Description |
| --- | --- | --- |
| `SERIATIM_OUTPUT_SCHEMA` | `seriatim-intermediate` | Output schema used when `--output-schema` is not explicitly provided. Allowed values are `seriatim-minimal`, `seriatim-intermediate`, and `seriatim-full`. The CLI flag takes precedence. |
| `SERIATIM_OVERLAP_WORD_RUN_GAP` | `1.0` | Maximum gap in seconds between adjacent timed words when `resolve-overlaps` builds word-run replacement segments. Must be a positive float. |
| `SERIATIM_OVERLAP_WORD_RUN_REORDER_WINDOW` | `1.0` | Near-start window in seconds for ordering replacement word runs shortest-first. Must be a positive float. |
| `SERIATIM_BACKCHANNEL_MAX_DURATION` | `2.0` | Maximum duration in seconds for `backchannel` classification. Must be a positive float. |
| `SERIATIM_FILLER_MAX_DURATION` | `1.25` | Maximum duration in seconds for `filler` classification. Must be a positive float. |
## Input JSON Format
Each input file must be valid JSON with a top-level `segments` array. The current parser accepts the WhisperX segment subset needed for merging:
```json
{
"segments": [
{
"start": 1.25,
"end": 3.5,
"text": "Hello there.",
"words": [
{"word": "Hello", "start": 1.25, "end": 1.55, "score": 0.98},
{"word": "there.", "start": 1.7, "end": 2.0}
]
}
]
}
```
Required segment fields:
- `start`: number, must be `>= 0`.
- `end`: number, must be `>= start`.
- `text`: string.
Optional word fields:
- `words`: array of word timing objects.
- `words[].word`: string.
- `words[].start`: optional number, must be `>= 0` when present.
- `words[].end`: optional number, must be `>= start` when present with `start`.
- `words[].score`: optional number.
- `words[].speaker`: optional raw speaker label string.
Word-level timing is preserved internally for overlap resolution. If a word is missing `start` or `end`, seriatim keeps the word text, emits a warning in the optional report, and does not use that word as a timing anchor. Word timing is not emitted in the final JSON artifact.
## Speaker Map Format
`speakers.yml` maps input files to canonical speaker names using ordered substring rules:
This file is optional. If `--speakers` is omitted, `seriatim` uses each input file basename as the segment speaker label.
```yaml
match:
- speaker: "Eric Rakestraw"
match:
- "Eric_Rakestraw"
- "Eric"
- speaker: "Mike Brown"
match:
- "Mike_Brown"
- "mb"
```
For each `--input-file`, `seriatim` takes the file basename and evaluates the rules in order. The first rule with a matching substring wins, and no later rules are evaluated.
For example, this input:
```text
samples/raw/2026-04-19-Eric_Rakestraw.json
```
matches this rule because the basename contains `Eric_Rakestraw`:
```yaml
- speaker: "Eric Rakestraw"
match:
- "Eric_Rakestraw"
```
Important details:
- Matching is against the input file basename, not the full path.
- Matching is case-insensitive.
- Rules are evaluated from first to last.
- Each rule must have a non-empty `speaker`.
- Each rule must have at least one non-empty `match` string.
- Duplicate speaker names are invalid.
- Every input file must match at least one rule or the command fails.
Deprecated old format:
```yaml
inputs:
eric.json:
speaker: "Eric Rakestraw"
```
The old `inputs:` direct mapping format is no longer supported.
## Output JSON Format
`--output-modules json` controls the writer. `--output-schema` controls the JSON contract that writer serializes.
The named schemas are stable public contracts. If a consumer depends on a specific shape, it should request that schema explicitly at runtime. The runtime default selection may change in a future release.
The `seriatim-intermediate` schema is the current default selection when neither `--output-schema` nor `SERIATIM_OUTPUT_SCHEMA` is set. It stays close to the minimal schema, but adds optional `categories` on each segment:
```json
{
"metadata": {
"application": "seriatim",
"version": "dev",
"output_schema": "seriatim-intermediate"
},
"segments": [
{
"id": 1,
"start": 1.25,
"end": 3.5,
"speaker": "Eric Rakestraw",
"text": "Hello there.",
"categories": ["backchannel"]
}
]
}
```
The `seriatim-full` schema uses the full seriatim envelope:
```json
{
"metadata": {
"application": "seriatim",
"version": "dev",
"input_reader": "json-files",
"input_files": ["eric.json", "mike.json"],
"preprocessing_modules": ["validate-raw", "normalize-speakers", "trim-text"],
"postprocessing_modules": ["detect-overlaps", "resolve-overlaps", "backchannel", "filler", "resolve-danglers", "coalesce", "detect-overlaps", "autocorrect", "assign-ids", "validate-output"],
"output_modules": ["json"]
},
"segments": [
{
"id": 1,
"source": "eric.json",
"source_segment_index": 0,
"speaker": "Eric Rakestraw",
"start": 1.25,
"end": 3.5,
"text": "Hello there.",
"overlap_group_id": 1
},
{
"id": 2,
"source": "eric.json",
"source_ref": "word-run:1:1:1",
"derived_from": ["eric.json#0"],
"speaker": "Eric Rakestraw",
"start": 2.0,
"end": 2.5,
"text": "Resolved word run",
"categories": ["backchannel"]
}
],
"overlap_groups": [
{
"id": 1,
"start": 1.25,
"end": 4.0,
"segments": ["eric.json#0", "mike.json#0"],
"speakers": ["Eric Rakestraw", "Mike Brown"],
"class": "unknown",
"resolution": "unresolved"
}
]
}
```
The `seriatim-minimal` schema emits minimal metadata and compact ordered segments:
```json
{
"metadata": {
"application": "seriatim",
"version": "dev",
"output_schema": "seriatim-minimal"
},
"segments": [
{
"id": 1,
"start": 1.25,
"end": 3.5,
"speaker": "Eric Rakestraw",
"text": "Hello there."
}
]
}
```
Minimal output intentionally omits categories, overlap groups, source/provenance fields, and pipeline configuration metadata.
Intermediate output intentionally omits overlap groups and source/provenance fields, but keeps optional `categories` and minimal metadata.
Segments are sorted deterministically by:
```text
(start, end, source, source_segment_index/source_ref, speaker)
```
Final segment IDs are assigned after sorting and start at `1`.
The public Go output contract is available from:
```go
import "gitea.maximumdirect.net/eric/seriatim/schema"
```
The same package embeds machine-readable JSON Schemas in `schema/full-output.schema.json`, `schema/intermediate-output.schema.json`, and `schema/minimal-output.schema.json`. The default `validate-output` postprocessor validates the selected output shape and verifies final segment IDs are present, sequential, and start at `1`.
## Overlap Detection
The default postprocessing pipeline detects overlapping segment groups.
Overlap behavior:
- A strict timing overlap is required: `next.start < current_group_end`.
- Segments that only touch at a boundary are not grouped.
- Groups require at least two distinct speakers.
- Transitive overlaps are grouped together.
- Segments in detected groups receive `overlap_group_id`.
- `overlap_groups[].segments` contains stable references in `source#source_segment_index` format.
- `class` is currently `unknown`.
- `resolution` is `unresolved` until `resolve-overlaps` replaces the group.
## Overlap Resolution
The default postprocessing pipeline runs `detect-overlaps`, then `resolve-overlaps`, then `backchannel`, then `filler`, then `resolve-danglers`, then `coalesce`, then a second `detect-overlaps` pass.
For each detected overlap group, `resolve-overlaps` uses preserved WhisperX word timing to build smaller word-run replacement segments:
- The resolution window expands the detected overlap group by `--coalesce-gap` seconds on both sides.
- Nearby same-speaker context segments are included when they intersect the expanded window and their start or end is within `--coalesce-gap` of the original overlap boundary.
- Once a segment is selected for replacement, all timed words from that segment participate in word-run construction; the window controls segment selection, not per-word clipping.
- Context segments that are part of another detected overlap group are not pulled into the current group.
- Untimed words are included in replacement text in original word order when nearby timed words create a replacement run.
- Untimed words do not affect replacement segment start/end times or word-run gap splitting.
- Words for the same speaker are merged into one run when the gap between adjacent words is no greater than `SERIATIM_OVERLAP_WORD_RUN_GAP`.
- The default word-run gap is `1.0` seconds.
- Set `SERIATIM_OVERLAP_WORD_RUN_GAP` to a positive number of seconds to override the default.
- Near-start replacement word runs are reordered so shorter segments come first when adjacent starts are within `SERIATIM_OVERLAP_WORD_RUN_REORDER_WINDOW`.
- The default word-run reorder window is `1.0` seconds.
- Set `SERIATIM_OVERLAP_WORD_RUN_REORDER_WINDOW` to a positive number of seconds to override the default.
- Replacement segment text is built by joining word text with single spaces.
- Replacement segments include `source_ref` and `derived_from`.
- Replacement segments omit `source_segment_index` because they are derived from one or more original segments.
- Resolved overlap groups are removed before the second detection pass.
- Replacement segments are left without `overlap_group_id` until the second detection pass annotates any remaining overlap.
- If a speaker has no usable word timing in a group, that speaker's original segment is kept.
- If no speakers in a group have usable word timing, the original group and annotations remain unchanged.
## Backchannels
The default pipeline runs `backchannel` before `coalesce`. It tags short acknowledgement segments with:
```json
"categories": ["backchannel"]
```
Backchannel matching is case-insensitive, ignores punctuation for matching and word-count purposes, trims surrounding whitespace, and requires a matching acknowledgement phrase, no more than three whitespace-delimited words, and duration no greater than `SERIATIM_BACKCHANNEL_MAX_DURATION` seconds. The default maximum duration is `2.0` seconds.
## Fillers
The default pipeline runs `filler` after `backchannel` and before `coalesce`. It tags short filler utterances with:
```json
"categories": ["filler"]
```
Filler matching is case-insensitive, ignores punctuation for matching and word-count purposes, trims surrounding whitespace, and requires only filler tokens such as `um`, `uh`, `er`, `erm`, `ah`, `eh`, `hmm`, `mm`, or repeated combinations of those tokens. Matching segments must contain no more than three whitespace-delimited words and have duration no greater than `SERIATIM_FILLER_MAX_DURATION` seconds. The default maximum duration is `1.25` seconds.
## Dangler Resolution
The default pipeline runs `resolve-danglers` before `coalesce` and before the second overlap detection pass. It repairs short derived fragments when they share provenance with a nearby segment:
- Dangling-end fragments have no more than two words and end in punctuation.
- Dangling-start fragments have no more than two words.
- Matching uses same-speaker segments with any shared `derived_from` value.
- Merged segments use `source_ref` values such as `resolve-danglers:1`, keep the target segment's transcript position, and union `derived_from`.
## Coalescing
The default pipeline runs `coalesce` after `resolve-danglers` and before the second overlap detection pass. It merges adjacent same-speaker segments in the transcript's current order when `next.start - current.end <= --coalesce-gap`.
Coalesced segments use `source_ref` values such as `coalesce:1`, include `derived_from`, and omit `source_segment_index`.
Different-speaker backchannel and filler segments do not block coalescing of surrounding same-speaker segments. Same-speaker backchannel and filler segments are merged normally when they are within `--coalesce-gap`. When same-speaker segments are coalesced, any `backchannel` or `filler` category from the merged inputs is dropped from the coalesced segment.
## Autocorrect
Autocorrect is included in the default postprocessing pipeline. If `--autocorrect` is omitted, the module leaves transcript text unchanged and records a skip event in the optional report.
Enable corrections by passing `--autocorrect`:
```sh
go run ./cmd/seriatim merge \
--input-file input.json \
--autocorrect autocorrect.yml \
--output-file merged.json
```
`autocorrect.yml` format:
```yaml
autocorrect:
- target: "Hrank"
match:
- "hrank"
- "Frank"
- target: "Mike Brown"
match:
- "Mike Pat"
```
Matching behavior:
- Matching is case-sensitive.
- Matches apply only to whole tokens, not substrings inside larger words.
- Punctuation and whitespace can surround a match.
- Multi-word and hyphenated matches are supported.
- Duplicate match strings are invalid, including duplicates across separate rules.
## Current Limitations
- Only JSON input is supported.
- Overlap resolution depends on WhisperX word timing; groups without usable word timing remain unresolved.
- Alternate output formats are not implemented yet.
## Release Builds
Local builds record version metadata as `dev`. Release builds should inject the release version with `ldflags`:
```sh
go build -ldflags "-X gitea.maximumdirect.net/eric/seriatim/internal/buildinfo.Version=v1.0.0" ./cmd/seriatim
```
## Commands
- `merge`: merge one or more input transcript JSON files.
- `trim`: keep/remove segment IDs from an existing seriatim artifact.
- `normalize`: canonicalize transcript-like JSON into a seriatim artifact.
## Documentation
- CLI reference: [docs/cli.md](docs/cli.md)
- Configuration reference: [docs/config.md](docs/config.md)
- Development architecture policy: [docs/policy/architecture.md](docs/policy/architecture.md)
- Documentation policy: [docs/policy/documentation.md](docs/policy/documentation.md)
- Public JSON schemas:
- [schema/minimal-output.schema.json](schema/minimal-output.schema.json)
- [schema/intermediate-output.schema.json](schema/intermediate-output.schema.json)
- [schema/full-output.schema.json](schema/full-output.schema.json)
- Remaining documentation migration work: [docs/roadmap/documentation.md](docs/roadmap/documentation.md)

176
docs/cli.md Normal file
View File

@@ -0,0 +1,176 @@
# CLI Reference
## Shortest useful command
```sh
go run ./cmd/seriatim merge \
--input-file speaker-a.json \
--input-file speaker-b.json \
--output-file merged.json
```
## Command overview
| Command | Purpose |
| --- | --- |
| `merge` | Merge one or more raw transcript JSON inputs into one seriatim artifact. |
| `trim` | Keep or remove segment IDs from an existing seriatim artifact. |
| `normalize` | Canonicalize transcript-like JSON into a seriatim artifact. |
Root usage:
```text
seriatim [command]
```
## Global flags
| Flag | Description |
| --- | --- |
| `-h, --help` | Show help. |
| `-v, --version` | Show build version. |
## `merge`
Usage:
```text
seriatim merge [flags]
```
Flags:
| Flag | Required | Default | Description |
| --- | --- | --- | --- |
| `--input-file stringArray` | Yes, repeat at least once | none | Input transcript JSON file(s). |
| `--output-file string` | Yes | none | Output transcript JSON file path. |
| `--report-file string` | No | none | Optional report JSON path. |
| `--speakers string` | No | none | Speaker-map YAML file. |
| `--autocorrect string` | No | none | Autocorrect YAML file. |
| `--input-reader string` | No | `json-files` | Input reader module name. |
| `--output-modules string` | No | `json` | Comma-separated output module names. |
| `--output-schema string` | No | `seriatim-intermediate` | Output schema name: `seriatim-minimal`, `seriatim-intermediate`, `seriatim-full`. |
| `--preprocessing-modules string` | No | `validate-raw,normalize-speakers,trim-text` | Comma-separated preprocessing module names, run in order. |
| `--postprocessing-modules string` | No | `detect-overlaps,resolve-overlaps,backchannel,filler,resolve-danglers,coalesce,detect-overlaps,autocorrect,assign-ids,validate-output` | Comma-separated postprocessing module names, run in order. |
| `--coalesce-gap string` | No | `3.0` | Non-negative seconds for coalescing and overlap-resolution context. |
`merge` behavior and validation:
- Unknown input reader, preprocessing module, postprocessing module, or output module fails the command.
- Preprocessing order must satisfy module state requirements (`raw` -> `canonical`); invalid order fails.
- Input files are validated, deduplicated, normalized, then sorted for deterministic processing.
- Optional report output is written only when `--report-file` is set.
- When `--output-schema` is omitted, schema resolution is: `SERIATIM_OUTPUT_SCHEMA` -> default `seriatim-intermediate`.
## `trim`
Usage:
```text
seriatim trim [flags]
```
Flags:
| Flag | Required | Default | Description |
| --- | --- | --- | --- |
| `--input-file string` | Yes | none | Input seriatim artifact JSON file. |
| `--output-file string` | Yes | none | Output transcript JSON file path. |
| `--keep string` | Exactly one of `--keep` / `--remove` | none | Segment ID selector to keep. |
| `--remove string` | Exactly one of `--keep` / `--remove` | none | Segment ID selector to remove. |
| `--output-schema string` | No | preserve input artifact schema | Output schema override: `seriatim-minimal`, `seriatim-intermediate`, `seriatim-full`. |
| `--report-file string` | No | none | Optional report JSON path. |
| `--allow-empty` | No | `false` | Allow output with zero segments. |
Selector rules:
- IDs must be positive integers.
- Single IDs and inclusive ranges are supported: `1`, `1-10`.
- Comma-separated selectors are supported: `1-10,15,20-25`.
- Whitespace around commas and hyphens is allowed.
- Descending ranges (example `10-1`) are invalid.
- Duplicates and overlapping ranges are normalized as a union.
`trim` behavior:
- Input must already be a valid seriatim artifact (not raw merge input JSON).
- Output keeps transcript order from input and renumbers retained segment IDs sequentially.
- If `--output-schema` is omitted, the input artifact schema is preserved.
- `trim` never runs merge preprocessing/postprocessing modules.
## `normalize`
Usage:
```text
seriatim normalize [flags]
```
Flags:
| Flag | Required | Default | Description |
| --- | --- | --- | --- |
| `--input-file string` | Yes | none | Input transcript JSON file. |
| `--output-file string` | Yes | none | Output transcript JSON file path. |
| `--output-schema string` | No | `seriatim-intermediate` | Output schema name: `seriatim-minimal`, `seriatim-intermediate`, `seriatim-full`. |
| `--output-modules string` | No | `json` | Comma-separated output module names (`json` only). |
| `--report-file string` | No | none | Optional report JSON path. |
`normalize` input shapes:
- Object with top-level `segments` array.
- Bare top-level segment array.
`normalize` behavior:
- Sorts deterministically and reassigns output IDs sequentially from `1`.
- Fills missing/blank speakers with `Unknown_Speaker`.
- Repairs/sanitizes timing fields deterministically; rejects invalid repaired timing.
- Drops segments with missing or blank text.
- Does not run merge modules.
- When `--output-schema` is omitted, schema resolution is: `SERIATIM_OUTPUT_SCHEMA` -> default `seriatim-intermediate`.
## Common workflows
Merge with a speaker map and report output:
```sh
go run ./cmd/seriatim merge \
--input-file speaker-a.json \
--input-file speaker-b.json \
--speakers speakers.yml \
--output-file merged.json \
--report-file merge-report.json
```
Trim to a segment subset:
```sh
go run ./cmd/seriatim trim \
--input-file merged.json \
--output-file trimmed.json \
--keep "1-20,25"
```
Normalize an external transcript JSON file:
```sh
go run ./cmd/seriatim normalize \
--input-file external.json \
--output-file normalized.json
```
## Exit and errors
- Commands return exit code `0` on success.
- On error, the CLI prints one error line to stderr and exits with status `1`.
- Cobra usage text is silenced on runtime errors; use `--help` for command usage.
## Related docs
- Configuration reference: [config.md](config.md)
- Public output schemas:
- [../schema/minimal-output.schema.json](../schema/minimal-output.schema.json)
- [../schema/intermediate-output.schema.json](../schema/intermediate-output.schema.json)
- [../schema/full-output.schema.json](../schema/full-output.schema.json)
- Remaining docs migration work (operations, troubleshooting, examples): [roadmap/documentation.md](roadmap/documentation.md)

162
docs/config.md Normal file
View File

@@ -0,0 +1,162 @@
# Configuration Reference
## Configuration surfaces
seriatim has no central JSON/TOML/YAML application config file.
Runtime configuration comes from:
1. CLI flags
2. Environment variables (`SERIATIM_*`)
3. Optional YAML rule files referenced by CLI flags (`--speakers`, `--autocorrect`)
## Output schema precedence
For `merge` and `normalize`:
1. `--output-schema` flag (when explicitly set)
2. `SERIATIM_OUTPUT_SCHEMA`
3. default `seriatim-intermediate`
For `trim`:
- If `--output-schema` is omitted, output preserves the input artifact schema.
- If `--output-schema` is set, it must be one of `seriatim-minimal`, `seriatim-intermediate`, `seriatim-full`.
## Merge module defaults
Default merge module selections:
- `--input-reader`: `json-files`
- `--preprocessing-modules`: `validate-raw,normalize-speakers,trim-text`
- `--postprocessing-modules`: `detect-overlaps,resolve-overlaps,backchannel,filler,resolve-danglers,coalesce,detect-overlaps,autocorrect,assign-ids,validate-output`
- `--output-modules`: `json`
Module-list notes:
- Lists are comma-separated.
- Empty module names are invalid.
- Unknown module names fail the command.
- Preprocessing order must satisfy state requirements.
## Environment variables
| Variable | Default | Used by | Rules |
| --- | --- | --- | --- |
| `SERIATIM_OUTPUT_SCHEMA` | `seriatim-intermediate` | `merge`, `normalize` | Must be `seriatim-minimal`, `seriatim-intermediate`, or `seriatim-full`. Ignored when `--output-schema` is explicitly set. |
| `SERIATIM_OVERLAP_WORD_RUN_GAP` | `1.0` | `merge` | Positive float (`> 0`). |
| `SERIATIM_OVERLAP_WORD_RUN_REORDER_WINDOW` | `1.0` | `merge` | Positive float (`> 0`). |
| `SERIATIM_BACKCHANNEL_MAX_DURATION` | `2.0` | `merge` | Positive float (`> 0`). |
| `SERIATIM_FILLER_MAX_DURATION` | `1.25` | `merge` | Positive float (`> 0`). |
Additional merge threshold flag:
- `--coalesce-gap` defaults to `3.0` and must be a non-negative float (`>= 0`).
## `speakers.yml`
Purpose:
- Maps each merge input filename basename to a canonical speaker label.
Top-level key:
- `match` (array of ordered rules)
Rule fields:
- `speaker` (required, non-empty)
- `match` (required, non-empty array of non-empty strings)
Example:
```yaml
match:
- speaker: "Alice"
match:
- "alice_track"
- "alice"
- speaker: "Bob"
match:
- "bob_track"
```
Behavior:
- Matching is case-insensitive.
- Matching is against basename only (not full path).
- First matching rule wins.
- Duplicate `speaker` values are invalid.
- If any input file has no match, merge fails.
## `autocorrect.yml`
Purpose:
- Applies ordered token-level text replacements during merge `autocorrect` postprocessing.
Top-level key:
- `autocorrect` (array of rules)
Rule fields:
- `target` (required, non-empty)
- `match` (required, non-empty array of non-empty strings)
Example:
```yaml
autocorrect:
- target: "Godfrey"
match:
- "God-free"
- target: "Mike Brown"
match:
- "Mike Pat"
```
Behavior:
- Match strings are case-sensitive.
- Replacements are whole-token only (no substring replacement inside larger tokens).
- Duplicate match strings within one rule are invalid.
- Duplicate match strings across different rules are invalid.
- If `--autocorrect` is not provided, the autocorrect module is skipped.
## Path and validation rules
All commands:
- `--input-file` paths must exist and must be files.
- Output/report parent directories must already exist.
- Paths are normalized before use.
`merge`:
- Requires at least one `--input-file`.
- Rejects duplicate `--input-file` paths.
- Sorts normalized input file paths for deterministic execution.
- `--speakers` and `--autocorrect` are optional, but when set they must point to existing files.
`trim`:
- Requires exactly one of `--keep` or `--remove`.
- `--keep` and `--remove` are mutually exclusive.
- Validates optional `--output-schema` when provided.
`normalize`:
- Validates `--output-schema` through the same schema set as `merge`.
- Currently accepts only `json` in `--output-modules`.
## Related docs
- CLI reference: [cli.md](cli.md)
- Public output schemas:
- [../schema/minimal-output.schema.json](../schema/minimal-output.schema.json)
- [../schema/intermediate-output.schema.json](../schema/intermediate-output.schema.json)
- [../schema/full-output.schema.json](../schema/full-output.schema.json)
- Remaining docs migration work (operations, troubleshooting, examples): [roadmap/documentation.md](roadmap/documentation.md)