diff --git a/README.md b/README.md index 667031e..861ab0f 100644 --- a/README.md +++ b/README.md @@ -27,6 +27,9 @@ go run ./cmd/seriatim merge \ - Configuration reference: [docs/config.md](docs/config.md) - Operations guide: [docs/operations.md](docs/operations.md) - Troubleshooting: [docs/troubleshooting.md](docs/troubleshooting.md) +- Integrations: + - [docs/integrations/whisperx-json.md](docs/integrations/whisperx-json.md) + - [docs/integrations/output-schemas.md](docs/integrations/output-schemas.md) - Development architecture policy: [docs/policy/architecture.md](docs/policy/architecture.md) - Contributor workflow: [docs/policy/development.md](docs/policy/development.md) - Documentation policy: [docs/policy/documentation.md](docs/policy/documentation.md) @@ -38,4 +41,5 @@ go run ./cmd/seriatim merge \ - [schema/minimal-output.schema.json](schema/minimal-output.schema.json) - [schema/intermediate-output.schema.json](schema/intermediate-output.schema.json) - [schema/full-output.schema.json](schema/full-output.schema.json) +- Synthetic examples: [examples/README.md](examples/README.md) - Remaining documentation migration work: [docs/roadmap/documentation.md](docs/roadmap/documentation.md) diff --git a/docs/cli.md b/docs/cli.md index df1a1dc..b5a58c7 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -136,28 +136,28 @@ Merge with a speaker map and report output: ```sh go run ./cmd/seriatim merge \ - --input-file speaker-a.json \ - --input-file speaker-b.json \ - --speakers speakers.yml \ - --output-file merged.json \ - --report-file merge-report.json + --input-file examples/minimal-merge/input-alice.json \ + --input-file examples/minimal-merge/input-bob.json \ + --speakers examples/minimal-merge/speakers.yml \ + --output-file /tmp/seriatim-example-merge.json \ + --report-file /tmp/seriatim-example-merge-report.json ``` Trim to a segment subset: ```sh go run ./cmd/seriatim trim \ - --input-file merged.json \ - --output-file trimmed.json \ - --keep "1-20,25" + --input-file examples/trim/input-full.json \ + --output-file /tmp/seriatim-example-trim.json \ + --keep "1-2" ``` Normalize an external transcript JSON file: ```sh go run ./cmd/seriatim normalize \ - --input-file external.json \ - --output-file normalized.json + --input-file examples/normalize/object-with-segments.json \ + --output-file /tmp/seriatim-example-normalize-object.json ``` ## Exit and errors @@ -171,6 +171,10 @@ go run ./cmd/seriatim normalize \ - Configuration reference: [config.md](config.md) - Operations guide: [operations.md](operations.md) - Troubleshooting: [troubleshooting.md](troubleshooting.md) +- Integration notes: + - [integrations/whisperx-json.md](integrations/whisperx-json.md) + - [integrations/output-schemas.md](integrations/output-schemas.md) +- Synthetic examples: [../examples/README.md](../examples/README.md) - Public output schemas: - [../schema/minimal-output.schema.json](../schema/minimal-output.schema.json) - [../schema/intermediate-output.schema.json](../schema/intermediate-output.schema.json) diff --git a/docs/config.md b/docs/config.md index d707624..bd3bfa2 100644 --- a/docs/config.md +++ b/docs/config.md @@ -157,6 +157,10 @@ All commands: - CLI reference: [cli.md](cli.md) - Operations guide: [operations.md](operations.md) - Troubleshooting: [troubleshooting.md](troubleshooting.md) +- YAML example files: + - [../examples/speakers.yml](../examples/speakers.yml) + - [../examples/autocorrect.yml](../examples/autocorrect.yml) +- Synthetic examples: [../examples/README.md](../examples/README.md) - Public output schemas: - [../schema/minimal-output.schema.json](../schema/minimal-output.schema.json) - [../schema/intermediate-output.schema.json](../schema/intermediate-output.schema.json) diff --git a/docs/integrations/output-schemas.md b/docs/integrations/output-schemas.md new file mode 100644 index 0000000..62ff2b4 --- /dev/null +++ b/docs/integrations/output-schemas.md @@ -0,0 +1,68 @@ +# Output Schemas + +## Scope + +seriatim emits one of three public JSON output contracts: + +- `seriatim-minimal` +- `seriatim-intermediate` +- `seriatim-full` + +These are used by `merge`, `trim`, and `normalize`. + +## Schema roles + +`seriatim-minimal`: + +- compact metadata plus ordered transcript segments +- no source/provenance fields +- no overlap groups + +`seriatim-intermediate`: + +- compact metadata plus ordered segments +- includes optional segment `categories` +- no source/provenance fields +- no overlap groups + +`seriatim-full`: + +- full metadata (`input_reader`, module lists, input files, output modules) +- source/provenance fields on segments +- overlap-group data +- version metadata populated from build info (`internal/buildinfo`) + +## Semantic invariants + +All schema outputs enforce: + +- segment IDs are sequential starting at `1` +- segment timing uses `end >= start` + +Full schema also enforces overlap-group timing (`end >= start`). + +## Validation APIs + +Go package: `gitea.maximumdirect.net/eric/seriatim/schema` + +Key validators: + +- `schema.ValidateMinimalTranscript` +- `schema.ValidateIntermediateTranscript` +- `schema.ValidateTranscript` +- `schema.ValidateMinimalJSON` +- `schema.ValidateIntermediateJSON` +- `schema.ValidateJSON` + +## Machine-readable schema files + +- [../../schema/minimal-output.schema.json](../../schema/minimal-output.schema.json) +- [../../schema/intermediate-output.schema.json](../../schema/intermediate-output.schema.json) +- [../../schema/full-output.schema.json](../../schema/full-output.schema.json) + +## Related docs and examples + +- CLI reference: [../cli.md](../cli.md) +- Artifact internals: [../internal/artifacts.md](../internal/artifacts.md) +- Trim example input artifact: + - [../../examples/trim/input-full.json](../../examples/trim/input-full.json) diff --git a/docs/integrations/whisperx-json.md b/docs/integrations/whisperx-json.md new file mode 100644 index 0000000..af914cd --- /dev/null +++ b/docs/integrations/whisperx-json.md @@ -0,0 +1,85 @@ +# WhisperX-Like JSON Input + +## Scope + +This document covers the implemented JSON subset consumed by `seriatim merge`. +It does not describe full WhisperX output. +No explicit WhisperX version is encoded in the repository. + +## Supported top-level shape + +Merge expects a JSON object with top-level `segments` array: + +```json +{ + "segments": [ + { + "start": 0.0, + "end": 1.2, + "text": "hello" + } + ] +} +``` + +## Supported segment fields + +Required per segment: + +- `start` (number, `>= 0`) +- `end` (number, `>= start`) +- `text` (string) + +Optional per segment: + +- `words` (array) + +## Supported word fields + +Required when a word object is present: + +- `word` (string) + +Optional word timing fields: + +- `start` (number) +- `end` (number) + +Timing rules: + +- if both `start` and `end` are present, they must be numeric and `end >= start` +- if either timing field is missing, the word is accepted but not used as a + timing anchor for overlap resolution + +Additional optional word fields: + +- `score` (number) +- `speaker` (string) + +## Validation and failure behavior + +Merge fails for: + +- malformed JSON +- missing top-level `segments` +- non-array `segments` +- missing required segment fields +- wrong field types +- negative segment/word start times +- segment/word end before start + +Word timing missing from a word does not fail merge; it emits a warning event +in the optional report. + +## Overlap-resolution impact + +- overlap resolution uses timed words when available +- untimed words are kept in replacement text but do not provide timing anchors + +## Related docs and examples + +- CLI reference: [../cli.md](../cli.md) +- Configuration reference: [../config.md](../config.md) +- Minimal merge example inputs: + - [../../examples/minimal-merge/input-alice.json](../../examples/minimal-merge/input-alice.json) + - [../../examples/minimal-merge/input-bob.json](../../examples/minimal-merge/input-bob.json) diff --git a/docs/operations.md b/docs/operations.md index edf3d6e..4d12c9e 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -130,3 +130,7 @@ Transcript artifacts and reports are local files and may contain sensitive conve - CLI reference: [cli.md](cli.md) - Configuration reference: [config.md](config.md) - Troubleshooting: [troubleshooting.md](troubleshooting.md) +- Integration notes: + - [integrations/whisperx-json.md](integrations/whisperx-json.md) + - [integrations/output-schemas.md](integrations/output-schemas.md) +- Synthetic examples: [../examples/README.md](../examples/README.md) diff --git a/examples/README.md b/examples/README.md new file mode 100644 index 0000000..95c92eb --- /dev/null +++ b/examples/README.md @@ -0,0 +1,61 @@ +# Examples + +These are small synthetic, copyable example assets for the implemented CLI +commands. +This directory is the canonical examples home for documentation. + +## Merge example + +Inputs: + +- `minimal-merge/input-alice.json` +- `minimal-merge/input-bob.json` +- `minimal-merge/speakers.yml` + +Run: + +```sh +go run ./cmd/seriatim merge \ + --input-file examples/minimal-merge/input-alice.json \ + --input-file examples/minimal-merge/input-bob.json \ + --speakers examples/minimal-merge/speakers.yml \ + --output-file /tmp/seriatim-example-merge.json +``` + +## Normalize examples + +Object-with-segments input: + +```sh +go run ./cmd/seriatim normalize \ + --input-file examples/normalize/object-with-segments.json \ + --output-file /tmp/seriatim-example-normalize-object.json +``` + +Bare-array input: + +```sh +go run ./cmd/seriatim normalize \ + --input-file examples/normalize/bare-segments-array.json \ + --output-file /tmp/seriatim-example-normalize-array.json +``` + +## Trim example + +Input artifact: + +- `trim/input-full.json` + +Run: + +```sh +go run ./cmd/seriatim trim \ + --input-file examples/trim/input-full.json \ + --output-file /tmp/seriatim-example-trim.json \ + --keep "1-2" +``` + +## YAML rule examples + +- `speakers.yml` +- `autocorrect.yml` diff --git a/examples/autocorrect.yml b/examples/autocorrect.yml new file mode 100644 index 0000000..532b591 --- /dev/null +++ b/examples/autocorrect.yml @@ -0,0 +1,8 @@ +autocorrect: + - target: "General Kenobi" + match: + - "General Kenobi." + + - target: "Okay" + match: + - "Okay." diff --git a/examples/minimal-merge/input-alice.json b/examples/minimal-merge/input-alice.json new file mode 100644 index 0000000..b97d109 --- /dev/null +++ b/examples/minimal-merge/input-alice.json @@ -0,0 +1,14 @@ +{ + "segments": [ + { + "start": 0.0, + "end": 1.2, + "text": " Hello there. " + }, + { + "start": 2.6, + "end": 3.1, + "text": "Okay." + } + ] +} diff --git a/examples/minimal-merge/input-bob.json b/examples/minimal-merge/input-bob.json new file mode 100644 index 0000000..d6e776d --- /dev/null +++ b/examples/minimal-merge/input-bob.json @@ -0,0 +1,9 @@ +{ + "segments": [ + { + "start": 1.3, + "end": 2.4, + "text": "General Kenobi." + } + ] +} diff --git a/examples/minimal-merge/speakers.yml b/examples/minimal-merge/speakers.yml new file mode 100644 index 0000000..438d10c --- /dev/null +++ b/examples/minimal-merge/speakers.yml @@ -0,0 +1,8 @@ +match: + - speaker: "Alice Example" + match: + - "alice" + + - speaker: "Bob Example" + match: + - "bob" diff --git a/examples/normalize/bare-segments-array.json b/examples/normalize/bare-segments-array.json new file mode 100644 index 0000000..72ce7d8 --- /dev/null +++ b/examples/normalize/bare-segments-array.json @@ -0,0 +1,13 @@ +[ + { + "start": 2.5, + "end": 3.0, + "speaker": "Bob", + "text": "later" + }, + { + "end": 2.0, + "speaker": "", + "text": "no start uses end" + } +] diff --git a/examples/normalize/object-with-segments.json b/examples/normalize/object-with-segments.json new file mode 100644 index 0000000..c2e56f9 --- /dev/null +++ b/examples/normalize/object-with-segments.json @@ -0,0 +1,19 @@ +{ + "segments": [ + { + "id": 7, + "start": 2.0, + "end": 2.5, + "speaker": "Bob", + "text": "second" + }, + { + "id": 1, + "start": 1.0, + "end": 1.3, + "speaker": "Alice", + "text": "first", + "categories": ["backchannel"] + } + ] +} diff --git a/examples/speakers.yml b/examples/speakers.yml new file mode 100644 index 0000000..438d10c --- /dev/null +++ b/examples/speakers.yml @@ -0,0 +1,8 @@ +match: + - speaker: "Alice Example" + match: + - "alice" + + - speaker: "Bob Example" + match: + - "bob" diff --git a/examples/trim/input-full.json b/examples/trim/input-full.json new file mode 100644 index 0000000..09a98b8 --- /dev/null +++ b/examples/trim/input-full.json @@ -0,0 +1,64 @@ +{ + "metadata": { + "application": "seriatim", + "version": "dev", + "input_reader": "json-files", + "input_files": [ + "examples/minimal-merge/input-alice.json", + "examples/minimal-merge/input-bob.json" + ], + "preprocessing_modules": [ + "validate-raw", + "normalize-speakers", + "trim-text" + ], + "postprocessing_modules": [ + "detect-overlaps", + "resolve-overlaps", + "backchannel", + "filler", + "resolve-danglers", + "coalesce", + "detect-overlaps", + "autocorrect", + "assign-ids", + "validate-output" + ], + "output_modules": [ + "json" + ] + }, + "segments": [ + { + "id": 1, + "source": "examples/minimal-merge/input-alice.json", + "source_segment_index": 0, + "speaker": "Alice Example", + "start": 0, + "end": 1.2, + "text": "Hello there." + }, + { + "id": 2, + "source": "examples/minimal-merge/input-bob.json", + "source_segment_index": 0, + "speaker": "Bob Example", + "start": 1.3, + "end": 2.4, + "text": "General Kenobi." + }, + { + "id": 3, + "source": "examples/minimal-merge/input-alice.json", + "source_segment_index": 1, + "speaker": "Alice Example", + "start": 2.6, + "end": 3.1, + "text": "Okay.", + "categories": [ + "backchannel" + ] + } + ], + "overlap_groups": [] +}