diff --git a/README.md b/README.md index acb3bbc..fad9ad4 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,8 @@ # seriatim -`seriatim` merges per-speaker WhisperX-style JSON transcripts into a single JSON transcript that preserves speaker identity and chronological order. +`seriatim` merges per-speaker WhisperX-style JSON transcripts into a single JSON transcript that preserves speaker identity and chronological order. It also trims existing seriatim output artifacts by segment ID. -The current implementation supports the `merge` command. It reads one or more input JSON files, optionally maps each input file to a canonical speaker using `speakers.yml`, sorts all segments by timestamp, detects and resolves overlaps when word-level timing is available, assigns consecutive numeric `id` values, and writes a merged JSON artifact. +The current implementation supports the `merge` and `trim` commands. `merge` reads one or more input JSON files, optionally maps each input file to a canonical speaker using `speakers.yml`, sorts all segments by timestamp, detects and resolves overlaps when word-level timing is available, assigns consecutive numeric `id` values, and writes a merged JSON artifact. `trim` reads an existing seriatim output artifact and projects it to a retained segment subset. ## Usage @@ -25,10 +25,20 @@ go run ./cmd/seriatim merge \ --report-file report.json ``` +Trim an existing seriatim artifact: + +```sh +go run ./cmd/seriatim trim \ + --input-file merged.json \ + --output-file trimmed.json \ + --keep "1-10, 15, 20-25" +``` + ## CLI ```text seriatim merge [flags] +seriatim trim [flags] ``` Global flags: @@ -54,6 +64,50 @@ Global flags: | `--postprocessing-modules` | No | `detect-overlaps,resolve-overlaps,backchannel,filler,resolve-danglers,coalesce,detect-overlaps,autocorrect,assign-ids,validate-output` | Comma-separated postprocessing modules, evaluated in order. | | `--coalesce-gap` | No | `3.0` | Maximum same-speaker gap in seconds for `coalesce`; also used as the `resolve-overlaps` context window. Must be a non-negative float. | +`trim` flags: + +| Flag | Required | Default | Description | +| --- | --- | --- | --- | +| `--input-file` | Yes | none | Input seriatim output artifact JSON file. | +| `--output-file` | Yes | none | Trimmed transcript JSON output path. | +| `--keep` | Exactly one of `--keep` or `--remove` is required | none | Segment ID selector to retain. | +| `--remove` | Exactly one of `--keep` or `--remove` is required | none | Segment ID selector to drop. | +| `--output-schema` | No | preserve input artifact schema | Optional output schema override: `seriatim-minimal`, `seriatim-intermediate`, or `seriatim-full`. | +| `--report-file` | No | none | Optional report JSON output path. | +| `--allow-empty` | No | `false` | Allow trimming to zero retained segments. | + +`trim` selection rules: + +- `--keep` and `--remove` are mutually exclusive. +- Exactly one of `--keep` or `--remove` is required. +- Selection is by segment ID only. +- Invalid selected segment IDs fail the command by default. + +`trim` selector syntax: + +- Segment IDs are positive 1-based integers. +- Inclusive ranges are supported: `1-10`. +- Comma-separated selectors are supported: `1-10,15,20-25`. +- Whitespace around numbers, commas, and hyphens is allowed: `1 - 10, 15, 20 - 25`. +- Duplicate and overlapping ranges are accepted and normalized as a union. +- Descending ranges (for example `10-1`) are rejected. + +`trim` behavior: + +- `trim` consumes existing seriatim JSON output artifacts only. +- `trim` does not accept raw WhisperX transcript JSON as input. +- Retained output segment IDs are renumbered sequentially from `1` to `N`. +- Transcript order is preserved from input transcript order; selector order does not reorder output. +- When output schema is `seriatim-full`, overlap groups are recomputed from retained segments. +- `--output-schema seriatim-full` is supported when trim has full-schema artifact data to emit; trim does not synthesize missing full-schema provenance from minimal/intermediate input artifacts. +- `trim` does not run merge postprocessors such as `resolve-overlaps`, `coalesce`, or `autocorrect`. + +`trim` report output: + +- When `--report-file` is provided, the report includes standard trim/validation/output events. +- The report includes a `trim-audit` event containing trim operation metadata, including selected IDs, retained/removed counts, removed IDs, and old-to-new segment ID mapping. +- Old-to-new ID mapping is emitted as a deterministic ordered array of `{old_id, new_id}` pairs. + Environment variables: | Environment Variable | Default | Description | diff --git a/architecture.md b/architecture.md index b1dbfb4..9cff91a 100644 --- a/architecture.md +++ b/architecture.md @@ -1,6 +1,9 @@ # seriatim Architecture -`seriatim` is a deterministic transcript merge utility for combining multiple per-speaker transcript inputs into a single chronologically ordered diarized transcript. +`seriatim` is a deterministic transcript utility for: + +- merging multiple per-speaker transcript inputs into a single chronologically ordered diarized transcript, and +- projecting existing seriatim transcript artifacts through deterministic segment-ID trimming. The initial use case is merging independently transcribed speaker audio tracks from the same recorded session, such as a weekly tabletop RPG session. The architecture should also support meetings, podcasts, interviews, and other multi-speaker events. @@ -20,6 +23,7 @@ The initial use case is merging independently transcribed speaker audio tracks f 8. Detect and annotate overlapping speech regions. 9. Emit one or more output artifacts through output writers. 10. Produce report data for validation findings, corrections, and transformations. +11. Support artifact-level transcript projection commands that operate on existing seriatim output. ## Non-goals @@ -56,6 +60,8 @@ configuration check Each stage has an explicit data contract. Input and output stages perform I/O. Processing stages should be deterministic transformations over in-memory models and should record report events for validation findings, corrections, and transformations. +`merge` runs this pipeline. `trim` is intentionally separate from this pipeline and operates at the artifact layer. + ## Stage Contracts ### 1. Configuration Check @@ -191,6 +197,23 @@ Future output formats may include: Output writers should be selected from an explicit registry and should consume the final transcript model read-only. Multiple output writers may run for a single invocation. +### 7. Artifact Projection Stage (`trim` command) + +`trim` is an artifact-level command that reads an existing seriatim output artifact and emits a projected artifact containing a segment-ID subset. + +Design constraints: + +- `trim` runs after `merge`, not as a merge postprocessor. +- `trim` validates the input artifact against supported seriatim output schemas. +- `trim` performs deterministic keep/remove selection by segment ID. +- `trim` renumbers retained IDs to `1..N` in transcript order. +- `trim` validates the final output against the selected output schema before writing. +- `trim` records audit metadata in report output. + +`trim` is intentionally separate from merge postprocessing because it consumes already-emitted public artifacts. This separation keeps merge semantics stable and avoids rerunning merge-only transforms on projected artifacts. + +`trim` must not rerun merge postprocessors such as `resolve-overlaps`, `coalesce`, or `autocorrect`. + ## Module Classification Modules should be classified by their contract and allowed effects. @@ -397,6 +420,8 @@ A valid merged transcript should satisfy: - Every referenced segment exists. - Output validates against the selected output schema. +For full-schema trim output, overlap groups are recomputed from retained segments so overlap annotations and group references remain internally consistent after projection. + ## Determinism Requirements Given the same inputs, config, and application version, `seriatim` should produce byte-stable JSON output where practical. @@ -411,6 +436,12 @@ To support this: - Record application version in output metadata. - Record enabled module names and module order in output metadata or report data. +Trim-specific determinism requirements: + +- Selector normalization and retained IDs are deterministic. +- Old-to-new ID mapping in trim reports is emitted in deterministic order. +- Full-schema overlap recomputation is deterministic for the same input artifact and selector. + ## Go Package Layout ```text @@ -419,6 +450,7 @@ internal/config/ CLI/env/config loading and validation internal/pipeline/ Pipeline orchestration and module registry internal/builtin/ Built-in pipeline modules internal/artifact/ Conversion from internal model to public output schema +internal/trim/ Artifact parsing, trim selection, schema conversion, overlap recomputation for full schema internal/buildinfo/ Build-time version metadata internal/speaker/ Speaker map parsing and lookup internal/model/ Canonical and merged transcript models @@ -430,6 +462,12 @@ schema/ Public output contract and JSON Schema validation Package boundaries should follow data ownership. Shared models belong in `internal/model`; stage-specific behavior belongs in the relevant stage package. +For trim: + +- `internal/trim` contains pure transformation logic over artifact structs. +- CLI command code handles only flag parsing, file I/O, and report emission. +- Transform logic is deterministic and pure except for command-layer I/O. + ## Default Modules The default pipeline is equivalent to explicit module lists.