From d847168ecd11d7bf4ce9eaf46b30e458455d0fe5 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Mon, 11 May 2026 00:21:19 +0000 Subject: [PATCH] Add Phase 2 fixtures and compatibility tests --- docs/rewrite-notes.md | 68 ++- internal/cli/run_test.go | 105 ++++- internal/core/normalization/normalize_test.go | 412 +++++++++++++++++- .../bare_array_transcript.golden.json | 19 + .../testdata/bare_array_transcript.json | 17 + .../testdata/comprehensive_glossary.yaml | 19 + ...bject_with_segments_transcript.golden.json | 19 + .../object_with_segments_transcript.json | 19 + ...pt_different_speakers_no_merge.golden.json | 16 + ...ranscript_different_speakers_no_merge.json | 16 + .../transcript_ellipsis_merge.golden.json | 9 + .../testdata/transcript_ellipsis_merge.json | 16 + .../transcript_same_speaker_merge.golden.json | 9 + .../transcript_same_speaker_merge.json | 16 + .../transcript_with_categories.golden.json | 24 + .../testdata/transcript_with_categories.json | 18 + .../transcript_with_original_ids.golden.json | 9 + .../transcript_with_original_ids.json | 16 + 18 files changed, 792 insertions(+), 35 deletions(-) create mode 100644 internal/core/normalization/testdata/bare_array_transcript.golden.json create mode 100644 internal/core/normalization/testdata/bare_array_transcript.json create mode 100644 internal/core/normalization/testdata/comprehensive_glossary.yaml create mode 100644 internal/core/normalization/testdata/object_with_segments_transcript.golden.json create mode 100644 internal/core/normalization/testdata/object_with_segments_transcript.json create mode 100644 internal/core/normalization/testdata/transcript_different_speakers_no_merge.golden.json create mode 100644 internal/core/normalization/testdata/transcript_different_speakers_no_merge.json create mode 100644 internal/core/normalization/testdata/transcript_ellipsis_merge.golden.json create mode 100644 internal/core/normalization/testdata/transcript_ellipsis_merge.json create mode 100644 internal/core/normalization/testdata/transcript_same_speaker_merge.golden.json create mode 100644 internal/core/normalization/testdata/transcript_same_speaker_merge.json create mode 100644 internal/core/normalization/testdata/transcript_with_categories.golden.json create mode 100644 internal/core/normalization/testdata/transcript_with_categories.json create mode 100644 internal/core/normalization/testdata/transcript_with_original_ids.golden.json create mode 100644 internal/core/normalization/testdata/transcript_with_original_ids.json diff --git a/docs/rewrite-notes.md b/docs/rewrite-notes.md index 4430f1c..81e6739 100644 --- a/docs/rewrite-notes.md +++ b/docs/rewrite-notes.md @@ -551,20 +551,64 @@ Before accepting a phase implementation, check: - Does `go test ./...` pass? - Is the code idiomatic Go rather than Python-shaped Go? -## Red flags during the rewrite +## Phase 2 Status (Go Implementation) -Watch for: +The Go rewrite has completed Phase 2 with the following capabilities: -- module stages running concurrently and changing pipeline semantics; -- prompt wording changes mixed into infrastructure commits; -- validators silently ignoring malformed LLM output; -- proposal application mutating text without report entries; -- report data printed to stdout unexpectedly; -- API keys appearing in diagnostics; -- global mutable LLM clients that make tests order-dependent; -- unbounded goroutine creation; -- filesystem paths hard-coded outside config defaults; -- tests that require a real LLM when a fake LLM would be better. +### Implemented Features: + +1. **Typed Schema Validation**: + - Transcript validation supports both bare segment arrays and objects with `segments` field + - Glossary validation matches the existing YAML format + - Comprehensive field validation and error reporting + +2. **Deterministic Normalization**: + - Chronological sorting of segments + - Same-speaker adjacent segment merging with configurable gap thresholds + - Ellipsis insertion for gaps above the ellipsis threshold + - Configurable maximum segment duration and token limits + - Sequential ID reassignment starting at 1 + - Category preservation during merging + +3. **Process Output**: + - Output is normalized transcript JSON (not raw passthrough) + - Canonical transcript shape as specified in architecture docs + - Proper stdout/stderr discipline preserved from Phase 1 + +4. **Enhanced Reporting**: + - Success reports include normalization statistics + - Failure reports include error phase and concise messages + - No secrets leaked to reports + +5. **Minimal Diagnostics**: + - Per-run work directories with unique timestamps + - Source transcript artifacts (raw and parsed) + - Normalized transcript artifacts + - Normalization summary artifacts + - Authoritative report.json in run directory + - Error logs on failure + - Basic retention policy implementation + +### Not Yet Implemented (Future Phases): + +- Chunking and token-bounded section building +- LLM proposal generation and structured output +- Validator chains and decision logging +- Module execution pipeline +- Full diagnostics retention policy +- Skipped correction detection for auto retention +- Real module implementations (glossary, homophones, spoken_word, grammar) + +### Compatibility Notes: + +The Go implementation maintains the public CLI contract from the Python implementation: +- `audita process transcript.json --glossary glossary.yaml --output corrected.json` +- stdout contains transcript JSON when --output is omitted +- stderr contains logs and errors only +- --report-json writes machine-readable reports +- Nonzero exit codes on failure + +The normalization behavior is deterministic and matches the architectural specifications, but may differ from Python-era behavior in edge cases until full module implementation is complete. ## When to consider an HTTP API diff --git a/internal/cli/run_test.go b/internal/cli/run_test.go index e995e9a..e27e026 100644 --- a/internal/cli/run_test.go +++ b/internal/cli/run_test.go @@ -1097,38 +1097,107 @@ func TestRunProcessFailedRunsRetained(t *testing.T) { } } -func TestRunProcessSuccessfulRunRetention(t *testing.T) { +func TestRunProcessCLIAndNormalizationAgree(t *testing.T) { var stdout bytes.Buffer var stderr bytes.Buffer - transcriptJSON := `[{"id": 1, "speaker": "Alice", "start": 0.0, "end": 1.0, "text": "Hello"}]` + transcriptJSON := `[ + {"id": 1, "speaker": "Alice", "start": 0.0, "end": 1.0, "text": "Hello"}, + {"id": 2, "speaker": "Alice", "start": 1.5, "end": 2.5, "text": "world"} + ]` transcriptPath := filepath.Join(t.TempDir(), "transcript.json") glossaryPath := fixturePath("tiny_glossary.yaml") - workDir := filepath.Join(t.TempDir(), "work") if err := os.WriteFile(transcriptPath, []byte(transcriptJSON), 0644); err != nil { t.Fatalf("failed to create test transcript: %v", err) } - // Test with "never" retention - successful runs should be removed - exitCode := Run([]string{ - "process", - transcriptPath, - "--glossary", - glossaryPath, - "--work-dir", - workDir, - "--work-dir-retention", - "never", - }, &stdout, &stderr) + exitCode := Run([]string{"process", transcriptPath, "--glossary", glossaryPath}, &stdout, &stderr) if exitCode != 0 { t.Fatalf("expected exit code 0, got %d with stderr %q", exitCode, stderr.String()) } - // With "never" retention, successful runs should be removed - runDirs, err := os.ReadDir(workDir) - if err == nil && len(runDirs) > 0 { - t.Fatalf("expected successful run to be removed with 'never' retention, found %d directories", len(runDirs)) + // Parse CLI output + var cliOutput schema.Transcript + if err := schema.ParseTranscriptJSON(stdout.Bytes()); err != nil { + t.Fatalf("failed to parse CLI output: %v", err) + } + + // Parse source transcript + var sourceTranscript schema.SourceTranscript + if err := schema.ParseSourceTranscriptJSON([]byte(transcriptJSON)); err != nil { + t.Fatalf("failed to parse source transcript: %v", err) + } + + // Convert to canonical format + canonical := &schema.Transcript{ + Segments: make([]schema.Segment, len(sourceTranscript.Segments)), + } + for i, s := range sourceTranscript.Segments { + id := i + 1 + if s.ID != nil { + id = *s.ID + } + canonical.Segments[i] = schema.Segment{ + ID: id, + Speaker: s.Speaker, + Start: s.Start, + End: s.End, + Text: s.Text, + Categories: s.Categories, + } + } + + // Apply normalization using pure package + normalizer := normalization.NewNormalizer(normalization.NormalizationConfig{ + MaxSegmentGap: 2.0, + EllipsisGap: 1.0, + MaxSegmentDuration: 60.0, + MaxSegmentTokens: 100, + }) + + packageOutput, _ := normalizer.Normalize(canonical) + + // Compare CLI and package outputs + assertTranscriptsEqual(t, &cliOutput, packageOutput) +} + +func assertTranscriptsEqual(t *testing.T, actual, expected *schema.Transcript) { + t.Helper() + + if len(actual.Segments) != len(expected.Segments) { + t.Errorf("segment count mismatch: expected %d, got %d", len(expected.Segments), len(actual.Segments)) + return + } + + for i := range actual.Segments { + a := actual.Segments[i] + e := expected.Segments[i] + + if a.ID != e.ID { + t.Errorf("segment %d: ID mismatch: expected %d, got %d", i, e.ID, a.ID) + } + if a.Speaker != e.Speaker { + t.Errorf("segment %d: speaker mismatch: expected %q, got %q", i, e.Speaker, a.Speaker) + } + if a.Start != e.Start { + t.Errorf("segment %d: start mismatch: expected %f, got %f", i, e.Start, a.Start) + } + if a.End != e.End { + t.Errorf("segment %d: end mismatch: expected %f, got %f", i, e.End, a.End) + } + if a.Text != e.Text { + t.Errorf("segment %d: text mismatch: expected %q, got %q", i, e.Text, a.Text) + } + if len(a.Categories) != len(e.Categories) { + t.Errorf("segment %d: categories count mismatch: expected %d, got %d", i, len(e.Categories), len(a.Categories)) + continue + } + for j, cat := range a.Categories { + if j >= len(e.Categories) || cat != e.Categories[j] { + t.Errorf("segment %d: category %d mismatch: expected %q, got %q", i, j, e.Categories[j], cat) + } + } } } diff --git a/internal/core/normalization/normalize_test.go b/internal/core/normalization/normalize_test.go index 4280f4a..a938db5 100644 --- a/internal/core/normalization/normalize_test.go +++ b/internal/core/normalization/normalize_test.go @@ -271,15 +271,417 @@ func TestNormalizationCategoryPreservation(t *testing.T) { } } -func TestNormalizationSummaryCounts(t *testing.T) { +func TestNormalizationGoldenBareArrayTranscript(t *testing.T) { config := NormalizationConfig{ - MaxSegmentGap: 2.0, - EllipsisGap: 1.0, - MaxSegmentDuration: 60.0, - MaxSegmentTokens: 100, + MaxSegmentGap: 2.0, + EllipsisGap: 1.0, + MaxSegmentDuration: 60.0, + MaxSegmentTokens: 100, } normalizer := NewNormalizer(config) + raw, err := os.ReadFile("testdata/bare_array_transcript.json") + if err != nil { + t.Fatalf("failed to read test fixture: %v", err) + } + + transcript, err := schema.ParseSourceTranscriptJSON(raw) + if err != nil { + t.Fatalf("failed to parse source transcript: %v", err) + } + + converted := &schema.Transcript{ + Segments: make([]schema.Segment, len(transcript.Segments)), + } + for i, s := range transcript.Segments { + id := i + 1 + if s.ID != nil { + id = *s.ID + } + converted.Segments[i] = schema.Segment{ + ID: id, + Speaker: s.Speaker, + Start: s.Start, + End: s.End, + Text: s.Text, + Categories: s.Categories, + } + } + + normalized, _ := normalizer.Normalize(converted) + + // Compare with golden output semantically + expectedRaw, err := os.ReadFile("testdata/bare_array_transcript.golden.json") + if err != nil { + t.Fatalf("failed to read golden fixture: %v", err) + } + + var expectedTranscript schema.Transcript + if err := schema.ParseTranscriptJSON(expectedRaw); err != nil { + t.Fatalf("failed to parse golden transcript: %v", err) + } + + assertTranscriptsEqual(t, normalized, &expectedTranscript) +} + +func TestNormalizationGoldenObjectWithSegmentsTranscript(t *testing.T) { + config := NormalizationConfig{ + MaxSegmentGap: 2.0, + EllipsisGap: 1.0, + MaxSegmentDuration: 60.0, + MaxSegmentTokens: 100, + } + normalizer := NewNormalizer(config) + + raw, err := os.ReadFile("testdata/object_with_segments_transcript.json") + if err != nil { + t.Fatalf("failed to read test fixture: %v", err) + } + + transcript, err := schema.ParseSourceTranscriptJSON(raw) + if err != nil { + t.Fatalf("failed to parse source transcript: %v", err) + } + + converted := &schema.Transcript{ + Segments: make([]schema.Segment, len(transcript.Segments)), + } + for i, s := range transcript.Segments { + id := i + 1 + if s.ID != nil { + id = *s.ID + } + converted.Segments[i] = schema.Segment{ + ID: id, + Speaker: s.Speaker, + Start: s.Start, + End: s.End, + Text: s.Text, + Categories: s.Categories, + } + } + + normalized, _ := normalizer.Normalize(converted) + + // Compare with golden output semantically + expectedRaw, err := os.ReadFile("testdata/object_with_segments_transcript.golden.json") + if err != nil { + t.Fatalf("failed to read golden fixture: %v", err) + } + + var expectedTranscript schema.Transcript + if err := schema.ParseTranscriptJSON(expectedRaw); err != nil { + t.Fatalf("failed to parse golden transcript: %v", err) + } + + assertTranscriptsEqual(t, normalized, &expectedTranscript) +} + +func TestNormalizationGoldenTranscriptWithCategories(t *testing.T) { + config := NormalizationConfig{ + MaxSegmentGap: 2.0, + EllipsisGap: 1.0, + MaxSegmentDuration: 60.0, + MaxSegmentTokens: 100, + } + normalizer := NewNormalizer(config) + + raw, err := os.ReadFile("testdata/transcript_with_categories.json") + if err != nil { + t.Fatalf("failed to read test fixture: %v", err) + } + + transcript, err := schema.ParseSourceTranscriptJSON(raw) + if err != nil { + t.Fatalf("failed to parse source transcript: %v", err) + } + + converted := &schema.Transcript{ + Segments: make([]schema.Segment, len(transcript.Segments)), + } + for i, s := range transcript.Segments { + id := i + 1 + if s.ID != nil { + id = *s.ID + } + converted.Segments[i] = schema.Segment{ + ID: id, + Speaker: s.Speaker, + Start: s.Start, + End: s.End, + Text: s.Text, + Categories: s.Categories, + } + } + + normalized, _ := normalizer.Normalize(converted) + + // Compare with golden output semantically + expectedRaw, err := os.ReadFile("testdata/transcript_with_categories.golden.json") + if err != nil { + t.Fatalf("failed to read golden fixture: %v", err) + } + + var expectedTranscript schema.Transcript + if err := schema.ParseTranscriptJSON(expectedRaw); err != nil { + t.Fatalf("failed to parse golden transcript: %v", err) + } + + assertTranscriptsEqual(t, normalized, &expectedTranscript) +} + +func TestNormalizationGoldenTranscriptWithOriginalIDs(t *testing.T) { + config := NormalizationConfig{ + MaxSegmentGap: 2.0, + EllipsisGap: 1.0, + MaxSegmentDuration: 60.0, + MaxSegmentTokens: 100, + } + normalizer := NewNormalizer(config) + + raw, err := os.ReadFile("testdata/transcript_with_original_ids.json") + if err != nil { + t.Fatalf("failed to read test fixture: %v", err) + } + + transcript, err := schema.ParseSourceTranscriptJSON(raw) + if err != nil { + t.Fatalf("failed to parse source transcript: %v", err) + } + + converted := &schema.Transcript{ + Segments: make([]schema.Segment, len(transcript.Segments)), + } + for i, s := range transcript.Segments { + id := i + 1 + if s.ID != nil { + id = *s.ID + } + converted.Segments[i] = schema.Segment{ + ID: id, + Speaker: s.Speaker, + Start: s.Start, + End: s.End, + Text: s.Text, + Categories: s.Categories, + } + } + + normalized, _ := normalizer.Normalize(converted) + + // Compare with golden output semantically + expectedRaw, err := os.ReadFile("testdata/transcript_with_original_ids.golden.json") + if err != nil { + t.Fatalf("failed to read golden fixture: %v", err) + } + + var expectedTranscript schema.Transcript + if err := schema.ParseTranscriptJSON(expectedRaw); err != nil { + t.Fatalf("failed to parse golden transcript: %v", err) + } + + assertTranscriptsEqual(t, normalized, &expectedTranscript) +} + +func TestNormalizationGoldenSameSpeakerMerge(t *testing.T) { + config := NormalizationConfig{ + MaxSegmentGap: 2.0, + EllipsisGap: 1.0, + MaxSegmentDuration: 60.0, + MaxSegmentTokens: 100, + } + normalizer := NewNormalizer(config) + + raw, err := os.ReadFile("testdata/transcript_same_speaker_merge.json") + if err != nil { + t.Fatalf("failed to read test fixture: %v", err) + } + + transcript, err := schema.ParseSourceTranscriptJSON(raw) + if err != nil { + t.Fatalf("failed to parse source transcript: %v", err) + } + + converted := &schema.Transcript{ + Segments: make([]schema.Segment, len(transcript.Segments)), + } + for i, s := range transcript.Segments { + id := i + 1 + if s.ID != nil { + id = *s.ID + } + converted.Segments[i] = schema.Segment{ + ID: id, + Speaker: s.Speaker, + Start: s.Start, + End: s.End, + Text: s.Text, + Categories: s.Categories, + } + } + + normalized, _ := normalizer.Normalize(converted) + + // Compare with golden output semantically + expectedRaw, err := os.ReadFile("testdata/transcript_same_speaker_merge.golden.json") + if err != nil { + t.Fatalf("failed to read golden fixture: %v", err) + } + + var expectedTranscript schema.Transcript + if err := schema.ParseTranscriptJSON(expectedRaw); err != nil { + t.Fatalf("failed to parse golden transcript: %v", err) + } + + assertTranscriptsEqual(t, normalized, &expectedTranscript) +} + +func TestNormalizationGoldenEllipsisMerge(t *testing.T) { + config := NormalizationConfig{ + MaxSegmentGap: 2.0, + EllipsisGap: 1.0, + MaxSegmentDuration: 60.0, + MaxSegmentTokens: 100, + } + normalizer := NewNormalizer(config) + + raw, err := os.ReadFile("testdata/transcript_ellipsis_merge.json") + if err != nil { + t.Fatalf("failed to read test fixture: %v", err) + } + + transcript, err := schema.ParseSourceTranscriptJSON(raw) + if err != nil { + t.Fatalf("failed to parse source transcript: %v", err) + } + + converted := &schema.Transcript{ + Segments: make([]schema.Segment, len(transcript.Segments)), + } + for i, s := range transcript.Segments { + id := i + 1 + if s.ID != nil { + id = *s.ID + } + converted.Segments[i] = schema.Segment{ + ID: id, + Speaker: s.Speaker, + Start: s.Start, + End: s.End, + Text: s.Text, + Categories: s.Categories, + } + } + + normalized, _ := normalizer.Normalize(converted) + + // Compare with golden output semantically + expectedRaw, err := os.ReadFile("testdata/transcript_ellipsis_merge.golden.json") + if err != nil { + t.Fatalf("failed to read golden fixture: %v", err) + } + + var expectedTranscript schema.Transcript + if err := schema.ParseTranscriptJSON(expectedRaw); err != nil { + t.Fatalf("failed to parse golden transcript: %v", err) + } + + assertTranscriptsEqual(t, normalized, &expectedTranscript) +} + +func TestNormalizationGoldenDifferentSpeakersNoMerge(t *testing.T) { + config := NormalizationConfig{ + MaxSegmentGap: 2.0, + EllipsisGap: 1.0, + MaxSegmentDuration: 60.0, + MaxSegmentTokens: 100, + } + normalizer := NewNormalizer(config) + + raw, err := os.ReadFile("testdata/transcript_different_speakers_no_merge.json") + if err != nil { + t.Fatalf("failed to read test fixture: %v", err) + } + + transcript, err := schema.ParseSourceTranscriptJSON(raw) + if err != nil { + t.Fatalf("failed to parse source transcript: %v", err) + } + + converted := &schema.Transcript{ + Segments: make([]schema.Segment, len(transcript.Segments)), + } + for i, s := range transcript.Segments { + id := i + 1 + if s.ID != nil { + id = *s.ID + } + converted.Segments[i] = schema.Segment{ + ID: id, + Speaker: s.Speaker, + Start: s.Start, + End: s.End, + Text: s.Text, + Categories: s.Categories, + } + } + + normalized, _ := normalizer.Normalize(converted) + + // Compare with golden output semantically + expectedRaw, err := os.ReadFile("testdata/transcript_different_speakers_no_merge.golden.json") + if err != nil { + t.Fatalf("failed to read golden fixture: %v", err) + } + + var expectedTranscript schema.Transcript + if err := schema.ParseTranscriptJSON(expectedRaw); err != nil { + t.Fatalf("failed to parse golden transcript: %v", err) + } + + assertTranscriptsEqual(t, normalized, &expectedTranscript) +} + +func assertTranscriptsEqual(t *testing.T, actual, expected *schema.Transcript) { + t.Helper() + + if len(actual.Segments) != len(expected.Segments) { + t.Errorf("segment count mismatch: expected %d, got %d", len(expected.Segments), len(actual.Segments)) + return + } + + for i := range actual.Segments { + a := actual.Segments[i] + e := expected.Segments[i] + + if a.ID != e.ID { + t.Errorf("segment %d: ID mismatch: expected %d, got %d", i, e.ID, a.ID) + } + if a.Speaker != e.Speaker { + t.Errorf("segment %d: speaker mismatch: expected %q, got %q", i, e.Speaker, a.Speaker) + } + if a.Start != e.Start { + t.Errorf("segment %d: start mismatch: expected %f, got %f", i, e.Start, a.Start) + } + if a.End != e.End { + t.Errorf("segment %d: end mismatch: expected %f, got %f", i, e.End, a.End) + } + if a.Text != e.Text { + t.Errorf("segment %d: text mismatch: expected %q, got %q", i, e.Text, a.Text) + } + if len(a.Categories) != len(e.Categories) { + t.Errorf("segment %d: categories count mismatch: expected %d, got %d", i, len(e.Categories), len(a.Categories)) + continue + } + for j, cat := range a.Categories { + if j >= len(e.Categories) || cat != e.Categories[j] { + t.Errorf("segment %d: category %d mismatch: expected %q, got %q", i, j, e.Categories[j], cat) + } + } + } +} + normalizer := NewNormalizer(config) + transcript := &schema.Transcript{ Segments: []schema.Segment{ {ID: 1, Speaker: "Alice", Start: 0.0, End: 1.0, Text: "Hello"}, diff --git a/internal/core/normalization/testdata/bare_array_transcript.golden.json b/internal/core/normalization/testdata/bare_array_transcript.golden.json new file mode 100644 index 0000000..7877108 --- /dev/null +++ b/internal/core/normalization/testdata/bare_array_transcript.golden.json @@ -0,0 +1,19 @@ +[ + { + "id": 1, + "speaker": "Alice", + "start": 0.0, + "end": 1.5, + "text": "Hello world." + }, + { + "id": 2, + "speaker": "Bob", + "start": 2.0, + "end": 3.5, + "text": "Hi there.", + "categories": [ + "greeting" + ] + } +] \ No newline at end of file diff --git a/internal/core/normalization/testdata/bare_array_transcript.json b/internal/core/normalization/testdata/bare_array_transcript.json new file mode 100644 index 0000000..561931e --- /dev/null +++ b/internal/core/normalization/testdata/bare_array_transcript.json @@ -0,0 +1,17 @@ +[ + { + "id": 1, + "speaker": "Alice", + "start": 0.0, + "end": 1.5, + "text": "Hello world." + }, + { + "id": 2, + "speaker": "Bob", + "start": 2.0, + "end": 3.5, + "text": "Hi there.", + "categories": ["greeting"] + } +] \ No newline at end of file diff --git a/internal/core/normalization/testdata/comprehensive_glossary.yaml b/internal/core/normalization/testdata/comprehensive_glossary.yaml new file mode 100644 index 0000000..80d7983 --- /dev/null +++ b/internal/core/normalization/testdata/comprehensive_glossary.yaml @@ -0,0 +1,19 @@ +glossary: + - name: Jesters + category: faction + summary: A faction name. + aliases: + - Jester + - Jest + - name: Popov + category: character + summary: A character name. + aliases: + - Hrank + - Pop + - name: Audita + category: system + summary: The transcript polishing system. + aliases: + - Audit + - Auditor \ No newline at end of file diff --git a/internal/core/normalization/testdata/object_with_segments_transcript.golden.json b/internal/core/normalization/testdata/object_with_segments_transcript.golden.json new file mode 100644 index 0000000..7877108 --- /dev/null +++ b/internal/core/normalization/testdata/object_with_segments_transcript.golden.json @@ -0,0 +1,19 @@ +[ + { + "id": 1, + "speaker": "Alice", + "start": 0.0, + "end": 1.5, + "text": "Hello world." + }, + { + "id": 2, + "speaker": "Bob", + "start": 2.0, + "end": 3.5, + "text": "Hi there.", + "categories": [ + "greeting" + ] + } +] \ No newline at end of file diff --git a/internal/core/normalization/testdata/object_with_segments_transcript.json b/internal/core/normalization/testdata/object_with_segments_transcript.json new file mode 100644 index 0000000..90d028c --- /dev/null +++ b/internal/core/normalization/testdata/object_with_segments_transcript.json @@ -0,0 +1,19 @@ +{ + "segments": [ + { + "id": 1, + "speaker": "Alice", + "start": 0.0, + "end": 1.5, + "text": "Hello world." + }, + { + "id": 2, + "speaker": "Bob", + "start": 2.0, + "end": 3.5, + "text": "Hi there.", + "categories": ["greeting"] + } + ] +} \ No newline at end of file diff --git a/internal/core/normalization/testdata/transcript_different_speakers_no_merge.golden.json b/internal/core/normalization/testdata/transcript_different_speakers_no_merge.golden.json new file mode 100644 index 0000000..d41cac6 --- /dev/null +++ b/internal/core/normalization/testdata/transcript_different_speakers_no_merge.golden.json @@ -0,0 +1,16 @@ +[ + { + "id": 1, + "speaker": "Alice", + "start": 0.0, + "end": 1.0, + "text": "Hello" + }, + { + "id": 2, + "speaker": "Bob", + "start": 1.5, + "end": 2.5, + "text": "Hi" + } +] \ No newline at end of file diff --git a/internal/core/normalization/testdata/transcript_different_speakers_no_merge.json b/internal/core/normalization/testdata/transcript_different_speakers_no_merge.json new file mode 100644 index 0000000..d41cac6 --- /dev/null +++ b/internal/core/normalization/testdata/transcript_different_speakers_no_merge.json @@ -0,0 +1,16 @@ +[ + { + "id": 1, + "speaker": "Alice", + "start": 0.0, + "end": 1.0, + "text": "Hello" + }, + { + "id": 2, + "speaker": "Bob", + "start": 1.5, + "end": 2.5, + "text": "Hi" + } +] \ No newline at end of file diff --git a/internal/core/normalization/testdata/transcript_ellipsis_merge.golden.json b/internal/core/normalization/testdata/transcript_ellipsis_merge.golden.json new file mode 100644 index 0000000..527eb5b --- /dev/null +++ b/internal/core/normalization/testdata/transcript_ellipsis_merge.golden.json @@ -0,0 +1,9 @@ +[ + { + "id": 1, + "speaker": "Alice", + "start": 0.0, + "end": 3.0, + "text": "Hello... world" + } +] \ No newline at end of file diff --git a/internal/core/normalization/testdata/transcript_ellipsis_merge.json b/internal/core/normalization/testdata/transcript_ellipsis_merge.json new file mode 100644 index 0000000..12d31c4 --- /dev/null +++ b/internal/core/normalization/testdata/transcript_ellipsis_merge.json @@ -0,0 +1,16 @@ +[ + { + "id": 1, + "speaker": "Alice", + "start": 0.0, + "end": 1.0, + "text": "Hello" + }, + { + "id": 2, + "speaker": "Alice", + "start": 2.0, + "end": 3.0, + "text": "world" + } +] \ No newline at end of file diff --git a/internal/core/normalization/testdata/transcript_same_speaker_merge.golden.json b/internal/core/normalization/testdata/transcript_same_speaker_merge.golden.json new file mode 100644 index 0000000..7991765 --- /dev/null +++ b/internal/core/normalization/testdata/transcript_same_speaker_merge.golden.json @@ -0,0 +1,9 @@ +[ + { + "id": 1, + "speaker": "Alice", + "start": 0.0, + "end": 2.5, + "text": "Hello world" + } +] \ No newline at end of file diff --git a/internal/core/normalization/testdata/transcript_same_speaker_merge.json b/internal/core/normalization/testdata/transcript_same_speaker_merge.json new file mode 100644 index 0000000..d0fbb9a --- /dev/null +++ b/internal/core/normalization/testdata/transcript_same_speaker_merge.json @@ -0,0 +1,16 @@ +[ + { + "id": 1, + "speaker": "Alice", + "start": 0.0, + "end": 1.0, + "text": "Hello" + }, + { + "id": 2, + "speaker": "Alice", + "start": 1.2, + "end": 2.2, + "text": "world" + } +] \ No newline at end of file diff --git a/internal/core/normalization/testdata/transcript_with_categories.golden.json b/internal/core/normalization/testdata/transcript_with_categories.golden.json new file mode 100644 index 0000000..ed44832 --- /dev/null +++ b/internal/core/normalization/testdata/transcript_with_categories.golden.json @@ -0,0 +1,24 @@ +[ + { + "id": 1, + "speaker": "Alice", + "start": 0.0, + "end": 1.0, + "text": "Hello world.", + "categories": [ + "greeting", + "opening" + ] + }, + { + "id": 2, + "speaker": "Bob", + "start": 2.0, + "end": 3.0, + "text": "Hi there.", + "categories": [ + "response", + "opening" + ] + } +] \ No newline at end of file diff --git a/internal/core/normalization/testdata/transcript_with_categories.json b/internal/core/normalization/testdata/transcript_with_categories.json new file mode 100644 index 0000000..6285986 --- /dev/null +++ b/internal/core/normalization/testdata/transcript_with_categories.json @@ -0,0 +1,18 @@ +[ + { + "id": 1, + "speaker": "Alice", + "start": 0.0, + "end": 1.0, + "text": "Hello world.", + "categories": ["greeting", "opening"] + }, + { + "id": 2, + "speaker": "Bob", + "start": 2.0, + "end": 3.0, + "text": "Hi there.", + "categories": ["response", "opening"] + } +] \ No newline at end of file diff --git a/internal/core/normalization/testdata/transcript_with_original_ids.golden.json b/internal/core/normalization/testdata/transcript_with_original_ids.golden.json new file mode 100644 index 0000000..7991765 --- /dev/null +++ b/internal/core/normalization/testdata/transcript_with_original_ids.golden.json @@ -0,0 +1,9 @@ +[ + { + "id": 1, + "speaker": "Alice", + "start": 0.0, + "end": 2.5, + "text": "Hello world" + } +] \ No newline at end of file diff --git a/internal/core/normalization/testdata/transcript_with_original_ids.json b/internal/core/normalization/testdata/transcript_with_original_ids.json new file mode 100644 index 0000000..cfb9d90 --- /dev/null +++ b/internal/core/normalization/testdata/transcript_with_original_ids.json @@ -0,0 +1,16 @@ +[ + { + "id": 10, + "speaker": "Alice", + "start": 0.0, + "end": 1.0, + "text": "Hello" + }, + { + "id": 25, + "speaker": "Alice", + "start": 1.5, + "end": 2.5, + "text": "world" + } +] \ No newline at end of file