147 lines
4.2 KiB
Markdown
147 lines
4.2 KiB
Markdown
# Artifact Internals
|
|
|
|
## Purpose
|
|
|
|
Describes public artifact conversion and validation internals for merge output,
|
|
trim, and normalize.
|
|
|
|
## Artifact contracts
|
|
|
|
Public contracts live in `schema/`:
|
|
|
|
- full: `schema.Transcript`
|
|
- intermediate: `schema.IntermediateTranscript`
|
|
- minimal: `schema.MinimalTranscript`
|
|
|
|
Machine-readable schemas:
|
|
|
|
- `schema/full-output.schema.json`
|
|
- `schema/intermediate-output.schema.json`
|
|
- `schema/minimal-output.schema.json`
|
|
|
|
## Schema selection
|
|
|
|
Merge pipeline conversion uses `internal/artifact.SelectedFromMerged`:
|
|
|
|
- `seriatim-full` -> `artifact.FromMerged`
|
|
- `seriatim-intermediate` -> `artifact.IntermediateFromMerged`
|
|
- `seriatim-minimal` -> `artifact.MinimalFromMerged`
|
|
|
|
Unknown/empty selection falls back to intermediate conversion.
|
|
|
|
## Merge conversion behavior
|
|
|
|
`internal/artifact` converts `model.MergedTranscript` to public contracts:
|
|
|
|
- full schema preserves source/provenance, overlap groups, and metadata module
|
|
lists.
|
|
- intermediate schema emits segment timing/text/speaker with optional
|
|
categories and compact metadata.
|
|
- minimal schema emits compact segment timing/text/speaker and compact
|
|
metadata.
|
|
|
|
## Validation behavior
|
|
|
|
`schema/output.go` validates both structure and semantics:
|
|
|
|
- embedded JSON Schema validation via `jsonschema/v6`
|
|
- semantic checks for sequential segment IDs starting at `1`
|
|
- semantic checks for non-inverted segment timing (`end >= start`)
|
|
- full schema overlap-group timing checks (`group.end >= group.start`)
|
|
|
|
## Trim internals
|
|
|
|
`internal/trim` is artifact-level projection, not merge reprocessing.
|
|
|
|
Core flow:
|
|
|
|
1. Parse selector (`internal/trim/selector.go`).
|
|
2. Parse input artifact and detect schema (`ParseArtifactJSON`).
|
|
3. Apply keep/remove projection with sequential ID renumbering.
|
|
4. Recompute overlap groups only for full-schema artifacts.
|
|
5. Optionally convert output schema when supported.
|
|
6. Validate output artifact before write.
|
|
|
|
Schema-conversion limits:
|
|
|
|
- full -> intermediate/minimal supported.
|
|
- intermediate -> minimal supported.
|
|
- minimal -> intermediate supported.
|
|
- intermediate/minimal -> full is rejected.
|
|
|
|
Trim invariants:
|
|
|
|
- selected IDs must exist in input.
|
|
- input IDs must be positive, unique, sequential.
|
|
- retained order follows input transcript order.
|
|
- output IDs are reassigned to `1..N`.
|
|
|
|
Run layer (`run.go`):
|
|
|
|
- executes end-to-end trim orchestration from validated config
|
|
- writes output JSON
|
|
- optionally writes report JSON with `trim-audit`
|
|
|
|
## Normalize internals
|
|
|
|
`internal/normalize` canonicalizes transcript-like JSON input into a selected
|
|
public schema.
|
|
|
|
Parse layer (`parse.go`):
|
|
|
|
- accepts object-with-`segments` or bare segment array
|
|
- repairs missing timing deterministically
|
|
- swaps inverted timing
|
|
- fills missing/blank speaker with `Unknown_Speaker`
|
|
- drops missing/blank text segments
|
|
|
|
Build layer (`build.go`):
|
|
|
|
- sorts deterministically by `(start, end, input_index, speaker)`
|
|
- reassigns output IDs sequentially
|
|
- builds minimal/intermediate/full output shape
|
|
- validates selected output schema before write
|
|
|
|
Run layer (`normalize.go`):
|
|
|
|
- writes output JSON
|
|
- optionally writes report with `normalize-audit`
|
|
|
|
Normalize invariant:
|
|
|
|
- report events do not embed transcript text.
|
|
|
|
## Boundaries
|
|
|
|
- CLI flag semantics belong to `docs/cli.md`.
|
|
- Runtime config/env surfaces belong to `docs/config.md`.
|
|
- This doc describes internal conversion/validation behavior only.
|
|
|
|
## Failure behavior
|
|
|
|
Representative failure classes:
|
|
|
|
- malformed or unsupported input JSON shape
|
|
- schema validation failure for parsed artifact or built output
|
|
- unsupported schema conversion path (trim)
|
|
- selector or input-ID consistency errors (trim)
|
|
- output/report file write failures from command paths
|
|
|
|
## Tests to inspect before changes
|
|
|
|
- `schema/output_test.go`
|
|
- `internal/artifact/transcript_test.go`
|
|
- `internal/trim/selector_test.go`
|
|
- `internal/trim/artifact_test.go`
|
|
- `internal/trim/apply_test.go`
|
|
- `internal/normalize/parse_test.go`
|
|
- `internal/cli/trim_test.go`
|
|
- `internal/cli/normalize_test.go`
|
|
|
|
## Invariants
|
|
|
|
- Public artifacts are validated through `schema` before acceptance.
|
|
- Segment IDs in emitted artifacts are sequential and deterministic.
|
|
- Internal-only fields are not emitted in minimal/intermediate contracts.
|
|
- Trim and normalize stay artifact-level and do not execute merge modules.
|