Protect transcript refinement resume semantics
This commit is contained in:
@@ -207,6 +207,13 @@ runner marks the aggregate record stale and executes it. Analyze's validator
|
||||
can still accept a partial selection when only unrelated artifact records are
|
||||
stale.
|
||||
|
||||
The current production semantic contracts cover prepare, transcribe, merge,
|
||||
polish, normalize, trim, and render. They record resolved choices Narratio can
|
||||
observe, not operational runner tuning. External model, module, prompt,
|
||||
profile, and configuration-file contents that a tool privately loads remain
|
||||
outside the manifest contract when their configured identifier is unchanged;
|
||||
operators must force the affected stage after such a private content change.
|
||||
|
||||
Session manifest is the authoritative stage-progress ledger across invocations.
|
||||
Run manifest is invocation-scoped audit state.
|
||||
|
||||
|
||||
@@ -26,9 +26,17 @@ Normalize polished transcript into final transcript using Seriatim.
|
||||
- final transcript must validate as processed transcript JSON (`segments` array).
|
||||
- normalize defaults are applied when `pipeline.normalize` is unset.
|
||||
|
||||
## Resume Semantics
|
||||
|
||||
The versioned semantic fingerprint covers the Seriatim normalize operation,
|
||||
output schema and canonical output identity, plus the configured transcript
|
||||
transformations. Seriatim's executable and timeout and optional report
|
||||
generation are operational and do not invalidate the normalized transcript.
|
||||
|
||||
## Related Contracts And Tests
|
||||
|
||||
- [Seriatim](../integrations/seriatim.md) owns subprocess and output semantics.
|
||||
- [Configuration](../config.md#pipeline) owns normalize fields and defaults.
|
||||
- Implementation and tests: `internal/stage/normalize.go`,
|
||||
`internal/stage/normalize_test.go`
|
||||
`internal/stage/normalize_test.go`,
|
||||
`internal/stage/semantic_contracts_refinement.go`
|
||||
|
||||
@@ -28,10 +28,26 @@ Run Audita polishing on base transcript and produce polished transcript.
|
||||
- polished transcript schema validation is mandatory.
|
||||
- report output is config-gated.
|
||||
|
||||
## Resume Semantics
|
||||
|
||||
The versioned semantic fingerprint covers the Audita service endpoint, model,
|
||||
validation model, module set, transcript description, output schema, selected
|
||||
external configuration path, and canonical polished-transcript identity. Module
|
||||
ordering is normalized because the configured modules form a set. Audita's
|
||||
executable, timeouts, concurrency, report and debug behavior, work retention,
|
||||
and credential environment name are operational and do not invalidate a
|
||||
successful result.
|
||||
|
||||
Narratio can fingerprint a selected model, module, or configuration identifier,
|
||||
but it cannot inspect content that Audita privately resolves behind that stable
|
||||
identifier. Force `polish` after changing such private content without changing
|
||||
its identifier.
|
||||
|
||||
## Related Contracts And Tests
|
||||
|
||||
- [Audita](../integrations/audita.md) owns subprocess, validation, and failure
|
||||
semantics.
|
||||
- [Configuration](../config.md#pipeline) owns operator-selected Audita values.
|
||||
- Implementation and tests: `internal/stage/polish.go`,
|
||||
`internal/stage/polish_test.go`
|
||||
`internal/stage/polish_test.go`,
|
||||
`internal/stage/semantic_contracts_refinement.go`
|
||||
|
||||
@@ -26,7 +26,8 @@ records that may consume rendered transcripts.
|
||||
- records input provenance, output paths, adapter metadata, logs, and generated config refs.
|
||||
- when `pipeline.render.enabled=false`, completes successfully with no outputs
|
||||
and records explanatory metadata. This is not an explicit self-skip: both
|
||||
manifests record success, and enabling render later requires a forced run.
|
||||
manifests record success. Because enablement is fingerprinted, enabling
|
||||
render later automatically makes the prior result non-resumable.
|
||||
|
||||
## Failure Semantics
|
||||
|
||||
@@ -40,9 +41,20 @@ records that may consume rendered transcripts.
|
||||
- only `format: markdown` is supported.
|
||||
- render stage owns production of built-in Markdown transcript sources.
|
||||
|
||||
## Resume Semantics
|
||||
|
||||
The versioned semantic fingerprint covers enablement, final format, resolved
|
||||
title (including the session-title fallback), timestamp, segment-ID and
|
||||
metadata inclusion, both canonical input identities, and both Markdown output
|
||||
identities. Seriatim's executable, timeout, and report behavior are operational
|
||||
and do not invalidate rendered transcripts. A render-only change leaves the
|
||||
independent `extract` sibling reusable while invalidating their shared
|
||||
downstream consumers.
|
||||
|
||||
## Related Contracts And Tests
|
||||
|
||||
- [Seriatim](../integrations/seriatim.md) owns render subprocess behavior.
|
||||
- [Configuration](../config.md#pipeline) owns render fields and defaults.
|
||||
- Implementation and tests: `internal/stage/render.go`,
|
||||
`internal/stage/render_test.go`
|
||||
`internal/stage/render_test.go`,
|
||||
`internal/stage/semantic_contracts_refinement.go`
|
||||
|
||||
@@ -33,6 +33,19 @@ When `trim.enabled=false`:
|
||||
- bounds output exists only in enabled trim path.
|
||||
- render-debug output is diagnostic and not a declared stage output.
|
||||
|
||||
## Resume Semantics
|
||||
|
||||
The versioned semantic fingerprint covers enablement, the bounds prompt and
|
||||
profile identifiers, the Scriptorium configuration identity, transcript input
|
||||
name, sticky session variable, bounds and trimmed output identities, and the
|
||||
Seriatim trim operation. Diagnostic bounds rendering, diagnostic output paths,
|
||||
timeouts, executable paths, and optional reports are operational and do not
|
||||
invalidate the canonical trimmed transcript.
|
||||
|
||||
Narratio cannot inspect prompt, profile, or configuration content that
|
||||
Scriptorium or Seriatim privately resolves behind a stable identifier. Force
|
||||
`trim` after changing such private content without changing its identifier.
|
||||
|
||||
## Related Contracts And Tests
|
||||
|
||||
- [Scriptorium](../integrations/scriptorium.md) owns bounds generation and
|
||||
@@ -40,4 +53,5 @@ When `trim.enabled=false`:
|
||||
- [Seriatim](../integrations/seriatim.md) owns transcript trimming behavior.
|
||||
- [Configuration](../config.md#pipeline) owns trim fields and defaults.
|
||||
- Implementation and tests: `internal/stage/trim.go`,
|
||||
`internal/stage/trim_test.go`
|
||||
`internal/stage/trim_test.go`,
|
||||
`internal/stage/semantic_contracts_refinement.go`
|
||||
|
||||
@@ -99,14 +99,20 @@ Execution rules:
|
||||
repeated self-skip with the same reason and no outputs is stable and does not
|
||||
perpetually rerun dependent work.
|
||||
|
||||
Prepare, transcribe, and merge currently provide semantic-configuration
|
||||
evidence. Changing prepare selection semantics reruns all fixed descendants;
|
||||
changing WhisperX language/service identity reuses prepare; and changing a
|
||||
Seriatim merge transformation reuses prepare and transcribe. Operational
|
||||
timeouts, retry/concurrency tuning, executable paths, workspace/cache/spool
|
||||
placement, reports, diagnostics, and secret values are excluded. Configuration,
|
||||
models, prompts, modules, or resources loaded privately inside external tools
|
||||
remain unobservable to Narratio and require an explicit `--force` after change.
|
||||
Prepare through render currently provide semantic-configuration evidence.
|
||||
Changing prepare selection semantics reruns all fixed descendants; changing
|
||||
WhisperX language/service identity reuses prepare; changing a Seriatim merge
|
||||
transformation reuses prepare and transcribe; and changing an Audita model
|
||||
reuses prepare, transcribe, and merge while rebuilding transcript refinement.
|
||||
A trim change invalidates both render and extract through the fixed dependency
|
||||
relation, while a render-only change preserves the extract sibling.
|
||||
|
||||
Operational timeouts, retry/concurrency tuning, executable paths,
|
||||
workspace/cache/spool placement, reports, diagnostics, and secret values are
|
||||
excluded. Configuration, models, prompts, modules, or resources loaded
|
||||
privately inside external tools remain unobservable to Narratio. If their
|
||||
contents change behind the same configured identifier, explicitly force the
|
||||
affected stage.
|
||||
|
||||
An explicit self-skip is a durable `skipped` stage outcome that later runs
|
||||
reconsider. It differs from successful no-output execution: disabled `render`
|
||||
|
||||
@@ -348,7 +348,7 @@ with stage-specific semantic configuration evidence.
|
||||
|
||||
## Stage 5 — Polish, Normalize, Trim, And Render Semantic Contracts
|
||||
|
||||
**Status: Pending**
|
||||
**Status: Completed**
|
||||
|
||||
### Goal
|
||||
|
||||
|
||||
@@ -130,7 +130,7 @@ func TestRunArtifactsWithSucceededAnalyzeSkipsUnlessForced(t *testing.T) {
|
||||
seed.MarkStageSucceeded(stageName, time.Date(2026, 5, 3, 10, 1, 0, 0, time.UTC), nil)
|
||||
}
|
||||
seed.MarkStageSkipped("extract", time.Date(2026, 5, 3, 10, 1, 0, 0, time.UTC), "notarius_disabled")
|
||||
seedCurrentSemanticEvidence(t, loadConfigForSemanticEvidence(t, pipelinePath, campaignPath, sessionPath), seed, "prepare", "transcribe", "merge")
|
||||
seedCurrentSemanticEvidence(t, loadConfigForSemanticEvidence(t, pipelinePath, campaignPath, sessionPath), seed, "prepare", "transcribe", "merge", "polish", "normalize", "trim", "render")
|
||||
if err := store.Save(context.Background(), manifestPath, seed); err != nil {
|
||||
t.Fatalf("save manifest: %v", err)
|
||||
}
|
||||
|
||||
@@ -61,7 +61,7 @@ func TestRunNoRemainingStagesRecordsSkippedStages(t *testing.T) {
|
||||
m.MarkStageSucceeded(name, time.Date(2026, 5, 3, 10, 1, 0, 0, time.UTC), nil)
|
||||
}
|
||||
m.MarkStageSkipped("extract", time.Date(2026, 5, 3, 10, 1, 0, 0, time.UTC), "notarius_disabled")
|
||||
seedCurrentSemanticEvidence(t, loadConfigForSemanticEvidence(t, pipelinePath, campaignPath, sessionPath), m, "prepare", "transcribe", "merge")
|
||||
seedCurrentSemanticEvidence(t, loadConfigForSemanticEvidence(t, pipelinePath, campaignPath, sessionPath), m, "prepare", "transcribe", "merge", "polish", "normalize", "trim", "render")
|
||||
if err := store.Save(context.Background(), manifestPath, m); err != nil {
|
||||
t.Fatalf("save manifest: %v", err)
|
||||
}
|
||||
@@ -174,6 +174,7 @@ func TestRunStageSkipAndForce(t *testing.T) {
|
||||
store := &manifest.LocalStore{}
|
||||
m := manifest.New("2026-05-03", time.Date(2026, 5, 3, 10, 0, 0, 0, time.UTC))
|
||||
m.MarkStageSucceeded("polish", time.Date(2026, 5, 3, 10, 1, 0, 0, time.UTC), nil)
|
||||
seedCurrentSemanticEvidence(t, loadConfigForSemanticEvidence(t, pipelinePath, campaignPath, sessionPath), m, "polish")
|
||||
if err := store.Save(context.Background(), manifestPath, m); err != nil {
|
||||
t.Fatalf("save manifest: %v", err)
|
||||
}
|
||||
|
||||
@@ -367,6 +367,74 @@ func TestInitialPipelineSemanticChangesRerunOnlyAffectedLineage(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestRefinementSemanticChangesRespectDependencyBranches(t *testing.T) {
|
||||
names := []string{"prepare", "transcribe", "merge", "polish", "normalize", "trim", "render"}
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
mutate func(*config.Config)
|
||||
wantRuns [7]int
|
||||
wantExtractStatus manifest.StageStatus
|
||||
}{
|
||||
{name: "polish model", mutate: func(cfg *config.Config) {
|
||||
cfg.Pipeline.Audita.Model = "production"
|
||||
}, wantRuns: [7]int{0, 0, 0, 1, 1, 1, 1}, wantExtractStatus: manifest.StatusStale},
|
||||
{name: "normalize schema", mutate: func(cfg *config.Config) {
|
||||
cfg.Pipeline.Normalize = &config.NormalizeConfig{OutputSchema: "seriatim.transcript.v2"}
|
||||
}, wantRuns: [7]int{0, 0, 0, 0, 1, 1, 1}, wantExtractStatus: manifest.StatusStale},
|
||||
{name: "trim prompt", mutate: func(cfg *config.Config) {
|
||||
enabled := true
|
||||
cfg.Pipeline.Trim = &config.TrimConfig{
|
||||
Enabled: &enabled,
|
||||
Bounds: config.TrimBoundsConfig{PromptID: "session-bounds-v2"},
|
||||
}
|
||||
}, wantRuns: [7]int{0, 0, 0, 0, 0, 1, 1}, wantExtractStatus: manifest.StatusStale},
|
||||
{name: "render format", mutate: func(cfg *config.Config) {
|
||||
cfg.Pipeline.Render = &config.RenderConfig{Format: "html"}
|
||||
}, wantRuns: [7]int{0, 0, 0, 0, 0, 0, 1}, wantExtractStatus: manifest.StatusSucceeded},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
cfg := testConfig(t)
|
||||
providers := make([]stage.SemanticConfigFingerprinter, len(names))
|
||||
seed := manifest.New(cfg.Session.SessionID, time.Now().UTC())
|
||||
for _, candidate := range stage.All() {
|
||||
seed.MarkStageSucceeded(candidate.Name(), time.Now().UTC(), nil)
|
||||
}
|
||||
for index, name := range names {
|
||||
providers[index] = canonicalSemanticProvider(t, name)
|
||||
fingerprint, err := providers[index].SemanticConfigFingerprint(&stage.Env{Config: cfg})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
seed.Stages[name].SemanticConfig = &fingerprint
|
||||
}
|
||||
store := &manifest.LocalStore{}
|
||||
if err := store.Save(context.Background(), manifestPathFor(cfg), seed); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
test.mutate(cfg)
|
||||
|
||||
runs := [7]int{}
|
||||
selected := make([]stage.Stage, 0, len(names))
|
||||
for index, name := range names {
|
||||
selected = append(selected, semanticContractRunStub{name: name, provider: providers[index], runs: &runs[index]})
|
||||
}
|
||||
if _, err := executeStages(context.Background(), cfg, selected, RunOptions{}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if runs != test.wantRuns {
|
||||
t.Fatalf("runs = %v, want %v", runs, test.wantRuns)
|
||||
}
|
||||
loaded, err := store.Load(context.Background(), manifestPathFor(cfg))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := loaded.Stages["extract"].Status; got != test.wantExtractStatus {
|
||||
t.Fatalf("extract status = %q, want %q", got, test.wantExtractStatus)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func canonicalSemanticProvider(t *testing.T, name string) stage.SemanticConfigFingerprinter {
|
||||
t.Helper()
|
||||
for _, candidate := range stage.All() {
|
||||
|
||||
168
internal/stage/semantic_contracts_refinement.go
Normal file
168
internal/stage/semantic_contracts_refinement.go
Normal file
@@ -0,0 +1,168 @@
|
||||
package stage
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/artifacts"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/manifest"
|
||||
)
|
||||
|
||||
const (
|
||||
polishSemanticConfigVersion = 1
|
||||
normalizeSemanticConfigVersion = 1
|
||||
trimSemanticConfigVersion = 1
|
||||
renderSemanticConfigVersion = 1
|
||||
)
|
||||
|
||||
type polishSemanticConfig struct {
|
||||
ServiceURL string `json:"service_url"`
|
||||
Model string `json:"model"`
|
||||
ValidationModel string `json:"validation_model"`
|
||||
Modules []string `json:"modules"`
|
||||
TranscriptDescription string `json:"transcript_description"`
|
||||
OutputSchema string `json:"output_schema"`
|
||||
ExternalConfig string `json:"external_config"`
|
||||
OutputIdentity string `json:"output_identity"`
|
||||
}
|
||||
|
||||
type normalizeSemanticConfig struct {
|
||||
Operation string `json:"operation"`
|
||||
OutputSchema string `json:"output_schema"`
|
||||
OutputIdentity string `json:"output_identity"`
|
||||
CoalesceGap *float64 `json:"coalesce_gap,omitempty"`
|
||||
OverlapWordRunGap *float64 `json:"overlap_word_run_gap,omitempty"`
|
||||
OverlapWordRunReorderWindow *float64 `json:"overlap_word_run_reorder_window,omitempty"`
|
||||
BackchannelMaxDuration *float64 `json:"backchannel_max_duration,omitempty"`
|
||||
FillerMaxDuration *float64 `json:"filler_max_duration,omitempty"`
|
||||
}
|
||||
|
||||
type trimSemanticConfig struct {
|
||||
Enabled bool `json:"enabled"`
|
||||
PromptID string `json:"prompt_id,omitempty"`
|
||||
ProfileID string `json:"profile_id,omitempty"`
|
||||
ExternalConfig string `json:"external_config,omitempty"`
|
||||
TranscriptInputName string `json:"transcript_input_name,omitempty"`
|
||||
InputIdentity string `json:"input_identity"`
|
||||
BoundsOutput string `json:"bounds_output,omitempty"`
|
||||
TrimmedOutput string `json:"trimmed_output"`
|
||||
StickySessionID string `json:"sticky_session_id,omitempty"`
|
||||
SeriatimOperation string `json:"seriatim_operation,omitempty"`
|
||||
}
|
||||
|
||||
type renderSemanticConfig struct {
|
||||
Enabled bool `json:"enabled"`
|
||||
Format string `json:"format"`
|
||||
Title string `json:"title"`
|
||||
IncludeTimestamps bool `json:"include_timestamps"`
|
||||
IncludeSegmentIDs bool `json:"include_segment_ids"`
|
||||
IncludeMetadata bool `json:"include_metadata"`
|
||||
InputIdentities []string `json:"input_identities"`
|
||||
OutputIdentities []string `json:"output_identities"`
|
||||
}
|
||||
|
||||
func (polishStage) SemanticConfigFingerprint(env *Env) (manifest.SemanticConfigFingerprint, error) {
|
||||
if env == nil || env.Config == nil || env.Config.Pipeline == nil {
|
||||
return manifest.SemanticConfigFingerprint{}, fmt.Errorf("polish semantic configuration requires resolved pipeline config")
|
||||
}
|
||||
audita := env.Config.Pipeline.Audita
|
||||
modules := append([]string(nil), audita.Modules...)
|
||||
for index := range modules {
|
||||
modules[index] = strings.TrimSpace(modules[index])
|
||||
}
|
||||
sort.Strings(modules)
|
||||
return FingerprintSemanticConfig(polishSemanticConfigVersion, polishSemanticConfig{
|
||||
ServiceURL: strings.TrimSpace(audita.BaseURL),
|
||||
Model: strings.TrimSpace(audita.Model),
|
||||
ValidationModel: strings.TrimSpace(audita.ValidationModel),
|
||||
Modules: modules,
|
||||
TranscriptDescription: strings.TrimSpace(audita.TranscriptDescription),
|
||||
OutputSchema: strings.TrimSpace(audita.OutputSchema),
|
||||
ExternalConfig: strings.TrimSpace(audita.ConfigPath),
|
||||
OutputIdentity: artifacts.ArtifactTranscriptPolished,
|
||||
})
|
||||
}
|
||||
|
||||
func (normalizeStage) SemanticConfigFingerprint(env *Env) (manifest.SemanticConfigFingerprint, error) {
|
||||
if env == nil || env.Config == nil || env.Config.Pipeline == nil {
|
||||
return manifest.SemanticConfigFingerprint{}, fmt.Errorf("normalize semantic configuration requires resolved pipeline config")
|
||||
}
|
||||
seriatim := env.Config.Pipeline.Seriatim
|
||||
normalize := normalizeConfigOrDefault(env.Config.Pipeline.Normalize)
|
||||
return FingerprintSemanticConfig(normalizeSemanticConfigVersion, normalizeSemanticConfig{
|
||||
Operation: "normalize",
|
||||
OutputSchema: strings.TrimSpace(normalize.OutputSchema),
|
||||
OutputIdentity: strings.TrimSpace(normalize.OutputPath),
|
||||
CoalesceGap: cloneFloat64(seriatim.CoalesceGap),
|
||||
OverlapWordRunGap: cloneFloat64(seriatim.Env.OverlapWordRunGap),
|
||||
OverlapWordRunReorderWindow: cloneFloat64(seriatim.Env.OverlapWordRunReorderWindow),
|
||||
BackchannelMaxDuration: cloneFloat64(seriatim.Env.BackchannelMaxDuration),
|
||||
FillerMaxDuration: cloneFloat64(seriatim.Env.FillerMaxDuration),
|
||||
})
|
||||
}
|
||||
|
||||
func (trimStage) SemanticConfigFingerprint(env *Env) (manifest.SemanticConfigFingerprint, error) {
|
||||
if env == nil || env.Config == nil || env.Config.Pipeline == nil || env.Config.Session == nil {
|
||||
return manifest.SemanticConfigFingerprint{}, fmt.Errorf("trim semantic configuration requires resolved pipeline and session config")
|
||||
}
|
||||
trim := env.Config.Pipeline.Trim
|
||||
enabled := trim != nil && trim.Enabled != nil && *trim.Enabled
|
||||
payload := trimSemanticConfig{
|
||||
Enabled: enabled,
|
||||
InputIdentity: artifacts.ArtifactTranscriptFinal,
|
||||
TrimmedOutput: artifacts.ArtifactTranscriptFinalTrimmed,
|
||||
StickySessionID: strings.TrimSpace(env.Config.Session.SessionID),
|
||||
}
|
||||
if trim != nil {
|
||||
payload.TrimmedOutput = strings.TrimSpace(trim.OutputPath)
|
||||
if payload.TrimmedOutput == "" {
|
||||
payload.TrimmedOutput = artifacts.TranscriptPathFinalTrimmed
|
||||
}
|
||||
}
|
||||
if enabled {
|
||||
bounds := trim.Bounds
|
||||
payload.PromptID = strings.TrimSpace(bounds.PromptID)
|
||||
payload.ProfileID = strings.TrimSpace(bounds.ProfileID)
|
||||
payload.TranscriptInputName = strings.TrimSpace(bounds.TranscriptInputName)
|
||||
payload.BoundsOutput = strings.TrimSpace(bounds.OutputPath)
|
||||
payload.SeriatimOperation = "trim"
|
||||
if env.Config.Pipeline.Scriptorium != nil {
|
||||
payload.ExternalConfig = strings.TrimSpace(env.Config.Pipeline.Scriptorium.ConfigPath)
|
||||
}
|
||||
}
|
||||
return FingerprintSemanticConfig(trimSemanticConfigVersion, payload)
|
||||
}
|
||||
|
||||
func (renderStage) SemanticConfigFingerprint(env *Env) (manifest.SemanticConfigFingerprint, error) {
|
||||
if env == nil || env.Config == nil || env.Config.Pipeline == nil || env.Config.Session == nil {
|
||||
return manifest.SemanticConfigFingerprint{}, fmt.Errorf("render semantic configuration requires resolved pipeline and session config")
|
||||
}
|
||||
render := renderConfigOrDefault(env.Config.Pipeline.Render)
|
||||
enabled := render.Enabled == nil || *render.Enabled
|
||||
format := strings.TrimSpace(render.Format)
|
||||
if format == "" {
|
||||
format = config.DefaultRenderFormat
|
||||
}
|
||||
includeTimestamps := render.IncludeTimestamps == nil || *render.IncludeTimestamps
|
||||
includeSegmentIDs := config.DefaultRenderSegmentIDs
|
||||
if render.IncludeSegmentIDs != nil {
|
||||
includeSegmentIDs = *render.IncludeSegmentIDs
|
||||
}
|
||||
return FingerprintSemanticConfig(renderSemanticConfigVersion, renderSemanticConfig{
|
||||
Enabled: enabled,
|
||||
Format: format,
|
||||
Title: resolveRenderTitle(render, env.Config.Session),
|
||||
IncludeTimestamps: includeTimestamps,
|
||||
IncludeSegmentIDs: includeSegmentIDs,
|
||||
IncludeMetadata: render.IncludeMetadata,
|
||||
InputIdentities: []string{artifacts.ArtifactTranscriptFinal, artifacts.ArtifactTranscriptFinalTrimmed},
|
||||
OutputIdentities: []string{artifacts.TranscriptPathFinalMarkdown, artifacts.TranscriptPathFinalTrimmedMarkdown},
|
||||
})
|
||||
}
|
||||
|
||||
var _ SemanticConfigFingerprinter = polishStage{}
|
||||
var _ SemanticConfigFingerprinter = normalizeStage{}
|
||||
var _ SemanticConfigFingerprinter = trimStage{}
|
||||
var _ SemanticConfigFingerprinter = renderStage{}
|
||||
196
internal/stage/semantic_contracts_refinement_test.go
Normal file
196
internal/stage/semantic_contracts_refinement_test.go
Normal file
@@ -0,0 +1,196 @@
|
||||
package stage
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/narratio/internal/config"
|
||||
)
|
||||
|
||||
func TestPolishSemanticConfigSensitivity(t *testing.T) {
|
||||
semantic := []struct {
|
||||
name string
|
||||
mutate func(*Env)
|
||||
}{
|
||||
{name: "model", mutate: func(env *Env) { env.Config.Pipeline.Audita.Model = "production" }},
|
||||
{name: "validation model", mutate: func(env *Env) { env.Config.Pipeline.Audita.ValidationModel = "validator-v2" }},
|
||||
{name: "modules", mutate: func(env *Env) { env.Config.Pipeline.Audita.Modules = []string{"terms", "speakers", "style"} }},
|
||||
{name: "description", mutate: func(env *Env) { env.Config.Pipeline.Audita.TranscriptDescription = "live campaign" }},
|
||||
{name: "schema", mutate: func(env *Env) { env.Config.Pipeline.Audita.OutputSchema = "audita.transcript.v2" }},
|
||||
{name: "external config", mutate: func(env *Env) { env.Config.Pipeline.Audita.ConfigPath = "configs/other.yml" }},
|
||||
{name: "service", mutate: func(env *Env) { env.Config.Pipeline.Audita.BaseURL = "https://other.example" }},
|
||||
}
|
||||
assertRefinementChanges(t, polishStage{}, semantic)
|
||||
assertRefinementUnchanged(t, polishStage{}, []struct {
|
||||
name string
|
||||
mutate func(*Env)
|
||||
}{
|
||||
{name: "module order", mutate: func(env *Env) { env.Config.Pipeline.Audita.Modules = []string{"terms", "speakers"} }},
|
||||
{name: "runner settings", mutate: func(env *Env) {
|
||||
env.Config.Pipeline.Audita.Binary = "/opt/audita"
|
||||
env.Config.Pipeline.Audita.Timeout = "1h"
|
||||
env.Config.Pipeline.Audita.TotalLLMConcurrency = intPointer(20)
|
||||
env.Config.Pipeline.Audita.ProposalLLMConcurrency = intPointer(10)
|
||||
env.Config.Pipeline.Audita.ValidationLLMConcurrency = intPointer(10)
|
||||
env.Config.Pipeline.Audita.LLMAPIKeyEnv = "OTHER_KEY"
|
||||
env.Config.Pipeline.Audita.WorkDirRetention = "always"
|
||||
env.Config.Pipeline.Audita.Report = boolPointer(false)
|
||||
}},
|
||||
})
|
||||
}
|
||||
|
||||
func TestNormalizeSemanticConfigSensitivity(t *testing.T) {
|
||||
assertRefinementChanges(t, normalizeStage{}, []struct {
|
||||
name string
|
||||
mutate func(*Env)
|
||||
}{
|
||||
{name: "schema", mutate: func(env *Env) { env.Config.Pipeline.Normalize.OutputSchema = "seriatim.transcript.v2" }},
|
||||
{name: "output", mutate: func(env *Env) { env.Config.Pipeline.Normalize.OutputPath = "transcripts/alternate.json" }},
|
||||
{name: "transformation", mutate: func(env *Env) { env.Config.Pipeline.Seriatim.Env.FillerMaxDuration = floatPointer(2.5) }},
|
||||
})
|
||||
assertRefinementUnchanged(t, normalizeStage{}, []struct {
|
||||
name string
|
||||
mutate func(*Env)
|
||||
}{
|
||||
{name: "runner settings", mutate: func(env *Env) {
|
||||
env.Config.Pipeline.Seriatim.Binary = "/opt/seriatim"
|
||||
env.Config.Pipeline.Seriatim.Timeout = "1h"
|
||||
env.Config.Pipeline.Normalize.Report = boolPointer(false)
|
||||
}},
|
||||
})
|
||||
}
|
||||
|
||||
func TestTrimSemanticConfigSensitivity(t *testing.T) {
|
||||
assertRefinementChanges(t, trimStage{}, []struct {
|
||||
name string
|
||||
mutate func(*Env)
|
||||
}{
|
||||
{name: "enablement", mutate: func(env *Env) { env.Config.Pipeline.Trim.Enabled = boolPointer(false) }},
|
||||
{name: "prompt", mutate: func(env *Env) { env.Config.Pipeline.Trim.Bounds.PromptID = "bounds-v2" }},
|
||||
{name: "profile", mutate: func(env *Env) { env.Config.Pipeline.Trim.Bounds.ProfileID = "strict" }},
|
||||
{name: "input identity", mutate: func(env *Env) { env.Config.Pipeline.Trim.Bounds.TranscriptInputName = "source_transcript" }},
|
||||
{name: "bounds output", mutate: func(env *Env) { env.Config.Pipeline.Trim.Bounds.OutputPath = "artifacts/other-bounds.json" }},
|
||||
{name: "trimmed output", mutate: func(env *Env) { env.Config.Pipeline.Trim.OutputPath = "transcripts/other-trimmed.json" }},
|
||||
{name: "external config", mutate: func(env *Env) { env.Config.Pipeline.Scriptorium.ConfigPath = "scriptorium/other.yml" }},
|
||||
})
|
||||
assertRefinementUnchanged(t, trimStage{}, []struct {
|
||||
name string
|
||||
mutate func(*Env)
|
||||
}{
|
||||
{name: "diagnostic render", mutate: func(env *Env) {
|
||||
env.Config.Pipeline.Trim.Bounds.RenderDebug = true
|
||||
env.Config.Pipeline.Trim.Bounds.RenderOutputPath = "debug/other.json"
|
||||
}},
|
||||
{name: "runner settings", mutate: func(env *Env) {
|
||||
env.Config.Pipeline.Trim.Bounds.Timeout = "1h"
|
||||
env.Config.Pipeline.Trim.Seriatim.Report = boolPointer(false)
|
||||
env.Config.Pipeline.Scriptorium.Binary = "/opt/scriptorium"
|
||||
env.Config.Pipeline.Scriptorium.Timeout = "2h"
|
||||
env.Config.Pipeline.Seriatim.Binary = "/opt/seriatim"
|
||||
}},
|
||||
})
|
||||
}
|
||||
|
||||
func TestRenderSemanticConfigSensitivity(t *testing.T) {
|
||||
assertRefinementChanges(t, renderStage{}, []struct {
|
||||
name string
|
||||
mutate func(*Env)
|
||||
}{
|
||||
{name: "enablement", mutate: func(env *Env) { env.Config.Pipeline.Render.Enabled = boolPointer(false) }},
|
||||
{name: "format", mutate: func(env *Env) { env.Config.Pipeline.Render.Format = "html" }},
|
||||
{name: "title", mutate: func(env *Env) { env.Config.Pipeline.Render.Title = "Explicit title" }},
|
||||
{name: "session title fallback", mutate: func(env *Env) { env.Config.Session.Title = "Changed session" }},
|
||||
{name: "timestamps", mutate: func(env *Env) { env.Config.Pipeline.Render.IncludeTimestamps = boolPointer(false) }},
|
||||
{name: "segment ids", mutate: func(env *Env) { env.Config.Pipeline.Render.IncludeSegmentIDs = boolPointer(true) }},
|
||||
{name: "metadata", mutate: func(env *Env) { env.Config.Pipeline.Render.IncludeMetadata = true }},
|
||||
})
|
||||
assertRefinementUnchanged(t, renderStage{}, []struct {
|
||||
name string
|
||||
mutate func(*Env)
|
||||
}{
|
||||
{name: "runner settings", mutate: func(env *Env) {
|
||||
env.Config.Pipeline.Seriatim.Binary = "/opt/seriatim"
|
||||
env.Config.Pipeline.Seriatim.Timeout = "1h"
|
||||
env.Config.Pipeline.Seriatim.Report = boolPointer(false)
|
||||
}},
|
||||
})
|
||||
}
|
||||
|
||||
func refinementSemanticEnv() *Env {
|
||||
env := initialSemanticEnv()
|
||||
env.Config.Session.Title = "Session title"
|
||||
env.Config.Pipeline.Audita = config.AuditaConfig{
|
||||
Binary: "audita", Timeout: "10m", LLMAPIKeyEnv: "AUDITA_KEY",
|
||||
Modules: []string{"speakers", "terms"}, BaseURL: "https://audita.example",
|
||||
Model: "testing", ValidationModel: "validator", TranscriptDescription: "tabletop session",
|
||||
ConfigPath: "configs/audita.yml", OutputSchema: "audita.transcript.v1",
|
||||
TotalLLMConcurrency: intPointer(4), ProposalLLMConcurrency: intPointer(2),
|
||||
ValidationLLMConcurrency: intPointer(2), WorkDirRetention: "failure", Report: boolPointer(true),
|
||||
}
|
||||
env.Config.Pipeline.Normalize = &config.NormalizeConfig{
|
||||
OutputPath: "transcripts/final.json", OutputSchema: "seriatim.transcript.v1", Report: boolPointer(true),
|
||||
}
|
||||
env.Config.Pipeline.Trim = &config.TrimConfig{
|
||||
Enabled: boolPointer(true), OutputPath: "transcripts/final.trimmed.json",
|
||||
Bounds: config.TrimBoundsConfig{
|
||||
PromptID: "bounds", ProfileID: "default", TranscriptInputName: "transcript",
|
||||
OutputPath: "artifacts/session-bounds.json", Timeout: "10m",
|
||||
RenderOutputPath: "debug/bounds.json",
|
||||
},
|
||||
}
|
||||
env.Config.Pipeline.Render = &config.RenderConfig{
|
||||
Enabled: boolPointer(true), Format: "markdown", IncludeTimestamps: boolPointer(true),
|
||||
IncludeSegmentIDs: boolPointer(false),
|
||||
}
|
||||
env.Config.Pipeline.Scriptorium = &config.ScriptoriumConfig{
|
||||
Binary: "scriptorium", ConfigPath: "scriptorium/config.yml", Timeout: "10m",
|
||||
}
|
||||
return env
|
||||
}
|
||||
|
||||
func assertRefinementChanges(t *testing.T, provider semanticFingerprintProvider, tests []struct {
|
||||
name string
|
||||
mutate func(*Env)
|
||||
}) {
|
||||
t.Helper()
|
||||
want, err := provider.SemanticConfigFingerprint(refinementSemanticEnv())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, test := range tests {
|
||||
t.Run("semantic "+test.name, func(t *testing.T) {
|
||||
env := refinementSemanticEnv()
|
||||
test.mutate(env)
|
||||
got, err := provider.SemanticConfigFingerprint(env)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if want.Equal(got) {
|
||||
t.Fatalf("semantic change %q retained fingerprint", test.name)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func assertRefinementUnchanged(t *testing.T, provider semanticFingerprintProvider, tests []struct {
|
||||
name string
|
||||
mutate func(*Env)
|
||||
}) {
|
||||
t.Helper()
|
||||
want, err := provider.SemanticConfigFingerprint(refinementSemanticEnv())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, test := range tests {
|
||||
t.Run("operational "+test.name, func(t *testing.T) {
|
||||
env := refinementSemanticEnv()
|
||||
test.mutate(env)
|
||||
got, err := provider.SemanticConfigFingerprint(env)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !want.Equal(got) {
|
||||
t.Fatalf("operational change %q changed fingerprint", test.name)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user