diff --git a/README.md b/README.md index 532fa85..01bb58b 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ needed. ```sh OPENROUTER_API_KEY=... \ go run ./cmd/notarius run dnd-session \ - --config examples/dnd-spells.config.yml \ + --config examples/dnd-minimal.config.yml \ --input examples/seriatim-minimal-transcript.json ``` @@ -29,7 +29,6 @@ Useful references: - [D&D spell artifact contract](docs/integrations/dnd-spell-artifacts.md) - [Developer guide](docs/development.md) - [Internal implementation docs](docs/internal/overview.md) -- [Maintained example config](examples/dnd-spells.config.yml) -- [NPC-grounded example config](examples/dnd-npc-grounded.config.yml) -- [NPC interaction example config](examples/dnd-npc-interactions.config.yml) +- [Minimal D&D configuration](examples/dnd-minimal.config.yml) +- [Complete D&D configuration](examples/dnd-complete.config.yml) - [Maintained example input](examples/seriatim-minimal-transcript.json) diff --git a/docs/cli.md b/docs/cli.md index 7510a99..39a785e 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -88,7 +88,7 @@ Use `slot=path` when the selected targets declare the slot unambiguously: ```sh go run ./cmd/notarius run dnd-session \ - --config examples/dnd-spells.config.yml \ + --config examples/dnd-minimal.config.yml \ --input examples/seriatim-minimal-transcript.json \ --reference roster=./campaign-roster.txt ``` @@ -98,7 +98,7 @@ when you want to target a specific target: ```sh go run ./cmd/notarius run dnd-session \ - --config examples/dnd-spells.config.yml \ + --config examples/dnd-minimal.config.yml \ --input examples/seriatim-minimal-transcript.json \ --reference spells.extract.glossary=./campaign-glossary.txt ``` @@ -109,8 +109,8 @@ it in memory and fans it out to spell extraction, combat extraction, and combat normalization: ```sh -go run ./cmd/notarius run dnd-npc-grounded \ - --config examples/dnd-npc-grounded.config.yml \ +go run ./cmd/notarius run dnd-session \ + --config examples/dnd-complete.config.yml \ --input examples/seriatim-minimal-transcript.json \ --output-dir ./npc-grounded-output ``` @@ -135,7 +135,7 @@ Use `--without-reference` to remove a configured optional binding for a run: ```sh go run ./cmd/notarius run dnd-session \ - --config examples/dnd-spells.config.yml \ + --config examples/dnd-minimal.config.yml \ --input examples/seriatim-minimal-transcript.json \ --without-reference glossary ``` @@ -145,7 +145,7 @@ run to share an identifier: ```sh go run ./cmd/notarius run dnd-session \ - --config examples/dnd-spells.config.yml \ + --config examples/dnd-minimal.config.yml \ --input examples/seriatim-minimal-transcript.json \ --session-id campaign-17-session-04 ``` @@ -156,7 +156,7 @@ recorded work; using it while checkpoint recording is disabled is an error: ```sh go run ./cmd/notarius run dnd-session \ - --config examples/dnd-spells.config.yml \ + --config examples/dnd-minimal.config.yml \ --input examples/seriatim-minimal-transcript.json \ --resume ``` @@ -166,10 +166,10 @@ from the configuration. The selected step and dependents are reported as `forced_recompute`; reusable predecessors are reported as `reused`: ```sh -go run ./cmd/notarius run dnd-npc-grounded \ - --config examples/dnd-npc-grounded.config.yml \ +go run ./cmd/notarius run dnd-session \ + --config examples/dnd-complete.config.yml \ --input examples/seriatim-minimal-transcript.json \ - --resume --recompute-step grounded-events + --resume --recompute-step extract-events ``` Checkpoint decisions use these categories: `reused`, `executed`, @@ -187,7 +187,7 @@ code `1`. ```sh go run ./cmd/notarius run dnd-session \ - --config examples/dnd-spells.config.yml \ + --config examples/dnd-minimal.config.yml \ --input examples/seriatim-minimal-transcript.json \ --debug --debug-dir ./notarius-debug ``` @@ -196,7 +196,7 @@ Use `refresh` when intentionally replacing the cached plan for the same source: ```sh go run ./cmd/notarius run dnd-session \ - --config examples/dnd-spells.config.yml \ + --config examples/dnd-minimal.config.yml \ --input examples/seriatim-minimal-transcript.json \ --chunk_cache refresh ``` @@ -205,7 +205,7 @@ Use `bypass` for a one-off run that must not inspect or create plan-cache state: ```sh go run ./cmd/notarius run dnd-session \ - --config examples/dnd-spells.config.yml \ + --config examples/dnd-minimal.config.yml \ --input examples/seriatim-minimal-transcript.json \ --chunk_cache bypass ``` @@ -230,10 +230,10 @@ Examples: ```sh go run ./cmd/notarius config validate \ - --config examples/dnd-spells.config.yml + --config examples/dnd-minimal.config.yml go run ./cmd/notarius config validate \ - --config examples/dnd-spells.config.yml \ + --config examples/dnd-minimal.config.yml \ --pipeline dnd-session \ --only spells ``` @@ -252,10 +252,10 @@ Examples: ```sh go run ./cmd/notarius pipelines list \ - --config examples/dnd-spells.config.yml + --config examples/dnd-minimal.config.yml go run ./cmd/notarius pipelines list \ - --config examples/dnd-spells.config.yml \ + --config examples/dnd-minimal.config.yml \ --json ``` diff --git a/docs/config.md b/docs/config.md index 4b37500..d5f9183 100644 --- a/docs/config.md +++ b/docs/config.md @@ -19,15 +19,14 @@ The explicit-path option is defined in the [CLI reference](cli.md). ## Maintained Examples -- [Minimal D&D spell configuration](../examples/dnd-spells.config.yml) -- [Production-oriented D&D spell configuration](../examples/dnd-spells-production.config.yml) -- [D&D NPC configuration](../examples/dnd-npcs.config.yml) -- [D&D combat-turn configuration](../examples/dnd-combat-turns.config.yml) -- [D&D NPC-grounded spell and combat configuration](../examples/dnd-npc-grounded.config.yml) -- [D&D NPC interaction configuration](../examples/dnd-npc-interactions.config.yml) -- [D&D scene chunk-map configuration](../examples/dnd-scene-chunk-map.config.yml) +- [Minimal D&D configuration](../examples/dnd-minimal.config.yml) shows the + shortest useful single-step, single-lane pipeline. +- [Complete D&D configuration](../examples/dnd-complete.config.yml) shows scene + chunking, ordered steps, all five D&D artifact lanes, a generated NPC + handoff, campaign and spell-catalog references, checkpoint storage, and + chunk-map output. -All are complete version 3 files. The fragments below illustrate individual +Both are complete version 3 files. The fragments below illustrate individual fields and are not alternate complete configurations. ## Top-Level Fields @@ -347,9 +346,9 @@ output: ``` The payload and index descriptor are defined by the -[Accepted Chunk Map contract](integrations/chunk-map.md). See the complete -[D&D scene chunk-map configuration](../examples/dnd-scene-chunk-map.config.yml) -for a copyable pipeline. +[Accepted Chunk Map contract](integrations/chunk-map.md). See the +[complete D&D configuration](../examples/dnd-complete.config.yml) for a +copyable pipeline that enables this option. Validator bindings use the same shorthand or object module-binding form, but only these fields are supported: @@ -563,17 +562,17 @@ bound. The `dnd/npcs` extractor declares the same optional campaign slots as the spell extractor, but it does not declare the `npcs` registry slot. Its normalizer -accepts no references. The maintained -[NPC-grounded example](../examples/dnd-npc-grounded.config.yml) binds its -accepted normalized output to the later spell and combat targets through an -explicit ordered step. +accepts no references. The +[complete D&D example](../examples/dnd-complete.config.yml) binds its accepted +normalized output to later spell, combat-turn, and NPC-interaction targets +through an explicit ordered step. The `dnd/scene-descriptions` extractor accepts only the optional `players`, `party`, and `glossary` campaign slots. They can disambiguate terms in the model-facing title and summary but are never source evidence. Its normalizer accepts no references. No NPC registry or generated artifact is required. See the [D&D scene-description artifact contract](integrations/dnd-scene-description-artifacts.md) -and the copyable [scene-description configuration](../examples/dnd-scene-descriptions.config.yml). +and the copyable [complete D&D configuration](../examples/dnd-complete.config.yml). The `dnd/combat-turns` extractor declares the optional campaign slots and the structured `npcs` slot. Campaign references guide only the LLM extraction diff --git a/docs/integrations/dnd-npc-interaction-artifacts.md b/docs/integrations/dnd-npc-interaction-artifacts.md index dee7ecc..f8bd248 100644 --- a/docs/integrations/dnd-npc-interaction-artifacts.md +++ b/docs/integrations/dnd-npc-interaction-artifacts.md @@ -145,4 +145,4 @@ framework provenance and dependency fingerprints. See [Configuration](../config.md#implemented-production-modules) for selectable keys and chains, [the NPC artifact contract](dnd-npc-artifacts.md) for the registry boundary, and the copyable -[maintained example](../../examples/dnd-npc-interactions.config.yml). +[complete D&D example](../../examples/dnd-complete.config.yml). diff --git a/docs/integrations/dnd-scene-description-artifacts.md b/docs/integrations/dnd-scene-description-artifacts.md index 76627b2..9224709 100644 --- a/docs/integrations/dnd-scene-description-artifacts.md +++ b/docs/integrations/dnd-scene-description-artifacts.md @@ -115,6 +115,6 @@ bounded and do not copy transcript or campaign-reference content. Use `dnd/scenes` to form accepted scene chunks, then select `dnd/scene-descriptions` for extraction and normalization. The maintained -[scene-description configuration](../../examples/dnd-scene-descriptions.config.yml) -is a copyable minimal lane. Selectable keys and default validator chains are -defined in [Configuration](../config.md). +[complete D&D configuration](../../examples/dnd-complete.config.yml) includes +this lane alongside the other D&D artifacts. Selectable keys and default +validator chains are defined in [Configuration](../config.md). diff --git a/docs/operations.md b/docs/operations.md index 783dc2f..507cc69 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -48,15 +48,16 @@ needed. This is data deletion, not cache cleanup. ## Ordered D&D Workflow -The maintained [NPC-grounded configuration](../examples/dnd-npc-grounded.config.yml) +The maintained [complete D&D configuration](../examples/dnd-complete.config.yml) contains one pipeline with two ordered steps. The first step extracts and -normalizes NPCs. Only after that lane reaches an accepted terminal result does -the second step begin; its generated NPC reference is supplied in memory to -spell extraction, combat extraction, and combat normalization. +normalizes NPCs and scene descriptions. Only after those lanes reach accepted +terminal results does the second step begin; its generated NPC reference is +supplied in memory to spell, combat-turn, and NPC-interaction extraction and +the applicable normalizers. ```sh -go run ./cmd/notarius run dnd-npc-grounded \ - --config examples/dnd-npc-grounded.config.yml \ +go run ./cmd/notarius run dnd-session \ + --config examples/dnd-complete.config.yml \ --input examples/seriatim-minimal-transcript.json \ --output-dir ./npc-grounded-output ``` diff --git a/examples/dnd-combat-turns.config.yml b/examples/dnd-combat-turns.config.yml deleted file mode 100644 index 2542633..0000000 --- a/examples/dnd-combat-turns.config.yml +++ /dev/null @@ -1,22 +0,0 @@ -version: 3 -output: - directory: ./notarius-output -cache: - chunk_plans: - mode: bypass - directory: ./notarius-cache/chunk-plans - checkpoints: - enabled: false - directory: ./notarius-cache/checkpoints -debug: - directory: ./notarius-debug -pipelines: - dnd-combat: - input: seriatim - chunk: generic - artifacts: - combat: - extract: - module: dnd/combat-turns - retries: 2 - normalize: dnd/combat-turns diff --git a/examples/dnd-complete.config.yml b/examples/dnd-complete.config.yml new file mode 100644 index 0000000..02c3372 --- /dev/null +++ b/examples/dnd-complete.config.yml @@ -0,0 +1,79 @@ +version: 3 +concurrency: + total_llm: 2 + stage_workers: + extract: 2 +output: + directory: ./notarius-output +cache: + chunk_plans: + mode: auto + directory: ./notarius-cache/chunk-plans + checkpoints: + enabled: true + directory: ./notarius-cache/checkpoints +debug: + directory: ./notarius-debug +pipelines: + dnd-session: + input: seriatim + # Stable campaign context is shared by every module that accepts these slots. + references: + party: ./dnd-party.txt + glossary: ./dnd-glossary.txt + chunk: + module: dnd/scenes + retries: 2 + output: + module: json + options: + include_chunk_map: true + steps: + # Establish session-wide reference artifacts before extracting events. + - id: describe-session + artifacts: + npcs: + extract: + module: dnd/npcs + retries: 2 + merge: appendorder + normalize: dnd/npcs + scene-descriptions: + extract: + module: dnd/scene-descriptions + retries: 2 + merge: appendorder + normalize: dnd/scene-descriptions + - id: extract-events + # The accepted NPC artifact is supplied in memory to every compatible + # extractor and normalizer in this step. + references: + npcs: + artifact: + step: describe-session + lane: npcs + artifacts: + spells: + extract: + module: dnd/spells + retries: 2 + references: + spell_catalog: ./dnd-spell-catalog.json + merge: appendorder + # Stage-local file references are intentionally bound at each stage. + normalize: + module: dnd/spells + references: + spell_catalog: ./dnd-spell-catalog.json + combat-turns: + extract: + module: dnd/combat-turns + retries: 2 + merge: appendorder + normalize: dnd/combat-turns + npc-interactions: + extract: + module: dnd/npc-interactions + retries: 2 + merge: appendorder + normalize: dnd/npc-interactions diff --git a/examples/dnd-spells-glossary.txt b/examples/dnd-glossary.txt similarity index 100% rename from examples/dnd-spells-glossary.txt rename to examples/dnd-glossary.txt diff --git a/examples/dnd-minimal.config.yml b/examples/dnd-minimal.config.yml new file mode 100644 index 0000000..0f74bd0 --- /dev/null +++ b/examples/dnd-minimal.config.yml @@ -0,0 +1,8 @@ +version: 3 +pipelines: + dnd-session: + input: seriatim + artifacts: + spells: + extract: dnd/spells + normalize: dnd/spells diff --git a/examples/dnd-npc-grounded.config.yml b/examples/dnd-npc-grounded.config.yml deleted file mode 100644 index f95cb7c..0000000 --- a/examples/dnd-npc-grounded.config.yml +++ /dev/null @@ -1,37 +0,0 @@ -version: 3 -output: - directory: ./notarius-output -cache: - chunk_plans: - mode: bypass - checkpoints: - enabled: false - directory: "" -debug: - directory: ./notarius-debug -pipelines: - dnd-npc-grounded: - input: seriatim - steps: - - id: identify-npcs - artifacts: - npcs: - extract: - module: dnd/npcs - retries: 2 - normalize: dnd/npcs - - id: grounded-events - references: - npcs: - artifact: - step: identify-npcs - lane: npcs - artifacts: - spells: - extract: dnd/spells - normalize: dnd/spells - combat: - extract: - module: dnd/combat-turns - retries: 2 - normalize: dnd/combat-turns diff --git a/examples/dnd-npc-interactions.config.yml b/examples/dnd-npc-interactions.config.yml deleted file mode 100644 index 8d377d7..0000000 --- a/examples/dnd-npc-interactions.config.yml +++ /dev/null @@ -1,34 +0,0 @@ -version: 3 -output: - directory: ./notarius-output -cache: - chunk_plans: - mode: bypass - checkpoints: - enabled: false - directory: "" -debug: - directory: ./notarius-debug -pipelines: - dnd-npc-interactions: - input: seriatim - steps: - - id: identify-npcs - artifacts: - npcs: - extract: - module: dnd/npcs - retries: 2 - normalize: dnd/npcs - - id: extract-interactions - references: - npcs: - artifact: - step: identify-npcs - lane: npcs - artifacts: - interactions: - extract: - module: dnd/npc-interactions - retries: 2 - normalize: dnd/npc-interactions diff --git a/examples/dnd-npcs.config.yml b/examples/dnd-npcs.config.yml deleted file mode 100644 index 279b14e..0000000 --- a/examples/dnd-npcs.config.yml +++ /dev/null @@ -1,21 +0,0 @@ -version: 3 -output: - directory: ./notarius-output -cache: - chunk_plans: - mode: bypass - checkpoints: - enabled: false - directory: "" -debug: - directory: ./notarius-debug -pipelines: - dnd-session: - input: seriatim - chunk: generic - artifacts: - npcs: - extract: - module: dnd/npcs - retries: 2 - normalize: dnd/npcs diff --git a/examples/dnd-spells-roster.txt b/examples/dnd-party.txt similarity index 100% rename from examples/dnd-spells-roster.txt rename to examples/dnd-party.txt diff --git a/examples/dnd-scene-chunk-map.config.yml b/examples/dnd-scene-chunk-map.config.yml deleted file mode 100644 index 3c06add..0000000 --- a/examples/dnd-scene-chunk-map.config.yml +++ /dev/null @@ -1,23 +0,0 @@ -version: 3 -output: - directory: ./notarius-output -cache: - chunk_plans: - mode: bypass - checkpoints: - enabled: false - directory: "" -debug: - directory: ./notarius-debug -pipelines: - dnd-scene-chunk-map: - input: seriatim - chunk: dnd/scenes - output: - module: json - options: - include_chunk_map: true - artifacts: - spells: - extract: dnd/spells - normalize: dnd/spells diff --git a/examples/dnd-scene-descriptions.config.yml b/examples/dnd-scene-descriptions.config.yml deleted file mode 100644 index bc92ec1..0000000 --- a/examples/dnd-scene-descriptions.config.yml +++ /dev/null @@ -1,20 +0,0 @@ -version: 3 -output: - directory: ./notarius-output -cache: - chunk_plans: - mode: bypass - directory: ./notarius-cache/chunk-plans - checkpoints: - enabled: false - directory: ./notarius-cache/checkpoints -debug: - directory: ./notarius-debug -pipelines: - dnd-scene-descriptions: - input: seriatim - chunk: dnd/scenes - artifacts: - scene-descriptions: - extract: dnd/scene-descriptions - normalize: dnd/scene-descriptions diff --git a/examples/dnd-spells-catalog.json b/examples/dnd-spell-catalog.json similarity index 100% rename from examples/dnd-spells-catalog.json rename to examples/dnd-spell-catalog.json diff --git a/examples/dnd-spells-production.config.yml b/examples/dnd-spells-production.config.yml deleted file mode 100644 index a870015..0000000 --- a/examples/dnd-spells-production.config.yml +++ /dev/null @@ -1,39 +0,0 @@ -version: 3 -concurrency: - total_llm: 1 - stage_workers: - extract: 1 -output: - directory: ./notarius-output -cache: - chunk_plans: - directory: /var/cache/notarius/chunk-plans - mode: auto - checkpoints: - enabled: false - directory: /var/cache/notarius/checkpoints -debug: - directory: ./notarius-debug -pipelines: - dnd-session: - input: seriatim - references: - party: ./dnd-spells-roster.txt - glossary: ./dnd-spells-glossary.txt - chunk: - module: generic - options: - max_units: 50 - artifacts: - spells: - extract: - module: dnd/spells - retries: 2 - # Overlay behavior binds the same catalog independently at each stage. - references: - spell_catalog: ./dnd-spells-catalog.json - normalize: - module: dnd/spells - # Normalize-stage references are local and must be bound explicitly. - references: - spell_catalog: ./dnd-spells-catalog.json diff --git a/examples/dnd-spells.config.yml b/examples/dnd-spells.config.yml deleted file mode 100644 index 5038cb6..0000000 --- a/examples/dnd-spells.config.yml +++ /dev/null @@ -1,19 +0,0 @@ -version: 3 -output: - directory: ./notarius-output -cache: - chunk_plans: - mode: bypass - checkpoints: - enabled: false - directory: "" -debug: - directory: ./notarius-debug -pipelines: - dnd-session: - input: seriatim - artifacts: - spells: - extract: dnd/spells - # Base-only behavior: normalization uses the embedded SRD catalog. - normalize: dnd/spells diff --git a/internal/cli/dnd_combat_contract_test.go b/internal/cli/dnd_combat_contract_test.go index 4d6b1f6..86b71f0 100644 --- a/internal/cli/dnd_combat_contract_test.go +++ b/internal/cli/dnd_combat_contract_test.go @@ -15,8 +15,7 @@ import ( func TestProductionCombatConfigurationResolvesTypedLane(t *testing.T) { components := productionTestComponents(t) - configPath := repositoryPath("examples", "dnd-combat-turns.config.yml") - cfg := loadMaintainedExample(t, configPath) + cfg := productionCombatContractConfig() effective, err := cfg.Resolve(config.ResolveInput{PipelineID: "dnd-combat", Catalog: catalogFromRegistries(components.registries)}) if err != nil { t.Fatalf("Resolve() error = %v, want nil", err) @@ -97,9 +96,8 @@ func TestProductionCombatConfigurationResolvesTypedLane(t *testing.T) { func TestProductionCombatConfigurationRejectsLooseOptionsAndLaneValidators(t *testing.T) { components := productionTestComponents(t) - configPath := repositoryPath("examples", "dnd-combat-turns.config.yml") resolve := func(mutate func(*pipeline.PipelineProfile)) error { - cfg := loadMaintainedExample(t, configPath) + cfg := productionCombatContractConfig() profile := cfg.Pipelines["dnd-combat"] mutate(&profile) cfg.Pipelines["dnd-combat"] = profile @@ -131,7 +129,7 @@ func TestProductionCombatConfigurationRejectsLooseOptionsAndLaneValidators(t *te func TestProductionCombatConfigurationResolvesTypedUnconditionalValidators(t *testing.T) { components := productionTestComponents(t) - cfg := loadMaintainedExample(t, repositoryPath("examples", "dnd-combat-turns.config.yml")) + cfg := productionCombatContractConfig() profile := cfg.Pipelines["dnd-combat"] lane := profile.Artifacts["combat"] lane.Extract.Validators = pipeline.ValidatorOverride{Set: true, Validators: []pipeline.ModuleBinding{pipeline.Binding("generic/always_accept")}} @@ -150,6 +148,22 @@ func TestProductionCombatConfigurationResolvesTypedUnconditionalValidators(t *te } } +func productionCombatContractConfig() config.Config { + cfg := config.Default() + cfg.Pipelines["dnd-combat"] = pipeline.PipelineProfile{ + ID: "dnd-combat", + Input: pipeline.Binding("seriatim"), + Chunk: pipeline.Binding(pipeline.DefaultChunkModule), + Artifacts: map[string]pipeline.ArtifactLaneProfile{ + "combat": { + Extract: pipeline.ModuleBinding{Module: combatextract.Key, Retries: 2}, + Normalize: pipeline.Binding(combatnormalize.Key), + }, + }, + } + return cfg +} + func hasReferenceSlot(slots []contracts.ReferenceSlot, name string) bool { for _, slot := range slots { if slot.Name == name { diff --git a/internal/cli/dnd_npc_contract_test.go b/internal/cli/dnd_npc_contract_test.go index d59c733..9437b7c 100644 --- a/internal/cli/dnd_npc_contract_test.go +++ b/internal/cli/dnd_npc_contract_test.go @@ -16,8 +16,7 @@ import ( func TestProductionNPCConfigurationResolvesTypedLane(t *testing.T) { components := productionTestComponents(t) catalog := catalogFromRegistries(components.registries) - configPath := repositoryPath("examples", "dnd-npcs.config.yml") - cfg := loadMaintainedExample(t, configPath) + cfg := productionNPCContractConfig() effective, err := cfg.Resolve(config.ResolveInput{PipelineID: "dnd-session", Catalog: catalog}) if err != nil { t.Fatalf("Resolve() error = %v, want nil", err) @@ -81,9 +80,8 @@ func TestProductionNPCConfigurationResolvesTypedLane(t *testing.T) { func TestProductionNPCConfigurationValidatesOptionsReferencesAndPlacement(t *testing.T) { components := productionTestComponents(t) - configPath := repositoryPath("examples", "dnd-npcs.config.yml") resolve := func(mutate func(*pipeline.PipelineProfile)) error { - cfg := loadMaintainedExample(t, configPath) + cfg := productionNPCContractConfig() profile := cfg.Pipelines["dnd-session"] mutate(&profile) cfg.Pipelines["dnd-session"] = profile @@ -123,6 +121,22 @@ func TestProductionNPCConfigurationValidatesOptionsReferencesAndPlacement(t *tes } } +func productionNPCContractConfig() config.Config { + cfg := config.Default() + cfg.Pipelines["dnd-session"] = pipeline.PipelineProfile{ + ID: "dnd-session", + Input: pipeline.Binding("seriatim"), + Chunk: pipeline.Binding(pipeline.DefaultChunkModule), + Artifacts: map[string]pipeline.ArtifactLaneProfile{ + "npcs": { + Extract: pipeline.ModuleBinding{Module: npcextract.Key, Retries: 2}, + Normalize: pipeline.Binding(npcnormalize.Key), + }, + }, + } + return cfg +} + func validatorChain(resolved pipeline.ResolvedPipeline, stage pipeline.ModuleStage, module string) []pipeline.ModuleBinding { for _, chain := range resolved.ValidatorChains { if chain.Stage == stage && chain.ModuleKey == module { diff --git a/internal/cli/example_contract_test.go b/internal/cli/example_contract_test.go index 250a7c7..8062135 100644 --- a/internal/cli/example_contract_test.go +++ b/internal/cli/example_contract_test.go @@ -4,6 +4,7 @@ import ( "encoding/json" "os" "path/filepath" + "sort" "strings" "testing" @@ -32,11 +33,14 @@ func TestMaintainedExamplesLoadResolveAndList(t *testing.T) { if err != nil { t.Fatalf("materialize maintained example references for %q: %v", pipelineID, err) } - if example.name == "production" { - if len(materialized.Steps[0].ArtifactLanes) != 1 || - len(materialized.Steps[0].ArtifactLanes[0].ExtractReferences.ReferenceSet.Slots["spell_catalog"].Items) != 1 || - len(materialized.Steps[0].ArtifactLanes[0].NormalizeReferences.ReferenceSet.Slots["spell_catalog"].Items) != 1 { - t.Fatalf("production spell catalog reference was not materialized: %#v", materialized.Steps[0].ArtifactLanes) + if example.name == "complete" { + if got := exampleStepLaneIDs(materialized); strings.Join(got, "|") != "describe-session:npcs,scene-descriptions|extract-events:combat-turns,npc-interactions,spells" { + t.Fatalf("complete example steps and lanes = %v, want every D&D extractor in the documented two-step composition", got) + } + spellLane := referenceContractLane(t, materialized, "spells") + if len(spellLane.ExtractReferences.ReferenceSet.Slots["spell_catalog"].Items) != 1 || + len(spellLane.NormalizeReferences.ReferenceSet.Slots["spell_catalog"].Items) != 1 { + t.Fatalf("complete example spell catalog reference was not materialized: %#v", spellLane) } } } @@ -49,6 +53,36 @@ func TestMaintainedExamplesLoadResolveAndList(t *testing.T) { } } +func TestMaintainedConfigurationExampleSet(t *testing.T) { + entries, err := os.ReadDir(repositoryPath("examples")) + if err != nil { + t.Fatal(err) + } + var names []string + for _, entry := range entries { + if !entry.IsDir() && strings.HasSuffix(entry.Name(), ".config.yml") { + names = append(names, entry.Name()) + } + } + sort.Strings(names) + if got := strings.Join(names, ","); got != "dnd-complete.config.yml,dnd-minimal.config.yml" { + t.Fatalf("maintained configuration examples = %q, want only the minimal and complete D&D examples", got) + } +} + +func exampleStepLaneIDs(resolved pipeline.ResolvedPipeline) []string { + result := make([]string, 0, len(resolved.Steps)) + for _, step := range resolved.Steps { + laneIDs := make([]string, 0, len(step.ArtifactLanes)) + for _, lane := range step.ArtifactLanes { + laneIDs = append(laneIDs, lane.ID) + } + sort.Strings(laneIDs) + result = append(result, step.ID+":"+strings.Join(laneIDs, ",")) + } + return result +} + func TestMaintainedMinimalInvocationProducesJSONBundle(t *testing.T) { outputRoot := filepath.Join(t.TempDir(), "output") fake := &productionFakeLLMClient{} @@ -56,7 +90,7 @@ func TestMaintainedMinimalInvocationProducesJSONBundle(t *testing.T) { var stdout, stderr strings.Builder code := RunWithOptions([]string{ "run", "dnd-session", - "--config", repositoryPath("examples", "dnd-spells.config.yml"), + "--config", repositoryPath("examples", "dnd-minimal.config.yml"), "--input", repositoryPath("examples", "seriatim-minimal-transcript.json"), "--only", "spells", "--chunk_cache", "bypass", "--output-dir", outputRoot, }, &stdout, &stderr, options) @@ -130,7 +164,7 @@ func TestMaintainedMalformedInputOnlyRecordsDebugFailureWhenRequested(t *testing options := productionRunOptions(t, &productionFakeLLMClient{}) args := []string{ "run", "dnd-session", - "--config", repositoryPath("examples", "dnd-spells.config.yml"), + "--config", repositoryPath("examples", "dnd-minimal.config.yml"), "--input", malformed, "--chunk_cache", "bypass", "--output-dir", outputRoot, } if debug { diff --git a/internal/cli/npc_registry_contract_test.go b/internal/cli/npc_registry_contract_test.go index 7621eb9..6d102c8 100644 --- a/internal/cli/npc_registry_contract_test.go +++ b/internal/cli/npc_registry_contract_test.go @@ -3,6 +3,7 @@ package cli import ( "context" "errors" + "fmt" "io/fs" "os" "path/filepath" @@ -22,9 +23,24 @@ func TestOversizedNPCRegistryFailsBeforeRuntimeAndCheckpointConstruction(t *test t.Fatal(err) } checkpointRoot := filepath.Join(t.TempDir(), "checkpoints") - content := string(readRepositoryFile(t, "examples", "dnd-spells.config.yml")) - content = replaceRequiredOnce(t, content, " extract: dnd/spells", " extract:\n module: dnd/spells\n references:\n npcs: "+npcPath) - content = replaceRequiredOnce(t, content, " enabled: false\n directory: \"\"", " enabled: true\n directory: "+checkpointRoot) + content := fmt.Sprintf(`version: 3 +cache: + chunk_plans: + mode: bypass + checkpoints: + enabled: true + directory: %q +pipelines: + dnd-session: + input: seriatim + artifacts: + spells: + extract: + module: dnd/spells + references: + npcs: %q + normalize: dnd/spells +`, checkpointRoot, npcPath) configPath := filepath.Join(t.TempDir(), "config.yml") if err := os.WriteFile(configPath, []byte(content), 0o600); err != nil { t.Fatal(err) diff --git a/internal/cli/production_contract_test.go b/internal/cli/production_contract_test.go index c1350dc..aed5861 100644 --- a/internal/cli/production_contract_test.go +++ b/internal/cli/production_contract_test.go @@ -147,7 +147,7 @@ func TestProductionCatalogCoversMaintainedConfigurations(t *testing.T) { func TestDefaultCLICompositionValidatesRepresentativeConfiguration(t *testing.T) { var stdout, stderr strings.Builder code := RunWithOptions([]string{ - "config", "validate", "--config", repositoryPath("examples", "dnd-spells.config.yml"), "--pipeline", "dnd-session", + "config", "validate", "--config", repositoryPath("examples", "dnd-minimal.config.yml"), "--pipeline", "dnd-session", }, &stdout, &stderr, Options{}) if code != 0 || stderr.Len() != 0 { t.Fatalf("validate representative config with default composition: code=%d stdout=%q stderr=%q", code, stdout.String(), stderr.String()) @@ -176,7 +176,7 @@ func TestProductionPromptAssetsPrepareWithoutProviderCredentials(t *testing.T) { func TestProductionSpellValidatorsPrepareFromMaterializedCatalog(t *testing.T) { components := productionTestComponents(t) - configPath := repositoryPath("examples", "dnd-spells-production.config.yml") + configPath := writeProductionSpellCatalogContractConfig(t) effective, err := loadMaintainedExample(t, configPath).Resolve(resolveInputForMaintainedExample(components, "dnd-session")) if err != nil { t.Fatalf("resolve production spell configuration: %v", err) @@ -203,7 +203,7 @@ func TestProductionSpellValidatorsPrepareFromMaterializedCatalog(t *testing.T) { func TestProductionSpellNormalizerRejectsInvalidCatalogReferencesBeforeExecution(t *testing.T) { components := productionTestComponents(t) - configPath := repositoryPath("examples", "dnd-spells-production.config.yml") + configPath := writeProductionSpellCatalogContractConfig(t) resolve := func(t *testing.T) pipeline.ResolvedPipeline { t.Helper() effective, err := loadMaintainedExample(t, configPath).Resolve(resolveInputForMaintainedExample(components, "dnd-session")) @@ -261,12 +261,15 @@ func TestProductionSpellNormalizerRejectsInvalidCatalogReferencesBeforeExecution t.Fatal(err) } checkpointRoot := filepath.Join(t.TempDir(), "checkpoints") - content := string(readRepositoryFile(t, "examples", "dnd-spells-production.config.yml")) - content = replaceRequiredOnce(t, content, "./dnd-spells-roster.txt", repositoryPath("examples", "dnd-spells-roster.txt")) - content = replaceRequiredOnce(t, content, "./dnd-spells-glossary.txt", repositoryPath("examples", "dnd-spells-glossary.txt")) - content = strings.Replace(content, "./dnd-spells-catalog.json", repositoryPath("examples", "dnd-spells-catalog.json"), 1) - content = replaceRequiredOnce(t, content, "./dnd-spells-catalog.json", catalogPath) - content = replaceRequiredOnce(t, content, " enabled: false\n directory: /var/cache/notarius/checkpoints", " enabled: true\n directory: "+checkpointRoot) + content := productionSpellCatalogContractConfig(t) + catalogSource := repositoryPath("examples", "dnd-spell-catalog.json") + if count := strings.Count(content, catalogSource); count != 2 { + t.Fatalf("spell catalog source occurs %d times, want extract and normalize bindings", count) + } + content = strings.Replace(content, catalogSource, "__extract_catalog__", 1) + content = replaceRequiredOnce(t, content, catalogSource, catalogPath) + content = replaceRequiredOnce(t, content, "__extract_catalog__", catalogSource) + content = replaceRequiredOnce(t, content, " enabled: false\n directory: \"\"", " enabled: true\n directory: "+checkpointRoot) configFile := filepath.Join(t.TempDir(), "config.yml") if err := os.WriteFile(configFile, []byte(content), 0o600); err != nil { t.Fatal(err) @@ -382,7 +385,7 @@ func TestProductionLLMClientFactoriesRejectInvalidConstruction(t *testing.T) { } func TestProductionConfigValidationCoversModuleAndVariantFailures(t *testing.T) { - base := string(readRepositoryFile(t, "examples", "dnd-spells.config.yml")) + base := string(readRepositoryFile(t, "examples", "dnd-minimal.config.yml")) validPath := writeProductionContractConfig(t, base) options := productionCLIOptions(t) var stdout, stderr strings.Builder @@ -438,7 +441,7 @@ func TestProductionConfigValidationCoversModuleAndVariantFailures(t *testing.T) } func TestProductionNormalizeValidatorOverrideRemainsAuthoritative(t *testing.T) { - base := string(readRepositoryFile(t, "examples", "dnd-spells.config.yml")) + base := string(readRepositoryFile(t, "examples", "dnd-minimal.config.yml")) content := replaceRequiredOnce(t, base, " normalize: dnd/spells\n", " normalize:\n module: dnd/spells\n validators:\n - module: generic/always_accept\n - module: generic/valid_json\n") path := writeProductionContractConfig(t, content) components := productionTestComponents(t) @@ -531,17 +534,45 @@ type maintainedExample struct { func maintainedExampleFiles(t *testing.T) []maintainedExample { t.Helper() return []maintainedExample{ - {name: "minimal", path: repositoryPath("examples", "dnd-spells.config.yml"), pipelineIDs: []string{"dnd-session"}}, - {name: "production", path: repositoryPath("examples", "dnd-spells-production.config.yml"), pipelineIDs: []string{"dnd-session"}}, - {name: "npcs", path: repositoryPath("examples", "dnd-npcs.config.yml"), pipelineIDs: []string{"dnd-session"}}, - {name: "combat", path: repositoryPath("examples", "dnd-combat-turns.config.yml"), pipelineIDs: []string{"dnd-combat"}}, - {name: "npc-grounded", path: repositoryPath("examples", "dnd-npc-grounded.config.yml"), pipelineIDs: []string{"dnd-npc-grounded"}}, - {name: "npc-interactions", path: repositoryPath("examples", "dnd-npc-interactions.config.yml"), pipelineIDs: []string{"dnd-npc-interactions"}}, - {name: "scene-descriptions", path: repositoryPath("examples", "dnd-scene-descriptions.config.yml"), pipelineIDs: []string{"dnd-scene-descriptions"}}, - {name: "scene-chunk-map", path: repositoryPath("examples", "dnd-scene-chunk-map.config.yml"), pipelineIDs: []string{"dnd-scene-chunk-map"}}, + {name: "minimal", path: repositoryPath("examples", "dnd-minimal.config.yml"), pipelineIDs: []string{"dnd-session"}}, + {name: "complete", path: repositoryPath("examples", "dnd-complete.config.yml"), pipelineIDs: []string{"dnd-session"}}, } } +func productionSpellCatalogContractConfig(t *testing.T) string { + t.Helper() + return fmt.Sprintf(`version: 3 +cache: + chunk_plans: + mode: bypass + checkpoints: + enabled: false + directory: "" +pipelines: + dnd-session: + input: seriatim + references: + party: %q + glossary: %q + artifacts: + spells: + extract: + module: dnd/spells + retries: 2 + references: + spell_catalog: %q + normalize: + module: dnd/spells + references: + spell_catalog: %q +`, repositoryPath("examples", "dnd-party.txt"), repositoryPath("examples", "dnd-glossary.txt"), repositoryPath("examples", "dnd-spell-catalog.json"), repositoryPath("examples", "dnd-spell-catalog.json")) +} + +func writeProductionSpellCatalogContractConfig(t *testing.T) string { + t.Helper() + return writeProductionContractConfig(t, productionSpellCatalogContractConfig(t)) +} + func loadMaintainedExample(t *testing.T, path string) config.Config { t.Helper() fileConfig, err := config.LoadFileConfig(path) diff --git a/internal/cli/reference_contract_test.go b/internal/cli/reference_contract_test.go index 7d4b98e..4cf5f62 100644 --- a/internal/cli/reference_contract_test.go +++ b/internal/cli/reference_contract_test.go @@ -418,9 +418,11 @@ func (referenceContractCodecB) Decode([]byte) (stateTestArtifact, error) { func referenceContractLane(t *testing.T, resolved pipeline.ResolvedPipeline, id string) pipeline.ResolvedArtifactLane { t.Helper() - for _, lane := range resolved.Steps[0].ArtifactLanes { - if lane.ID == id { - return lane + for _, step := range resolved.Steps { + for _, lane := range step.ArtifactLanes { + if lane.ID == id { + return lane + } } } t.Fatalf("lane %q not found", id) diff --git a/internal/cli/spell_catalog_identity_contract_test.go b/internal/cli/spell_catalog_identity_contract_test.go index 2746a68..50560b7 100644 --- a/internal/cli/spell_catalog_identity_contract_test.go +++ b/internal/cli/spell_catalog_identity_contract_test.go @@ -24,7 +24,7 @@ import ( func TestSpellCatalogBytesAffectCheckpointIdentityButNotSemanticDigest(t *testing.T) { components := productionTestComponents(t) - configPath := repositoryPath("examples", "dnd-spells-production.config.yml") + configPath := writeProductionSpellCatalogContractConfig(t) effective, err := loadMaintainedExample(t, configPath).Resolve(resolveInputForMaintainedExample(components, "dnd-session")) if err != nil { t.Fatalf("resolve production configuration: %v", err) @@ -108,8 +108,8 @@ func TestSpellCatalogBytesAffectCheckpointIdentityButNotSemanticDigest(t *testin } func TestConfiguredSpellCatalogBindingChangesResolvedPipelineIdentity(t *testing.T) { - base := string(readRepositoryFile(t, "examples", "dnd-spells-production.config.yml")) - changed := strings.Replace(base, "./dnd-spells-catalog.json", "./alternate-spell-catalog.json", 1) + base := productionSpellCatalogContractConfig(t) + changed := strings.Replace(base, repositoryPath("examples", "dnd-spell-catalog.json"), filepath.Join(t.TempDir(), "alternate-spell-catalog.json"), 1) if changed == base { t.Fatal("production configuration did not contain the maintained catalog binding") } @@ -138,7 +138,7 @@ func TestConfiguredSpellCatalogBindingChangesResolvedPipelineIdentity(t *testing func TestSemanticSpellCatalogFingerprintChangesCheckpointIdentityWithoutReferenceChange(t *testing.T) { components := productionTestComponents(t) - configPath := repositoryPath("examples", "dnd-spells-production.config.yml") + configPath := writeProductionSpellCatalogContractConfig(t) effective, err := loadMaintainedExample(t, configPath).Resolve(resolveInputForMaintainedExample(components, "dnd-session")) if err != nil { t.Fatal(err) @@ -200,7 +200,7 @@ func TestSemanticSpellCatalogFingerprintChangesCheckpointIdentityWithoutReferenc func TestChangedSemanticSpellCatalogFingerprintCannotResumeRecordedCheckpoint(t *testing.T) { components := productionTestComponents(t) - configPath := repositoryPath("examples", "dnd-spells-production.config.yml") + configPath := writeProductionSpellCatalogContractConfig(t) effective, err := loadMaintainedExample(t, configPath).Resolve(resolveInputForMaintainedExample(components, "dnd-session")) if err != nil { t.Fatal(err) @@ -328,7 +328,7 @@ func TestMaintainedProductionOverlayRunAlignsGroundingValidationAndProvenance(t var stdout, stderr strings.Builder code := RunWithOptions([]string{ "run", "dnd-session", - "--config", repositoryPath("examples", "dnd-spells-production.config.yml"), + "--config", writeProductionSpellCatalogContractConfig(t), "--input", repositoryPath("examples", "seriatim-minimal-transcript.json"), "--only", "spells", "--chunk_cache", "bypass", "--output-dir", outputRoot, }, &stdout, &stderr, options) @@ -373,12 +373,12 @@ func TestMaintainedProductionOverlayRunAlignsGroundingValidationAndProvenance(t if len(catalogProvenances) != 2 { t.Fatalf("manifest references = %#v, want independently materialized extract and normalize catalog provenance", manifest.References) } - overlayBytes := readRepositoryFile(t, "examples", "dnd-spells-catalog.json") + overlayBytes := readRepositoryFile(t, "examples", "dnd-spell-catalog.json") for _, catalogProvenance := range catalogProvenances { if catalogProvenance.Stage != "extract" && catalogProvenance.Stage != "normalize" { t.Fatalf("catalog provenance = %#v, want extract or normalize scope", catalogProvenance) } - if catalogProvenance.LaneID != "spells" || catalogProvenance.OriginType != "file" || catalogProvenance.MediaType != "application/json" || catalogProvenance.SizeBytes != int64(len(overlayBytes)) || catalogProvenance.Digest != digestBytes(overlayBytes) || !strings.Contains(catalogProvenance.OriginURI, "dnd-spells-catalog.json") { + if catalogProvenance.LaneID != "spells" || catalogProvenance.OriginType != "file" || catalogProvenance.MediaType != "application/json" || catalogProvenance.SizeBytes != int64(len(overlayBytes)) || catalogProvenance.Digest != digestBytes(overlayBytes) || !strings.Contains(catalogProvenance.OriginURI, "dnd-spell-catalog.json") { t.Fatalf("catalog provenance = %#v, want raw overlay provenance in both scopes", catalogProvenance) } } diff --git a/internal/cli/spell_catalog_retry_contract_test.go b/internal/cli/spell_catalog_retry_contract_test.go index d4f7584..a6c54e6 100644 --- a/internal/cli/spell_catalog_retry_contract_test.go +++ b/internal/cli/spell_catalog_retry_contract_test.go @@ -4,6 +4,7 @@ import ( "context" "encoding/json" "fmt" + "path/filepath" "sync" "testing" @@ -50,7 +51,7 @@ func TestProductionSpellCatalogValidationRetries(t *testing.T) { for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { components := productionTestComponents(t) - configPath := repositoryPath("examples", "dnd-spells-production.config.yml") + configPath := writeProductionSpellCatalogContractConfig(t) cfg := loadMaintainedExample(t, configPath) effective, err := cfg.Resolve(config.ResolveInput{PipelineID: "dnd-session", Catalog: catalogFromRegistries(components.registries)}) if err != nil { @@ -58,7 +59,7 @@ func TestProductionSpellCatalogValidationRetries(t *testing.T) { } materialized, _, err := pipeline.MaterializeReferences(effective.ResolvedPipeline, catalogFromRegistries(components.registries), pipeline.ReferenceMaterializationOptions{ ConfigPath: configPath, - WorkingDir: repositoryPath("examples"), + WorkingDir: filepath.Dir(configPath), }) if err != nil { t.Fatalf("materialize production references: %v", err)