Compare commits

..

7 Commits

28 changed files with 2248 additions and 8 deletions

View File

@@ -1,3 +1,6 @@
Please carefully review the documents in `docs/policy` before making any changes to this repository.
- `architecture.md` provides the canonical high-level architecture policy for this repository.
- `documentation.md` provides the canonical documentation policy for this repository.
Please review `docs/internal/overview.md` for initial orientation in this repository.
Additionally, please carefully review the relevant documents in `docs/policy` before making any changes to this repository.
- `development.md` defines the contributor workflow for this application.
- `architecture.md` provides the canonical high-level architecture policy for this repository, and should be reviewed before writing or changing any code.
- `documentation.md` provides the canonical documentation policy for this repository, and should be reviewed before writing or changing any documentation.

View File

@@ -117,7 +117,7 @@ go run ./cmd/notarius pipelines list \
The production CLI currently registers these module keys:
- input: `seriatim`
- chunk: `generic`
- chunk: `generic`, `dnd/scenes`
- extract: `dnd/spells`
- merge: `appendorder`
- normalize: `noop`

View File

@@ -167,6 +167,7 @@ one configured profile.
| --- | --- | --- |
| input | `seriatim` | Reads Seriatim transcript JSON. |
| chunk | `generic` | Splits source units into ordered chunks. |
| chunk | `dnd/scenes` | Uses an LLM to split transcript source units into D&D scenes. |
| extract | `dnd/spells` | Extracts `dnd.spell_cast` artifacts. |
| merge | `appendorder` | Keeps candidates in append order. |
| normalize | `noop` | Passes merged artifacts through unchanged. |
@@ -178,6 +179,9 @@ The `generic` chunker accepts:
- `overlap_units`: non-negative integer, default `0`, and must be less than
`max_units`.
The `dnd/scenes` chunker requires transcript source capabilities, calls the
configured structured LLM provider, and does not accept module options.
## Diagnostics
`diagnostics` fields:

View File

@@ -20,6 +20,11 @@ A production module package should provide:
Module specs should describe capabilities accurately. Resolution uses specs to
reject incompatible pipelines before execution.
Chunk modules receive the structured LLM client through `contracts.ChunkRequest`
when they need model-backed chunking. The pipeline runner validates generic
chunk result invariants before extraction; module-owned policies may be stricter
but must stay within the module package.
## `seriatim` Input
Package: `internal/modules/input/seriatim`
@@ -54,6 +59,37 @@ Provides:
- `chunks`
## `dnd/scenes` Chunker
Package: `internal/modules/chunk/dnd/scenes`
The `dnd/scenes` chunker uses the structured LLM client to divide transcript
source units into coherent D&D scenes. It renders embedded prompts, loads the
embedded structured response schema, validates model-authored source-unit
boundaries, and converts each scene into a deterministic source chunk.
Requires:
- `source.transcript`
Provides:
- `chunks`
- `chunks.scenes`
Options: none. Non-empty options are rejected.
The chunker enforces full source-unit coverage from the first source unit to the
last, exact source-unit IDs, sequential contiguous scenes, and no overlap. It
assigns chunk IDs such as `scene-000001` and stores scene metadata including
title, primary mode, participants, summary, boundary note, confidence, boundary
unit IDs, and unit count. Boundary caveats become warnings with reason code
`scene_boundary_caveat`.
Malformed model output fails explicitly rather than falling back to another
chunker. The chunker exposes prompt and response-schema provenance through its
metadata provider without raw prompts, raw schemas, source text, or secrets.
## `dnd/spells` Extractor
Package: `internal/modules/extract/dnd/spells`

View File

@@ -73,8 +73,34 @@ The runner:
2. builds the input adapter and parses the raw input into a source document;
3. validates the source document;
4. builds the chunker and produces source chunks;
5. runs each selected artifact lane in sorted resolved order;
6. builds the output encoder and validates logical output file names.
5. validates source chunks against framework invariants;
6. runs each selected artifact lane in sorted resolved order;
7. builds the output encoder and validates logical output file names.
## Chunk Results
Chunkers implement `contracts.Chunker` and receive a `contracts.ChunkRequest`
with the validated source document, the structured LLM client, the configured
LLM profile, module options, and run metadata. Deterministic and LLM-backed
chunkers use the same contract; provider construction stays outside chunk
modules.
After `Chunk` returns, the runner appends chunker warnings before returning any
chunker error. When chunking succeeds, the runner validates generic chunk
invariants before running extractors:
- chunk IDs must be non-empty and unique in the chunk result;
- each chunk `SourceID` must match the source document ID;
- each chunk `Index` must match its zero-based returned order;
- each chunk must contain at least one source unit;
- a chunk must not repeat a source unit;
- every chunk source unit must exist in the source document;
- source units inside each chunk must appear in source-document order.
The framework does not require complete source-unit coverage and does not reject
overlap between different chunks. Stricter policies, such as full coverage or
non-overlap, belong to individual chunk modules when they are part of that
module's contract.
Within an artifact lane, the runner:

104
docs/roadmap/chunk.md Normal file
View File

@@ -0,0 +1,104 @@
# Chunk Module Roadmap
Current Notarius behavior is documented in the canonical README, CLI,
configuration, operations, internal, and integration docs. This roadmap records
future chunk-module behavior only.
## Goal
Chunk modules should be a clear module-author boundary, and LLM-backed chunking
should be a first-class capability.
The immediate target is a D&D-specific scene chunker that divides transcript
source units into coherent scenes before extraction. The broader target is that
any chunk module can be implemented as a black box when it satisfies the
framework chunk contract.
## Target Chunk Contract
The framework chunk contract should support deterministic and LLM-backed
chunkers through the same module interface.
Chunkers should receive runtime dependencies from the runner, including the
structured LLM client when a chunker needs model calls. Chunkers should not
construct provider clients internally.
The framework should validate these result invariants for every chunk module:
- chunk IDs are non-empty and unique within a run;
- each chunk references the input source document ID;
- chunk indexes are deterministic and sequential in returned order;
- each chunk contains at least one source unit;
- each chunk source unit comes from the source document;
- source units within each chunk appear in source-document order.
The framework should not require complete source-unit coverage and should not
forbid overlap between chunks. Individual chunk modules may enforce stricter
policies, such as full coverage or non-overlap, when those policies are part of
the module's own contract.
Chunk metadata should remain flexible and module-owned. Framework code should
preserve chunk metadata and pass it to downstream modules, but it should not
adopt transcript-specific or D&D-specific metadata fields.
## D&D Scene Chunker Target
The D&D scene chunker should live under
`internal/modules/chunk/dnd/scenes` and use the module key `dnd/scenes`.
It should require transcript source capabilities and provide the generic
`chunks` capability plus a scene-specific chunk capability. D&D scene-boundary
prompt logic, response-schema interpretation, and stricter scene policies belong
inside the module.
The module should use the framework structured LLM client for scene-boundary
detection. The model response should describe source-unit boundaries and useful
scene metadata; the Go module should validate the response and convert it into
`contracts.SourceChunk` values.
For `dnd/scenes`, the module-owned policy should be:
- cover the full source document from first source unit to last source unit;
- return sequential, contiguous, non-overlapping scenes;
- use exact source-unit IDs for boundaries;
- fail with actionable errors for malformed model output rather than silently
falling back to a generic chunker.
Scene chunk IDs and indexes should be assigned by the module, not trusted from
model output. Useful scene information should be stored in chunk metadata, such
as title, primary mode, participants, summary, boundary note, and boundary
confidence. Overall boundary caveats should be surfaced as chunker warnings.
## Draft Asset Target
Initial D&D scene chunker prompt and schema drafts exist under
`internal/modules/chunk/dnd/scenes/assets`. They should be revised before the
module is implemented.
The response schema should be versioned and named consistently with existing
module-owned response schemas, such as `dnd_scenes.v1.json`, with a schema key,
schema ID, schema version, and OpenAI-compatible response schema name.
Boundary fields should use source-unit ID strings, not integer segment IDs.
The schema should focus on boundary and metadata decisions rather than final
framework chunk fields. Prompt terminology and schema terminology should match
exactly, including primary mode enum values and boundary field names.
The user prompt should be a Go template that includes source document ID,
chunking scope, ordered source units, and selected metadata such as speaker and
timestamps when available. It may contain D&D-specific scene guidance, but it
should not imply that the framework itself is transcript-specific.
## Documentation Target
Current-behavior docs should be updated only after the corresponding behavior is
implemented.
Internal module-author documentation should eventually define the chunk module
API, including `Chunker`, `ChunkRequest`, `ChunkResult`, `SourceChunk`,
validation invariants, warning semantics, LLM-backed chunker expectations,
module specs, capability guidance, and option parsing expectations.
When `dnd/scenes` becomes production behavior, configuration, CLI, internal
module, and troubleshooting docs should describe the implemented module and its
failure modes.

View File

@@ -0,0 +1,312 @@
# Chunk Module Implementation Plan
This plan implements the accepted target state in
[Chunk Module Roadmap](chunk.md). It is written for an LLM coding agent that
will implement each stage in order.
Before beginning any stage, review:
- `docs/policy/architecture.md`
- `docs/policy/development.md`
- `docs/policy/documentation.md`
- `docs/roadmap/chunk.md`
Do not move planned behavior into non-roadmap docs until the corresponding code
is implemented. Do not revert unrelated user changes.
## Stage 1: Framework Chunk Contract
Goal: make LLM-backed chunking first-class and enforce generic chunk result
invariants without adding any D&D-specific framework behavior.
Code changes:
- Add `LLMClient contracts.StructuredLLMClient` to `contracts.ChunkRequest` in
`internal/framework/contracts/contracts.go`.
- Update `internal/framework/pipeline/runner.go` so the runner passes
`input.LLMClient` to the chunker in `contracts.ChunkRequest`.
- Add framework-level chunk result validation after `chunker.Chunk` returns and
before lanes execute.
- Keep validation source-generic. The validator should reject:
- empty chunk ID;
- duplicate chunk ID;
- chunk `SourceID` that does not match the source document ID;
- chunk `Index` that does not match returned order;
- empty chunk units;
- repeated source unit inside one chunk;
- source unit not found in the source document;
- chunk units that do not appear in source-document order.
- The validator must not require complete coverage and must not reject overlap
between different chunks.
- Preserve existing warning behavior: append chunker warnings before returning
chunk errors, as the runner does today.
Documentation changes:
- Update implemented internal docs under `docs/internal/` to define the chunk
module API and validation invariants once the code exists.
- Keep examples and user docs unchanged in this stage unless an existing doc
becomes inaccurate.
Tests:
- Update contract tests for the new `ChunkRequest.LLMClient` field where useful.
- Add focused pipeline runner tests for each invalid chunk result case listed
above.
- Add runner tests proving partial coverage and overlapping chunks remain
accepted.
- Run:
```sh
go test ./internal/framework/contracts ./internal/framework/pipeline
```
Stage completion criteria:
- Existing generic chunking still works.
- A fake chunker can receive the structured LLM client through `ChunkRequest`.
- Framework tests prove the accepted generic chunk invariants.
## Stage 2: D&D Scene Assets And Module Skeleton
Goal: revise the draft D&D scene prompt and schema into module-owned assets and
add load/render plumbing without registering production behavior.
Asset decisions:
- Rename `internal/modules/chunk/dnd/scenes/assets/schemas/scene_map.schema.json`
to `internal/modules/chunk/dnd/scenes/assets/schemas/dnd_scenes.v1.json`.
- Use these schema constants unless a code-local naming conflict requires a
mechanical adjustment:
- prompt ID: `dnd.scenes`
- response schema key: `dnd_scenes`
- response schema ID: `notarius.dnd.scenes`
- response schema version: `v1`
- response schema name: `notarius_dnd_scenes_v1`
- Use this structured response shape:
```json
{
"scenes": [
{
"start_unit_id": "seg-001",
"end_unit_id": "seg-010",
"short_title": "Ambush at the gate",
"primary_mode": "Combat",
"main_participants": ["Aria", "Bandit mage"],
"summary": "The party fights the bandit mage at the gate.",
"boundary_note": "The scene begins when combat starts and ends when the immediate threat is resolved.",
"boundary_confidence": "High"
}
],
"boundary_caveats": []
}
```
- Required top-level fields: `scenes`, `boundary_caveats`.
- Required scene fields: `start_unit_id`, `end_unit_id`, `short_title`,
`primary_mode`, `main_participants`, `summary`, `boundary_note`,
`boundary_confidence`.
- Boundary fields are source-unit ID strings, not integers.
- `primary_mode` enum: `Recap`, `Discussion`, `Combat`, `Narrative`.
- `boundary_confidence` enum: `High`, `Medium`, `Low`.
- Keep `additionalProperties: false` throughout the schema.
- Do not include model-authored final chunk IDs or chunk indexes in the schema.
The Go module assigns deterministic chunk IDs and indexes.
Prompt decisions:
- Keep D&D-specific scene guidance in the D&D scene module.
- Make the user prompt a Go template similar to the spell extractor prompt.
- Include source document ID and ordered source units.
- Include selected source-unit metadata when present: `speaker`, `start`, and
`end`.
- Align prompt terms exactly with schema field names and enum values.
- Keep `dnd/scenes` module policy explicit in the prompt: full coverage,
sequential scenes, no gaps, no overlap, exact source-unit IDs.
Code changes:
- Add `assets.go` with an `embed.FS` for prompts and schemas.
- Add `schema.go` with the constants and a `loadResponseSchema` function using
`llm.LoadResponseSchema`, following the pattern in
`internal/modules/extract/dnd/spells/schema.go`.
- Add prompt rendering code using `framework/prompt.Bundle`, following the
pattern in `internal/modules/extract/dnd/spells/prompt.go`.
- Add internal response structs for the schema shape.
- Do not register the module in `internal/cli/catalog.go` in this stage.
Tests:
- Add tests that the schema loads, is valid JSON, has the expected metadata, and
rejects the old integer-boundary assumption through Go-side type expectations.
- Add prompt rendering tests that source unit IDs and selected metadata appear
in the rendered user prompt.
- Run:
```sh
go test ./internal/modules/chunk/dnd/scenes
```
Stage completion criteria:
- The scene schema and prompts are loadable embedded assets.
- The prompt/schema terminology is internally consistent.
- No production catalog behavior changes yet.
## Stage 3: D&D Scene Chunker Implementation
Goal: implement `dnd/scenes` as a contract-compliant chunk module with strict
module-owned validation.
Module decisions:
- Package path: `internal/modules/chunk/dnd/scenes`.
- Package name: `scenes`.
- Module key: `dnd/scenes`.
- `ModuleSpec`:
- `Stage`: `pipeline.StageChunk`
- `Requires`: `source.transcript`
- `Provides`: `chunks`, `chunks.scenes`
- Constructor: `New() *Chunker`.
- Registration function: `Register(registry *pipeline.ChunkerRegistry) error`.
- No module options initially. Reject non-empty options with an actionable
module-prefixed error unless a clear option is implemented in the same stage.
Chunking behavior:
- Validate `context.Context`, source document, non-empty source units, and
non-nil `LLMClient`.
- Render the scene prompt over the full source document.
- Call `LLMClient.CompleteStructured` with:
- `StageName`: `dnd/scenes`
- response schema name and schema JSON from the module schema loader.
- Validate the decoded response before producing chunks:
- `scenes` must be present and non-empty;
- every boundary ID must exist in the source document;
- each scene start must be at or before its end;
- the first scene starts at the first source unit;
- the final scene ends at the final source unit;
- scenes are contiguous in source order;
- scenes do not overlap;
- required metadata fields are non-empty after trimming;
- `main_participants` entries are trimmed and empty entries rejected.
- Assign deterministic chunk fields:
- `ID`: `scene-000001`, `scene-000002`, and so on;
- `SourceID`: source document ID;
- `Index`: zero-based returned order;
- `Units`: defensive copies of the source units in the scene range.
- Store per-scene metadata on each chunk:
- `scene_title`
- `primary_mode`
- `main_participants`
- `summary`
- `boundary_note`
- `boundary_confidence`
- `start_unit_id`
- `end_unit_id`
- `unit_count`
- Convert each `boundary_caveats` entry into a `contracts.Warning` with:
- `Scope`: `dnd/scenes`
- `ReasonCode`: `scene_boundary_caveat`
- `Message`: the caveat text.
- Fail explicitly for malformed model output. Do not fall back to `generic`.
- Implement `contracts.ManifestMetadataProvider` and include prompt and
response-schema provenance without raw prompts, raw schemas, source text, or
secrets.
Tests:
- Registration and `ModuleSpec`.
- Successful chunking from a fake LLM response.
- Prompt request uses the expected schema name and schema JSON.
- Caveats become warnings.
- Defensive copy behavior for source units and metadata.
- Errors for nil context, nil source, invalid source, nil LLM client, empty
model scenes, unknown boundary ID, out-of-order boundaries, gaps, overlap,
incomplete coverage, empty metadata fields, and non-empty unsupported options.
- Manifest metadata contains prompt/schema provenance.
- Run:
```sh
go test ./internal/modules/chunk/dnd/scenes
go test ./internal/framework/pipeline
```
Stage completion criteria:
- `dnd/scenes` works in focused tests with fake LLM clients.
- It is still not production-registered unless Stage 4 is completed.
## Stage 4: Production Registration And Implemented Docs
Goal: make `dnd/scenes` available in production configuration and document only
the behavior that now exists.
Code changes:
- Register `dnd/scenes` in `internal/cli/catalog.go`.
- Add or update catalog/default module tests so the production catalog exposes
the new chunk module.
- Add CLI/config validation tests proving a pipeline can select
`chunk: dnd/scenes`.
- Do not change the existing maintained example config unless the related CLI
fixture tests are updated to keep it loadable and useful.
Documentation changes:
- Update `docs/config.md` implemented production module tables and chunk module
notes.
- Update `docs/cli.md` implemented production module list.
- Update `docs/internal/modules.md` with `dnd/scenes` behavior, capabilities,
metadata, and failure policy.
- Update or add internal chunk-module documentation if Stage 1 did not already
create a clear API reference.
- Update `docs/troubleshooting.md` for common scene chunker failures:
malformed model output, invalid boundaries, incomplete coverage, and provider
failures during chunking.
- Keep roadmap docs for any deferred options or future prompt tuning.
Tests:
```sh
go test ./internal/cli
go test ./internal/core/config
go test ./internal/framework/pipeline
go test ./internal/modules/chunk/dnd/scenes
```
Stage completion criteria:
- Config resolution can bind `dnd/scenes`.
- User and internal docs describe the implemented module accurately.
- Existing examples and CLI docs remain truthful.
## Stage 5: Full Verification
Goal: verify the complete feature across contracts, production wiring, docs, and
the command entry point.
Run:
```sh
go test ./...
go vet ./...
go build ./cmd/notarius
```
Inspect diagnostics-sensitive output manually in tests or fixtures where
relevant:
- no raw prompts, source text, provider payloads, API keys, or secrets in
manifest metadata;
- errors name the module and operation;
- warnings are preserved in `RunOutput.Warnings`;
- run manifests record the `dnd/scenes` chunker when selected.
Stage completion criteria:
- Full validation commands pass.
- The feature is documented as implemented only where code supports it.
- `docs/roadmap/chunk.md` retains target-state context and does not duplicate
current-behavior reference material.

View File

@@ -177,6 +177,31 @@ Fix:
Provider error messages are redacted for configured API key values.
## Scene Chunking Failure
Symptoms include:
- `dnd scenes chunker`
- `malformed structured output`
- `start_unit_id`
- `end_unit_id`
- `gap`
- `overlap`
- `final scene`
- `complete structured output`
Fix:
- Validate the pipeline configuration and confirm the input module provides a
transcript source when using `chunk: dnd/scenes`.
- Confirm the LLM profile has a working OpenAI-compatible `base_url`, `model`,
and credentials.
- Inspect retained diagnostics for the run error and resolved pipeline.
- If the error names malformed structured output, retry with a model that
follows structured response schemas reliably.
- Scene boundaries must use exact source-unit IDs, cover the full source
document, be contiguous, and not overlap.
## Output Write Failure
Symptoms include:

View File

@@ -10,6 +10,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/chunk/dnd/scenes"
"gitea.maximumdirect.net/eric/notarius/internal/modules/chunk/generic"
"gitea.maximumdirect.net/eric/notarius/internal/modules/extract/dnd/spells"
"gitea.maximumdirect.net/eric/notarius/internal/modules/input/seriatim"
@@ -34,6 +35,9 @@ func productionRegistries() (pipeline.Registries, error) {
if err := generic.Register(registries.Chunkers); err != nil {
return pipeline.Registries{}, fmt.Errorf("register generic chunker: %w", err)
}
if err := scenes.Register(registries.Chunkers); err != nil {
return pipeline.Registries{}, fmt.Errorf("register dnd scenes chunker: %w", err)
}
if err := spells.Register(registries.Extractors); err != nil {
return pipeline.Registries{}, fmt.Errorf("register dnd spells extractor: %w", err)
}

View File

@@ -16,6 +16,7 @@ import (
"gitea.maximumdirect.net/eric/notarius/internal/core/diagnostics"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
"gitea.maximumdirect.net/eric/notarius/internal/modules/chunk/dnd/scenes"
"gitea.maximumdirect.net/eric/notarius/internal/modules/chunk/generic"
"gitea.maximumdirect.net/eric/notarius/internal/modules/extract/dnd/spells"
"gitea.maximumdirect.net/eric/notarius/internal/modules/input/seriatim"
@@ -136,6 +137,11 @@ func TestProductionCatalogIncludesDefaultModules(t *testing.T) {
got: func() (pipeline.ModuleSpec, bool) { return catalog.Chunkers.Spec(generic.Key) },
want: generic.ModuleSpec(),
},
{
name: "dnd scenes chunker",
got: func() (pipeline.ModuleSpec, bool) { return catalog.Chunkers.Spec(scenes.Key) },
want: scenes.ModuleSpec(),
},
{
name: "dnd spells extractor",
got: func() (pipeline.ModuleSpec, bool) { return catalog.Extractors.Spec(spells.Key) },
@@ -186,6 +192,21 @@ func TestRunConfigValidateUsesProductionCatalogByDefault(t *testing.T) {
}
}
func TestRunConfigValidateAcceptsDNDScenesChunker(t *testing.T) {
configPath := writeTestConfig(t, mvpConfigYAMLWithChunk("dnd-session", scenes.Key, "dnd/spells"))
var stdout bytes.Buffer
var stderr bytes.Buffer
code := RunWithOptions([]string{"config", "validate", "--config", configPath, "--pipeline", "dnd-session"}, &stdout, &stderr, Options{})
if code != 0 {
t.Fatalf("RunWithOptions() code = %d, stderr=%q", code, stderr.String())
}
if !strings.Contains(stdout.String(), "is valid for pipeline") {
t.Fatalf("stdout = %q, want validation success", stdout.String())
}
}
func TestRunConfigValidateUnknownProductionModuleIncludesContext(t *testing.T) {
configPath := writeTestConfig(t, mvpConfigYAML("dnd-session", "missing/extract"))
var stdout bytes.Buffer
@@ -1041,6 +1062,50 @@ func TestExampleFixtureRunWritesExpectedJSON(t *testing.T) {
}
}
func TestExampleFixtureRunWithDNDScenesRecordsChunkerAndWarnings(t *testing.T) {
configPath := writeTestConfig(t, mvpConfigYAMLWithChunk("dnd-session", scenes.Key, "dnd/spells"))
inputPath := fixturePath(t, "examples/seriatim-minimal-transcript.json")
outputDir := t.TempDir()
diagnosticsDir := t.TempDir()
client := newSceneRunLLMClient("Scene boundary was ambiguous.")
var stdout bytes.Buffer
var stderr bytes.Buffer
code := RunWithOptions([]string{"run", "dnd-session", "--config", configPath, "--input", inputPath, "--output-dir", outputDir, "--diagnostics-dir", diagnosticsDir}, &stdout, &stderr, Options{
LLMClientFactory: fakeLLMFactory(client, nil),
})
if code != 0 {
t.Fatalf("RunWithOptions() code = %d, stderr=%q", code, stderr.String())
}
if client.calls != 2 {
t.Fatalf("LLM calls = %d, want chunking and extraction calls", client.calls)
}
if !strings.Contains(stderr.String(), "1 warning") {
t.Fatalf("stderr = %q, want warning count", stderr.String())
}
runOutputDir := onlyChildDir(t, outputDir)
manifestBytes := readFile(t, filepath.Join(runOutputDir, "manifest.json"))
var manifest artifacts.RunManifest
if err := json.Unmarshal(manifestBytes, &manifest); err != nil {
t.Fatalf("unmarshal manifest: %v", err)
}
if manifest.Chunker != scenes.Key {
t.Fatalf("manifest chunker = %q, want %q", manifest.Chunker, scenes.Key)
}
for _, forbidden := range []string{"Source document ID:", "Aria casts Cure Wounds.", "spell_casts", "Scene boundary was ambiguous."} {
if strings.Contains(string(manifestBytes), forbidden) {
t.Fatalf("manifest leaked %q: %s", forbidden, manifestBytes)
}
}
warnings := string(readFile(t, filepath.Join(runOutputDir, "warnings.json")))
if !strings.Contains(warnings, "scene_boundary_caveat") || !strings.Contains(warnings, "Scene boundary was ambiguous.") {
t.Fatalf("warnings output = %s, want scene boundary caveat", warnings)
}
}
func TestExampleFixtureRunOnlySpells(t *testing.T) {
configPath := fixturePath(t, "examples/dnd-spells.config.yml")
inputPath := fixturePath(t, "examples/seriatim-minimal-transcript.json")
@@ -1220,6 +1285,18 @@ pipelines:
`
}
func mvpConfigYAMLWithChunk(pipelineID string, chunker string, extractor string) string {
return `version: 1
pipelines:
` + pipelineID + `:
input: seriatim
chunk: ` + chunker + `
artifacts:
spells:
extract: ` + extractor + `
`
}
func mvpConfigYAMLForLanes(pipelineID string, laneIDs ...string) string {
var b strings.Builder
b.WriteString("version: 1\n")
@@ -1287,6 +1364,7 @@ type fakeRunLLMClient struct {
calls int
err error
payload map[string]any
sceneCaveat string
}
func newFakeRunLLMClient(invalidSourceRef bool) *fakeRunLLMClient {
@@ -1301,11 +1379,43 @@ func newMalformedRunLLMClient() *fakeRunLLMClient {
return &fakeRunLLMClient{payload: map[string]any{}}
}
func newSceneRunLLMClient(caveat string) *fakeRunLLMClient {
return &fakeRunLLMClient{sceneCaveat: caveat}
}
func (client *fakeRunLLMClient) CompleteStructured(ctx context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
client.calls++
if client.err != nil {
return contracts.StructuredCompletionResponse{}, client.err
}
if req.StageName == scenes.Key && client.payload == nil {
payload := map[string]any{
"scenes": []map[string]any{
{
"start_unit_id": "seg-001",
"end_unit_id": "seg-002",
"short_title": "Opening spell",
"primary_mode": "Narrative",
"main_participants": []string{"Aria"},
"summary": "Aria casts a spell.",
"boundary_note": "The provided source units form one scene.",
"boundary_confidence": "High",
},
},
"boundary_caveats": []string{},
}
if client.sceneCaveat != "" {
payload["boundary_caveats"] = []string{client.sceneCaveat}
}
encoded, err := json.Marshal(payload)
if err != nil {
return contracts.StructuredCompletionResponse{}, err
}
if err := json.Unmarshal(encoded, out); err != nil {
return contracts.StructuredCompletionResponse{}, err
}
return contracts.StructuredCompletionResponse{Content: encoded}, nil
}
startUnitID := "seg-001"
if client.invalidSourceRef {
startUnitID = "missing-segment"

View File

@@ -98,6 +98,43 @@ func TestResolveSurfacesMissingCapabilityThroughCatalog(t *testing.T) {
}
}
func TestResolveCanBindSceneChunkerFromCatalog(t *testing.T) {
cfg := validConfig()
profile := cfg.Pipelines["example"]
profile.Chunk = pipeline.Binding("dnd/scenes")
lane := profile.Artifacts["events"]
profile.Artifacts = map[string]pipeline.ArtifactLaneProfile{"events": lane}
cfg.Pipelines["example"] = profile
catalog := fakeCatalog(t,
pipeline.ModuleSpec{
Key: "fake/input",
Stage: pipeline.StageInput,
Provides: []string{"source.transcript"},
},
pipeline.ModuleSpec{
Key: "fake/extract",
Stage: pipeline.StageExtract,
Requires: []string{"chunks", "source.transcript"},
Provides: []string{"artifact"},
},
)
mustRegisterChunker(t, catalog.Chunkers, pipeline.ModuleSpec{
Key: "dnd/scenes",
Stage: pipeline.StageChunk,
Requires: []string{"source.transcript"},
Provides: []string{"chunks", "chunks.scenes"},
})
effective, err := cfg.Resolve(ResolveInput{PipelineID: "example", Catalog: catalog})
if err != nil {
t.Fatalf("Resolve() error = %v, want nil", err)
}
if got := effective.ResolvedPipeline.Chunk.Module; got != "dnd/scenes" {
t.Fatalf("Chunk.Module = %q, want dnd/scenes", got)
}
}
func TestResolveDigestChangesWhenEffectiveConfigChanges(t *testing.T) {
cfg := validConfig()
first, err := cfg.Resolve(ResolveInput{PipelineID: "example", Catalog: fakeCatalog(t)})

View File

@@ -17,6 +17,7 @@ var _ contracts.Extractor = compositionExtractor{}
var _ contracts.Merger = compositionMerger{}
var _ contracts.Normalizer = compositionNormalizer{}
var _ contracts.Validator = compositionValidator{}
var _ contracts.StructuredLLMClient = compositionLLMClient{}
var _ contracts.OutputEncoder = compositionOutputEncoder{}
func TestContractsComposeAcrossPackages(t *testing.T) {
@@ -38,8 +39,9 @@ func TestContractsComposeAcrossPackages(t *testing.T) {
}
chunking, err := chunker.Chunk(ctx, contracts.ChunkRequest{
Source: doc,
Metadata: map[string]any{"max_units": 2},
Source: doc,
LLMClient: compositionLLMClient{},
Metadata: map[string]any{"max_units": 2},
})
if err != nil {
t.Fatalf("Chunk() error = %v, want nil", err)
@@ -164,6 +166,9 @@ func (chunker compositionChunker) Chunk(ctx context.Context, req contracts.Chunk
if req.Source == nil {
return contracts.ChunkResult{}, errors.New("source document is required")
}
if req.LLMClient == nil {
return contracts.ChunkResult{}, errors.New("structured llm client is required")
}
return contracts.ChunkResult{
Chunks: []contracts.SourceChunk{
@@ -178,6 +183,12 @@ func (chunker compositionChunker) Chunk(ctx context.Context, req contracts.Chunk
}, nil
}
type compositionLLMClient struct{}
func (client compositionLLMClient) CompleteStructured(ctx context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
return contracts.StructuredCompletionResponse{}, nil
}
type compositionExtractor struct{}
func (extractor compositionExtractor) Key() string {

View File

@@ -58,6 +58,7 @@ type SourceChunk struct {
type ChunkRequest struct {
Source *source.SourceDocument `json:"-"`
LLMClient StructuredLLMClient `json:"-"`
LLMProfile string `json:"llm_profile,omitempty"`
Options map[string]any `json:"options,omitempty"`
Metadata map[string]any `json:"metadata,omitempty"`

View File

@@ -117,6 +117,27 @@ func TestFakeChunkerReturnsSourceChunks(t *testing.T) {
}
}
func TestFakeChunkerReceivesLLMClient(t *testing.T) {
doc := &source.SourceDocument{
ID: "source-1",
Kind: "document",
Format: "text/plain",
Digest: "sha256:abc123",
Units: []source.SourceUnit{
{ID: "u1", Kind: "section", Text: "Source text."},
},
}
client := fakeLLMClient{}
chunker := &recordingChunker{key: "llm-chunker"}
if _, err := chunker.Chunk(context.Background(), ChunkRequest{Source: doc, LLMClient: client}); err != nil {
t.Fatalf("Chunk() error = %v, want nil", err)
}
if chunker.request.LLMClient == nil {
t.Fatal("ChunkRequest.LLMClient = nil, want structured LLM client")
}
}
func TestFakeExtractorReceivesChunkAndAmbientContext(t *testing.T) {
extractor := fakeExtractor{
key: "generic-extractor",
@@ -305,6 +326,20 @@ func (chunker fakeChunker) Chunk(ctx context.Context, req ChunkRequest) (ChunkRe
}, nil
}
type recordingChunker struct {
key string
request ChunkRequest
}
func (chunker *recordingChunker) Key() string {
return chunker.key
}
func (chunker *recordingChunker) Chunk(ctx context.Context, req ChunkRequest) (ChunkResult, error) {
chunker.request = req
return fakeChunker{key: chunker.key}.Chunk(ctx, req)
}
type fakeExtractor struct {
key string
artifactType string

View File

@@ -0,0 +1,60 @@
package pipeline
import (
"fmt"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
)
func validateChunkResult(doc *source.SourceDocument, chunks []contracts.SourceChunk) error {
sourceUnitIndexes := make(map[string]int, len(doc.Units))
for index, unit := range doc.Units {
sourceUnitIndexes[unit.ID] = index
}
seenChunkIDs := make(map[string]struct{}, len(chunks))
for chunkIndex, chunk := range chunks {
if strings.TrimSpace(chunk.ID) == "" {
return fmt.Errorf("chunk[%d].id must not be empty", chunkIndex)
}
if _, ok := seenChunkIDs[chunk.ID]; ok {
return fmt.Errorf("chunk id %q is duplicated", chunk.ID)
}
seenChunkIDs[chunk.ID] = struct{}{}
if chunk.SourceID != doc.ID {
return fmt.Errorf("chunk %q source_id %q does not match source document id %q", chunk.ID, chunk.SourceID, doc.ID)
}
if chunk.Index != chunkIndex {
return fmt.Errorf("chunk %q index %d does not match returned order %d", chunk.ID, chunk.Index, chunkIndex)
}
if len(chunk.Units) == 0 {
return fmt.Errorf("chunk %q units must not be empty", chunk.ID)
}
seenUnitIDs := make(map[string]struct{}, len(chunk.Units))
previousSourceIndex := -1
for unitIndex, unit := range chunk.Units {
if strings.TrimSpace(unit.ID) == "" {
return fmt.Errorf("chunk %q unit[%d].id must not be empty", chunk.ID, unitIndex)
}
if _, ok := seenUnitIDs[unit.ID]; ok {
return fmt.Errorf("chunk %q repeats source unit %q", chunk.ID, unit.ID)
}
seenUnitIDs[unit.ID] = struct{}{}
sourceIndex, ok := sourceUnitIndexes[unit.ID]
if !ok {
return fmt.Errorf("chunk %q source unit %q was not found in source document %q", chunk.ID, unit.ID, doc.ID)
}
if sourceIndex <= previousSourceIndex {
return fmt.Errorf("chunk %q source units must appear in source document order", chunk.ID)
}
previousSourceIndex = sourceIndex
}
}
return nil
}

View File

@@ -91,6 +91,7 @@ func (r *Runner) Run(ctx context.Context, input RunInput) (RunOutput, error) {
}
chunkResult, err := chunker.Chunk(ctx, contracts.ChunkRequest{
Source: doc,
LLMClient: input.LLMClient,
LLMProfile: input.Pipeline.Chunk.LLMProfile,
Options: cloneOptions(input.Pipeline.Chunk.Options),
Metadata: input.Metadata,
@@ -102,6 +103,9 @@ func (r *Runner) Run(ctx context.Context, input RunInput) (RunOutput, error) {
if len(chunkResult.Chunks) == 0 {
return failOutput(output), fmt.Errorf("chunker %q returned no chunks", chunker.Key())
}
if err := validateChunkResult(doc, chunkResult.Chunks); err != nil {
return failOutput(output), fmt.Errorf("validate chunks from chunker %q: %w", chunker.Key(), err)
}
nextCandidateIndex := 0
for _, lane := range input.Pipeline.ArtifactLanes {

View File

@@ -256,6 +256,91 @@ func TestRunRejectsChunkerBuildChunkAndEmptyChunkErrors(t *testing.T) {
}
}
func TestRunRejectsInvalidChunks(t *testing.T) {
tests := []struct {
name string
chunks []contracts.SourceChunk
want string
}{
{
name: "empty chunk id",
chunks: []contracts.SourceChunk{{ID: "", SourceID: "source-1", Index: 0, Units: []source.SourceUnit{unitWithID("u1")}}},
want: "id must not be empty",
},
{
name: "duplicate chunk id",
chunks: []contracts.SourceChunk{
{ID: "chunk-0", SourceID: "source-1", Index: 0, Units: []source.SourceUnit{unitWithID("u1")}},
{ID: "chunk-0", SourceID: "source-1", Index: 1, Units: []source.SourceUnit{unitWithID("u2")}},
},
want: "duplicated",
},
{
name: "wrong source id",
chunks: []contracts.SourceChunk{{ID: "chunk-0", SourceID: "other-source", Index: 0, Units: []source.SourceUnit{unitWithID("u1")}}},
want: "source_id",
},
{
name: "wrong index",
chunks: []contracts.SourceChunk{{ID: "chunk-0", SourceID: "source-1", Index: 1, Units: []source.SourceUnit{unitWithID("u1")}}},
want: "index",
},
{
name: "empty units",
chunks: []contracts.SourceChunk{{ID: "chunk-0", SourceID: "source-1", Index: 0}},
want: "units must not be empty",
},
{
name: "repeated unit inside chunk",
chunks: []contracts.SourceChunk{{ID: "chunk-0", SourceID: "source-1", Index: 0, Units: []source.SourceUnit{unitWithID("u1"), unitWithID("u1")}}},
want: "repeats source unit",
},
{
name: "unknown unit",
chunks: []contracts.SourceChunk{{ID: "chunk-0", SourceID: "source-1", Index: 0, Units: []source.SourceUnit{unitWithID("u9")}}},
want: "was not found",
},
{
name: "units out of source order",
chunks: []contracts.SourceChunk{{ID: "chunk-0", SourceID: "source-1", Index: 0, Units: []source.SourceUnit{unitWithID("u2"), unitWithID("u1")}}},
want: "source document order",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
modules := defaultRunnerModules()
modules.chunker.chunks = test.chunks
output, err := New(newRunnerRegistries(t, modules)).Run(context.Background(), RunInput{Pipeline: resolvedPipeline()})
assertRunError(t, err, test.want)
if output.Manifest.ValidationStatus != "failed" {
t.Fatalf("ValidationStatus = %q, want failed", output.Manifest.ValidationStatus)
}
if len(modules.extractors["extract-alpha"].requests) != 0 {
t.Fatalf("extractor calls = %d, want none after invalid chunks", len(modules.extractors["extract-alpha"].requests))
}
})
}
}
func TestRunAllowsPartialCoverageAndOverlappingChunks(t *testing.T) {
modules := defaultRunnerModules()
modules.chunker.chunks = []contracts.SourceChunk{
{ID: "chunk-0", SourceID: "source-1", Index: 0, Units: []source.SourceUnit{unitWithID("u1"), unitWithID("u2")}},
{ID: "chunk-1", SourceID: "source-1", Index: 1, Units: []source.SourceUnit{unitWithID("u2")}},
}
output, err := New(newRunnerRegistries(t, modules)).Run(context.Background(), RunInput{Pipeline: resolvedPipeline()})
if err != nil {
t.Fatalf("Run() error = %v, want nil", err)
}
if len(output.Approved) != 2 {
t.Fatalf("len(Approved) = %d, want one candidate per accepted chunk", len(output.Approved))
}
}
func TestRunExecutesChunksAndPassesChunkAndLLMClient(t *testing.T) {
modules := defaultRunnerModules()
llmClient := fakeLLMClient{}
@@ -273,6 +358,9 @@ func TestRunExecutesChunksAndPassesChunkAndLLMClient(t *testing.T) {
if !reflect.DeepEqual(extractor.seenChunkIDs, []string{"chunk-0", "chunk-1"}) {
t.Fatalf("seen chunks = %#v, want both chunks", extractor.seenChunkIDs)
}
if len(modules.chunker.requests) != 1 || modules.chunker.requests[0].LLMClient == nil {
t.Fatalf("chunker LLM client = %#v, want client on chunk request", modules.chunker.requests)
}
if len(extractor.seenLLMClients) != 2 || extractor.seenLLMClients[0] == nil || extractor.seenLLMClients[1] == nil {
t.Fatalf("seen LLM clients = %#v, want client for each chunk", extractor.seenLLMClients)
}
@@ -1179,6 +1267,8 @@ func validSourceDocument() *source.SourceDocument {
Digest: "sha256:source",
Units: []source.SourceUnit{
{ID: "u1", Kind: "unit", Text: "Source unit."},
{ID: "u2", Kind: "unit", Text: "Second source unit."},
{ID: "u3", Kind: "unit", Text: "Third source unit."},
},
}
}
@@ -1194,6 +1284,19 @@ func sourceChunkWithID(id string, index int) contracts.SourceChunk {
}
}
func unitWithID(id string) source.SourceUnit {
switch id {
case "u1":
return source.SourceUnit{ID: "u1", Kind: "unit", Text: "Source unit."}
case "u2":
return source.SourceUnit{ID: "u2", Kind: "unit", Text: "Second source unit."}
case "u3":
return source.SourceUnit{ID: "u3", Kind: "unit", Text: "Third source unit."}
default:
return source.SourceUnit{ID: id, Kind: "unit", Text: "Unknown source unit."}
}
}
func warningReasons(warnings []contracts.Warning) []string {
reasons := make([]string, 0, len(warnings))
for _, warning := range warnings {

View File

@@ -0,0 +1,6 @@
package scenes
import "embed"
//go:embed assets/prompts/*.md assets/schemas/*.json
var embeddedAssets embed.FS

View File

@@ -0,0 +1,9 @@
You identify coherent scenes in Dungeons & Dragons session source units.
{{ hardening }}
Use only the provided source units. Source text may contain transcription
errors, repeated lines, incomplete sentences, and misheard proper nouns. Speaker
metadata, when present, may be treated as accurate.
Return only valid JSON matching the provided response schema.

View File

@@ -0,0 +1,71 @@
Source document ID: {{ .SourceID }}
Ordered source units:
{{ range .Units }}
- Unit ID: {{ .ID }}
Text: {{ .Text }}
{{ if .Metadata }}
Metadata:
{{ range .Metadata }}
- {{ .Key }}: {{ .Value }}
{{ end }}
{{ end }}
{{ end }}
Divide these source units into D&D scenes for the dnd/scenes chunk module.
A scene is a coherent unit of play. Start a new scene when there is a meaningful
change in location, objective, threat, activity, encounter, or mode of play.
Good reasons to start a new scene include:
- the party moves to a new location;
- a combat encounter begins or ends;
- combat changes into a substantially different phase;
- the party shifts between combat, exploration, social interaction, discussion,
planning, travel, rest, or downtime;
- a new NPC, faction, threat, or objective becomes central;
- the party completes one immediate goal and begins another;
- a major table-level rules discussion interrupts and materially changes play.
Do not start a new scene merely because:
- the speaker changes;
- a new combat round begins;
- a player asks a brief rules question;
- there is a joke, aside, or short table comment;
- a character takes a routine turn;
- the same encounter continues without a meaningful change in situation.
dnd/scenes boundary policy:
- cover the full provided source document from the first source unit to the last
source unit;
- return sequential scenes with no gaps;
- do not overlap scenes;
- preserve source-unit order;
- use exact source-unit IDs from the ordered source units;
- each scene must have start_unit_id and end_unit_id;
- do not include final chunk IDs or chunk indexes.
For each scene:
- short_title should be brief and factual;
- primary_mode must be Recap, Discussion, Combat, or Narrative;
- main_participants should include only principal characters, NPCs, factions, or
groups involved;
- summary should be factual and compact, usually one to three sentences;
- boundary_note should explain why the scene begins at start_unit_id and ends at
end_unit_id;
- boundary_confidence must be High, Medium, or Low.
Primary mode guidance:
- Use Recap for opening recap, initiative setup, session framing, or immediate
continuation from prior events.
- Use Discussion when the party is primarily discussing options or choosing a
course of action.
- Use Combat when active combat or combat-resolution mechanics dominate.
- Use Narrative for all other non-combat gameplay, including exploration, social
interactions, shopping, preparation, travel, rest, and downtime.
In boundary_caveats, list overall caveats about scene divisions. Include scenes
that could reasonably be split differently, combat phases that were kept
together, gradual transitions, or places where map context would have helped.
Return exactly one JSON object and no explanatory text.

View File

@@ -0,0 +1,83 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "notarius.dnd.scenes",
"type": "object",
"additionalProperties": false,
"required": [
"scenes",
"boundary_caveats"
],
"properties": {
"scenes": {
"type": "array",
"minItems": 1,
"items": {
"type": "object",
"additionalProperties": false,
"required": [
"start_unit_id",
"end_unit_id",
"short_title",
"primary_mode",
"main_participants",
"summary",
"boundary_note",
"boundary_confidence"
],
"properties": {
"start_unit_id": {
"type": "string",
"minLength": 1
},
"end_unit_id": {
"type": "string",
"minLength": 1
},
"short_title": {
"type": "string",
"minLength": 1
},
"primary_mode": {
"type": "string",
"enum": [
"Recap",
"Discussion",
"Combat",
"Narrative"
]
},
"main_participants": {
"type": "array",
"minItems": 1,
"items": {
"type": "string",
"minLength": 1
}
},
"summary": {
"type": "string",
"minLength": 1
},
"boundary_note": {
"type": "string",
"minLength": 1
},
"boundary_confidence": {
"type": "string",
"enum": [
"High",
"Medium",
"Low"
]
}
}
}
},
"boundary_caveats": {
"type": "array",
"items": {
"type": "string"
}
}
}
}

View File

@@ -0,0 +1,303 @@
package scenes
import (
"context"
"fmt"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
const Key = "dnd/scenes"
var requiredCapabilities = []string{
"source.transcript",
}
var providedCapabilities = []string{
"chunks",
"chunks.scenes",
}
var _ contracts.Chunker = (*Chunker)(nil)
var _ contracts.ManifestMetadataProvider = (*Chunker)(nil)
type Chunker struct{}
func New() *Chunker {
return &Chunker{}
}
func (c *Chunker) Key() string {
return Key
}
func (c *Chunker) ManifestMetadata() map[string]any {
promptMetadata := scenesPromptBundle.Metadata()
metadata := map[string]any{
"prompt_id": PromptID,
"prompt_version": promptMetadata.PromptVersion,
"prompt_sha256": promptMetadata.SHA256,
"response_schema_key": string(ResponseSchemaKey),
"response_schema_id": ResponseSchemaID,
"response_schema_name": ResponseSchemaName,
}
if schema, err := loadResponseSchema(); err == nil {
metadata["response_schema_version"] = schema.Version
metadata["response_schema_sha256"] = schema.SHA256
}
return metadata
}
func (c *Chunker) Chunk(ctx context.Context, req contracts.ChunkRequest) (contracts.ChunkResult, error) {
if c == nil {
return contracts.ChunkResult{}, chunkerErrorf("chunker must not be nil")
}
if ctx == nil {
return contracts.ChunkResult{}, chunkerErrorf("context must not be nil")
}
if err := ctx.Err(); err != nil {
return contracts.ChunkResult{}, chunkerErrorf("context error before chunking: %w", err)
}
if req.Source == nil {
return contracts.ChunkResult{}, chunkerErrorf("source must not be nil")
}
if len(req.Source.Units) == 0 {
return contracts.ChunkResult{}, chunkerErrorf("source units must not be empty")
}
if err := source.ValidateDocument(req.Source); err != nil {
return contracts.ChunkResult{}, chunkerErrorf("validate source document: %w", err)
}
if req.LLMClient == nil {
return contracts.ChunkResult{}, chunkerErrorf("LLM client must not be nil")
}
if len(req.Options) > 0 {
return contracts.ChunkResult{}, chunkerErrorf("options are not supported")
}
system, user, _, err := renderPrompt(req)
if err != nil {
return contracts.ChunkResult{}, chunkerErrorf("render prompt: %w", err)
}
schema, err := loadResponseSchema()
if err != nil {
return contracts.ChunkResult{}, chunkerErrorf("load response schema %q: %w", ResponseSchemaKey, err)
}
var response chunkResponse
if _, err := req.LLMClient.CompleteStructured(ctx, contracts.StructuredCompletionRequest{
StageName: Key,
Messages: []contracts.LLMMessage{
{Role: "system", Content: system},
{Role: "user", Content: user},
},
ResponseSchemaName: schema.Name,
ResponseSchema: schema.JSONSchema,
}, &response); err != nil {
return contracts.ChunkResult{}, chunkerErrorf("complete structured output: %w", err)
}
chunks, err := chunksFromResponse(req.Source, response)
if err != nil {
return contracts.ChunkResult{}, chunkerErrorf("malformed structured output: %w", err)
}
return contracts.ChunkResult{
Chunks: chunks,
Warnings: warningsFromCaveats(response.BoundaryCaveats),
}, nil
}
func ModuleSpec() pipeline.ModuleSpec {
return pipeline.ModuleSpec{
Key: Key,
Stage: pipeline.StageChunk,
Requires: append([]string(nil), requiredCapabilities...),
Provides: append([]string(nil), providedCapabilities...),
}
}
func Register(registry *pipeline.ChunkerRegistry) error {
return registry.RegisterWithSpec(ModuleSpec(), func() (contracts.Chunker, error) {
return New(), nil
})
}
func chunksFromResponse(doc *source.SourceDocument, response chunkResponse) ([]contracts.SourceChunk, error) {
if response.Scenes == nil {
return nil, fmt.Errorf("scenes must be present")
}
if len(response.Scenes) == 0 {
return nil, fmt.Errorf("scenes must not be empty")
}
unitIndexes := make(map[string]int, len(doc.Units))
for i, unit := range doc.Units {
unitIndexes[unit.ID] = i
}
chunks := make([]contracts.SourceChunk, 0, len(response.Scenes))
previousEnd := -1
for i, scene := range response.Scenes {
normalized, err := normalizeScene(i, scene)
if err != nil {
return nil, err
}
startIndex, ok := unitIndexes[normalized.StartUnitID]
if !ok {
return nil, fmt.Errorf("scene[%d] start_unit_id %q was not found", i, normalized.StartUnitID)
}
endIndex, ok := unitIndexes[normalized.EndUnitID]
if !ok {
return nil, fmt.Errorf("scene[%d] end_unit_id %q was not found", i, normalized.EndUnitID)
}
if startIndex > endIndex {
return nil, fmt.Errorf("scene[%d] start_unit_id %q appears after end_unit_id %q", i, normalized.StartUnitID, normalized.EndUnitID)
}
if i == 0 && startIndex != 0 {
return nil, fmt.Errorf("first scene must start at first source unit %q", doc.Units[0].ID)
}
if i > 0 {
if startIndex <= previousEnd {
return nil, fmt.Errorf("scene[%d] overlaps previous scene", i)
}
if startIndex > previousEnd+1 {
return nil, fmt.Errorf("scene[%d] leaves a gap after previous scene", i)
}
}
previousEnd = endIndex
units := cloneUnits(doc.Units[startIndex : endIndex+1])
chunks = append(chunks, contracts.SourceChunk{
ID: fmt.Sprintf("scene-%06d", i+1),
SourceID: doc.ID,
Index: i,
Units: units,
Metadata: map[string]any{
"scene_title": normalized.ShortTitle,
"primary_mode": normalized.PrimaryMode,
"main_participants": append([]string(nil), normalized.MainParticipants...),
"summary": normalized.Summary,
"boundary_note": normalized.BoundaryNote,
"boundary_confidence": normalized.BoundaryConfidence,
"start_unit_id": normalized.StartUnitID,
"end_unit_id": normalized.EndUnitID,
"unit_count": len(units),
},
})
}
if previousEnd != len(doc.Units)-1 {
return nil, fmt.Errorf("final scene must end at final source unit %q", doc.Units[len(doc.Units)-1].ID)
}
return chunks, nil
}
func normalizeScene(index int, scene sceneResponse) (sceneResponse, error) {
out := sceneResponse{
StartUnitID: strings.TrimSpace(scene.StartUnitID),
EndUnitID: strings.TrimSpace(scene.EndUnitID),
ShortTitle: strings.TrimSpace(scene.ShortTitle),
PrimaryMode: strings.TrimSpace(scene.PrimaryMode),
Summary: strings.TrimSpace(scene.Summary),
BoundaryNote: strings.TrimSpace(scene.BoundaryNote),
BoundaryConfidence: strings.TrimSpace(scene.BoundaryConfidence),
}
required := map[string]string{
"start_unit_id": out.StartUnitID,
"end_unit_id": out.EndUnitID,
"short_title": out.ShortTitle,
"primary_mode": out.PrimaryMode,
"summary": out.Summary,
"boundary_note": out.BoundaryNote,
"boundary_confidence": out.BoundaryConfidence,
}
for field, value := range required {
if value == "" {
return sceneResponse{}, fmt.Errorf("scene[%d] %s must not be empty", index, field)
}
}
if !validPrimaryMode(out.PrimaryMode) {
return sceneResponse{}, fmt.Errorf("scene[%d] primary_mode %q is not supported", index, out.PrimaryMode)
}
if !validBoundaryConfidence(out.BoundaryConfidence) {
return sceneResponse{}, fmt.Errorf("scene[%d] boundary_confidence %q is not supported", index, out.BoundaryConfidence)
}
if len(scene.MainParticipants) == 0 {
return sceneResponse{}, fmt.Errorf("scene[%d] main_participants must not be empty", index)
}
out.MainParticipants = make([]string, 0, len(scene.MainParticipants))
for participantIndex, participant := range scene.MainParticipants {
trimmed := strings.TrimSpace(participant)
if trimmed == "" {
return sceneResponse{}, fmt.Errorf("scene[%d] main_participants[%d] must not be empty", index, participantIndex)
}
out.MainParticipants = append(out.MainParticipants, trimmed)
}
return out, nil
}
func validPrimaryMode(value string) bool {
switch value {
case "Recap", "Discussion", "Combat", "Narrative":
return true
default:
return false
}
}
func validBoundaryConfidence(value string) bool {
switch value {
case "High", "Medium", "Low":
return true
default:
return false
}
}
func warningsFromCaveats(caveats []string) []contracts.Warning {
if len(caveats) == 0 {
return nil
}
warnings := make([]contracts.Warning, 0, len(caveats))
for _, caveat := range caveats {
warnings = append(warnings, contracts.Warning{
Scope: Key,
ReasonCode: "scene_boundary_caveat",
Message: caveat,
})
}
return warnings
}
func cloneUnits(units []source.SourceUnit) []source.SourceUnit {
out := make([]source.SourceUnit, 0, len(units))
for _, unit := range units {
out = append(out, source.SourceUnit{
ID: unit.ID,
Kind: unit.Kind,
Text: unit.Text,
Metadata: cloneMetadata(unit.Metadata),
})
}
return out
}
func cloneMetadata(metadata map[string]any) map[string]any {
if len(metadata) == 0 {
return nil
}
out := make(map[string]any, len(metadata))
for key, value := range metadata {
out[key] = value
}
return out
}
func chunkerErrorf(format string, args ...any) error {
return fmt.Errorf("dnd scenes chunker: "+format, args...)
}

View File

@@ -0,0 +1,465 @@
package scenes
import (
"bytes"
"context"
"encoding/json"
"errors"
"reflect"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/pipeline"
)
func TestNewModuleSpecAndRegister(t *testing.T) {
chunker := New()
if chunker == nil {
t.Fatal("New() = nil, want chunker")
}
if chunker.Key() != Key {
t.Fatalf("Key() = %q, want %q", chunker.Key(), Key)
}
want := pipeline.ModuleSpec{
Key: Key,
Stage: pipeline.StageChunk,
Requires: []string{"source.transcript"},
Provides: []string{"chunks", "chunks.scenes"},
}
if got := ModuleSpec(); !reflect.DeepEqual(got, want) {
t.Fatalf("ModuleSpec() = %#v, want %#v", got, want)
}
got := ModuleSpec()
got.Requires[0] = "changed"
got.Provides[0] = "changed"
if again := ModuleSpec(); !reflect.DeepEqual(again, want) {
t.Fatalf("ModuleSpec() after caller mutation = %#v, want %#v", again, want)
}
registry := pipeline.NewChunkerRegistry()
if err := Register(registry); err != nil {
t.Fatalf("Register() error = %v, want nil", err)
}
registered, ok := registry.Spec(Key)
if !ok {
t.Fatalf("Spec(%q) ok = false, want true", Key)
}
if !reflect.DeepEqual(registered, want) {
t.Fatalf("registered spec = %#v, want %#v", registered, want)
}
built, err := registry.Build(Key)
if err != nil {
t.Fatalf("Build(%q) error = %v, want nil", Key, err)
}
if built.Key() != Key {
t.Fatalf("built Key() = %q, want %q", built.Key(), Key)
}
}
func TestRegisterNilRegistryReturnsError(t *testing.T) {
err := Register(nil)
if err == nil {
t.Fatal("Register(nil) error = nil, want error")
}
if !strings.Contains(err.Error(), "chunker registry") {
t.Fatalf("Register(nil) error = %q, want registry context", err.Error())
}
}
func TestChunkReturnsSceneChunksFromStructuredOutput(t *testing.T) {
client := &fakeScenesLLMClient{
response: chunkResponse{
Scenes: []sceneResponse{
{
StartUnitID: "seg-001",
EndUnitID: "seg-002",
ShortTitle: " Goblin parley ",
PrimaryMode: "Discussion",
MainParticipants: []string{" Aria ", "Goblin scout"},
Summary: " The party negotiates with a scout. ",
BoundaryNote: " The scene covers the discussion before fighting starts. ",
BoundaryConfidence: "High",
},
{
StartUnitID: "seg-003",
EndUnitID: "seg-004",
ShortTitle: "Ambush at the gate",
PrimaryMode: "Combat",
MainParticipants: []string{"Aria", "Goblin ambushers"},
Summary: "The goblins attack at the gate.",
BoundaryNote: "Combat begins and resolves the immediate threat.",
BoundaryConfidence: "Medium",
},
},
BoundaryCaveats: []string{"The transition into combat is gradual."},
},
}
result, err := New().Chunk(context.Background(), chunkRequestWithClient(client))
if err != nil {
t.Fatalf("Chunk() error = %v, want nil", err)
}
if len(client.requests) != 1 {
t.Fatalf("LLM calls = %d, want 1", len(client.requests))
}
req := client.requests[0]
if req.StageName != Key {
t.Fatalf("StageName = %q, want %q", req.StageName, Key)
}
schema, err := loadResponseSchema()
if err != nil {
t.Fatalf("loadResponseSchema() error = %v, want nil", err)
}
if req.ResponseSchemaName != schema.Name {
t.Fatalf("ResponseSchemaName = %q, want %q", req.ResponseSchemaName, schema.Name)
}
if !bytes.Equal(req.ResponseSchema, schema.JSONSchema) {
t.Fatal("ResponseSchema does not match D&D scenes schema")
}
if len(req.Messages) != 2 || req.Messages[0].Role != "system" || req.Messages[1].Role != "user" {
t.Fatalf("Messages = %#v, want system then user", req.Messages)
}
for _, want := range []string{"session-alpha", "seg-001", "seg-004", "start_unit_id", "boundary_confidence"} {
if !strings.Contains(req.Messages[1].Content, want) {
t.Fatalf("user message = %q, want substring %q", req.Messages[1].Content, want)
}
}
if got := chunkIDs(result.Chunks); !reflect.DeepEqual(got, []string{"scene-000001", "scene-000002"}) {
t.Fatalf("chunk IDs = %#v, want deterministic scene IDs", got)
}
gotUnits := [][]string{unitIDs(result.Chunks[0].Units), unitIDs(result.Chunks[1].Units)}
wantUnits := [][]string{{"seg-001", "seg-002"}, {"seg-003", "seg-004"}}
if !reflect.DeepEqual(gotUnits, wantUnits) {
t.Fatalf("chunk units = %#v, want %#v", gotUnits, wantUnits)
}
first := result.Chunks[0]
if first.SourceID != "session-alpha" || first.Index != 0 {
t.Fatalf("first chunk = %#v, want source and index fields", first)
}
if first.Metadata["scene_title"] != "Goblin parley" ||
first.Metadata["primary_mode"] != "Discussion" ||
first.Metadata["summary"] != "The party negotiates with a scout." ||
first.Metadata["boundary_note"] != "The scene covers the discussion before fighting starts." ||
first.Metadata["boundary_confidence"] != "High" ||
first.Metadata["start_unit_id"] != "seg-001" ||
first.Metadata["end_unit_id"] != "seg-002" ||
first.Metadata["unit_count"] != 2 {
t.Fatalf("first metadata = %#v, want scene metadata", first.Metadata)
}
if got, ok := first.Metadata["main_participants"].([]string); !ok || !reflect.DeepEqual(got, []string{"Aria", "Goblin scout"}) {
t.Fatalf("main_participants = %#v, want trimmed participant slice", first.Metadata["main_participants"])
}
if got := result.Warnings; len(got) != 1 ||
got[0].Scope != Key ||
got[0].ReasonCode != "scene_boundary_caveat" ||
got[0].Message != "The transition into combat is gradual." {
t.Fatalf("Warnings = %#v, want boundary caveat warning", got)
}
}
func TestChunkDefensivelyCopiesSourceUnitsAndMetadata(t *testing.T) {
doc := sceneSourceDocument()
client := &fakeScenesLLMClient{response: validSceneResponse()}
result, err := New().Chunk(context.Background(), contracts.ChunkRequest{
Source: doc,
LLMClient: client,
})
if err != nil {
t.Fatalf("Chunk() error = %v, want nil", err)
}
doc.Units[0].ID = "mutated"
doc.Units[0].Metadata["speaker"] = "mutated"
client.response.Scenes[0].MainParticipants[0] = "mutated"
if result.Chunks[0].Units[0].ID != "seg-001" {
t.Fatalf("chunk unit ID changed after source mutation: %#v", result.Chunks[0].Units[0])
}
if result.Chunks[0].Units[0].Metadata["speaker"] != "Alice" {
t.Fatalf("chunk unit metadata changed after source mutation: %#v", result.Chunks[0].Units[0].Metadata)
}
participants, ok := result.Chunks[0].Metadata["main_participants"].([]string)
if !ok || participants[0] != "Aria" {
t.Fatalf("participants = %#v, want defensive copy", result.Chunks[0].Metadata["main_participants"])
}
}
func TestChunkerManifestMetadataIncludesPromptAndSchemaProvenance(t *testing.T) {
metadata := New().ManifestMetadata()
tests := map[string]string{
"prompt_id": PromptID,
"prompt_version": ResponseSchemaVersion,
"response_schema_key": string(ResponseSchemaKey),
"response_schema_id": ResponseSchemaID,
"response_schema_name": ResponseSchemaName,
"response_schema_version": ResponseSchemaVersion,
}
for key, want := range tests {
if metadata[key] != want {
t.Fatalf("metadata[%q] = %#v, want %q", key, metadata[key], want)
}
}
for _, key := range []string{"prompt_sha256", "response_schema_sha256"} {
value, ok := metadata[key].(string)
if !ok || !strings.HasPrefix(value, "sha256:") {
t.Fatalf("metadata[%q] = %#v, want sha256 value", key, metadata[key])
}
}
for _, forbidden := range []string{"prompt", "schema", "source", "text"} {
if _, ok := metadata[forbidden]; ok {
t.Fatalf("metadata includes raw %q field: %#v", forbidden, metadata)
}
}
}
func TestChunkRejectsInvalidRequests(t *testing.T) {
validClient := &fakeScenesLLMClient{response: validSceneResponse()}
validReq := chunkRequestWithClient(validClient)
canceledCtx, cancel := context.WithCancel(context.Background())
cancel()
invalidDoc := sceneSourceDocument()
invalidDoc.Units[0].ID = ""
emptyDoc := sceneSourceDocument()
emptyDoc.Units = nil
tests := []struct {
name string
chunker *Chunker
ctx context.Context
req contracts.ChunkRequest
want string
}{
{name: "nil chunker", chunker: nil, ctx: context.Background(), req: validReq, want: "chunker"},
{name: "nil context", chunker: New(), ctx: nil, req: validReq, want: "context"},
{name: "canceled context", chunker: New(), ctx: canceledCtx, req: validReq, want: "context"},
{name: "nil source", chunker: New(), ctx: context.Background(), req: contracts.ChunkRequest{LLMClient: validClient}, want: "source"},
{name: "empty source units", chunker: New(), ctx: context.Background(), req: contracts.ChunkRequest{Source: emptyDoc, LLMClient: validClient}, want: "units"},
{name: "invalid source", chunker: New(), ctx: context.Background(), req: contracts.ChunkRequest{Source: invalidDoc, LLMClient: validClient}, want: "validate source document"},
{name: "nil LLM client", chunker: New(), ctx: context.Background(), req: contracts.ChunkRequest{Source: sceneSourceDocument()}, want: "LLM client"},
{name: "unsupported options", chunker: New(), ctx: context.Background(), req: requestWithOptions(validReq), want: "options"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
_, err := tt.chunker.Chunk(tt.ctx, tt.req)
if err == nil {
t.Fatal("Chunk() error = nil, want error")
}
if !strings.Contains(err.Error(), "dnd scenes") || !strings.Contains(err.Error(), tt.want) {
t.Fatalf("Chunk() error = %q, want module context and %q", err.Error(), tt.want)
}
})
}
}
func TestChunkRejectsMalformedStructuredOutput(t *testing.T) {
tests := []struct {
name string
response chunkResponse
want string
}{
{name: "missing scenes", response: chunkResponse{}, want: "scenes"},
{name: "empty scenes", response: chunkResponse{Scenes: []sceneResponse{}}, want: "scenes"},
{
name: "unknown boundary id",
response: replaceScenes(validSceneResponse(), []sceneResponse{
scene("seg-001", "seg-999"),
}),
want: "was not found",
},
{
name: "out of order boundaries",
response: replaceScenes(validSceneResponse(), []sceneResponse{
scene("seg-003", "seg-002"),
}),
want: "appears after",
},
{
name: "gap",
response: replaceScenes(validSceneResponse(), []sceneResponse{
scene("seg-001", "seg-001"),
scene("seg-003", "seg-004"),
}),
want: "gap",
},
{
name: "overlap",
response: replaceScenes(validSceneResponse(), []sceneResponse{
scene("seg-001", "seg-002"),
scene("seg-002", "seg-004"),
}),
want: "overlap",
},
{
name: "incomplete coverage",
response: replaceScenes(validSceneResponse(), []sceneResponse{
scene("seg-001", "seg-003"),
}),
want: "final scene",
},
{
name: "empty metadata field",
response: replaceScenes(validSceneResponse(), []sceneResponse{
{
StartUnitID: "seg-001",
EndUnitID: "seg-004",
ShortTitle: " ",
PrimaryMode: "Narrative",
MainParticipants: []string{"Aria"},
Summary: "Summary.",
BoundaryNote: "Note.",
BoundaryConfidence: "High",
},
}),
want: "short_title",
},
{
name: "empty participant",
response: replaceScenes(validSceneResponse(), []sceneResponse{
{
StartUnitID: "seg-001",
EndUnitID: "seg-004",
ShortTitle: "Title",
PrimaryMode: "Narrative",
MainParticipants: []string{"Aria", " "},
Summary: "Summary.",
BoundaryNote: "Note.",
BoundaryConfidence: "High",
},
}),
want: "main_participants",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
client := &fakeScenesLLMClient{response: tt.response}
_, err := New().Chunk(context.Background(), chunkRequestWithClient(client))
if err == nil {
t.Fatal("Chunk() error = nil, want error")
}
if !strings.Contains(err.Error(), "dnd scenes") || !strings.Contains(err.Error(), tt.want) {
t.Fatalf("Chunk() error = %q, want module context and %q", err.Error(), tt.want)
}
})
}
}
func TestChunkWrapsLLMClientError(t *testing.T) {
client := &fakeScenesLLMClient{err: errors.New("provider unavailable")}
_, err := New().Chunk(context.Background(), chunkRequestWithClient(client))
if err == nil {
t.Fatal("Chunk() error = nil, want LLM error")
}
if !strings.Contains(err.Error(), "dnd scenes") || !strings.Contains(err.Error(), "provider unavailable") {
t.Fatalf("Chunk() error = %q, want wrapped LLM context", err.Error())
}
}
func chunkRequestWithClient(client contracts.StructuredLLMClient) contracts.ChunkRequest {
return contracts.ChunkRequest{
Source: sceneSourceDocument(),
LLMClient: client,
}
}
func requestWithOptions(req contracts.ChunkRequest) contracts.ChunkRequest {
req.Options = map[string]any{"max_units": 2}
return req
}
func sceneSourceDocument() *source.SourceDocument {
return &source.SourceDocument{
ID: "session-alpha",
Kind: "transcript",
Format: "application/vnd.seriatim.minimal+json",
Digest: "sha256:source",
Units: []source.SourceUnit{
{ID: "seg-001", Kind: "transcript_segment", Text: "Aria asks whether the goblin will parley.", Metadata: map[string]any{"speaker": "Alice"}},
{ID: "seg-002", Kind: "transcript_segment", Text: "The goblin scout describes the gate guards."},
{ID: "seg-003", Kind: "transcript_segment", Text: "The guards rush out with blades drawn."},
{ID: "seg-004", Kind: "transcript_segment", Text: "The party defeats the ambushers."},
},
}
}
func validSceneResponse() chunkResponse {
return chunkResponse{
Scenes: []sceneResponse{
scene("seg-001", "seg-004"),
},
BoundaryCaveats: []string{},
}
}
func replaceScenes(response chunkResponse, scenes []sceneResponse) chunkResponse {
response.Scenes = scenes
return response
}
func scene(startUnitID string, endUnitID string) sceneResponse {
return sceneResponse{
StartUnitID: startUnitID,
EndUnitID: endUnitID,
ShortTitle: "Scene title",
PrimaryMode: "Narrative",
MainParticipants: []string{"Aria"},
Summary: "A compact summary.",
BoundaryNote: "The source units form one coherent scene.",
BoundaryConfidence: "High",
}
}
func chunkIDs(chunks []contracts.SourceChunk) []string {
ids := make([]string, 0, len(chunks))
for _, chunk := range chunks {
ids = append(ids, chunk.ID)
}
return ids
}
func unitIDs(units []source.SourceUnit) []string {
ids := make([]string, 0, len(units))
for _, unit := range units {
ids = append(ids, unit.ID)
}
return ids
}
type fakeScenesLLMClient struct {
response chunkResponse
err error
requests []contracts.StructuredCompletionRequest
}
func (client *fakeScenesLLMClient) CompleteStructured(ctx context.Context, req contracts.StructuredCompletionRequest, out any) (contracts.StructuredCompletionResponse, error) {
client.requests = append(client.requests, contracts.StructuredCompletionRequest{
StageName: req.StageName,
Messages: append([]contracts.LLMMessage(nil), req.Messages...),
Model: req.Model,
ResponseSchemaName: req.ResponseSchemaName,
ResponseSchema: append(json.RawMessage(nil), req.ResponseSchema...),
})
if client.err != nil {
return contracts.StructuredCompletionResponse{}, client.err
}
target, ok := out.(*chunkResponse)
if !ok {
return contracts.StructuredCompletionResponse{}, errors.New("unexpected output target")
}
*target = client.response
content, err := json.Marshal(client.response)
if err != nil {
return contracts.StructuredCompletionResponse{}, err
}
return contracts.StructuredCompletionResponse{Content: content}, nil
}

View File

@@ -0,0 +1,17 @@
package scenes
type chunkResponse struct {
Scenes []sceneResponse `json:"scenes"`
BoundaryCaveats []string `json:"boundary_caveats"`
}
type sceneResponse struct {
StartUnitID string `json:"start_unit_id"`
EndUnitID string `json:"end_unit_id"`
ShortTitle string `json:"short_title"`
PrimaryMode string `json:"primary_mode"`
MainParticipants []string `json:"main_participants"`
Summary string `json:"summary"`
BoundaryNote string `json:"boundary_note"`
BoundaryConfidence string `json:"boundary_confidence"`
}

View File

@@ -0,0 +1,97 @@
package scenes
import (
"fmt"
"strings"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/prompt"
)
type promptData struct {
SourceID string
Units []promptUnit
}
type promptUnit struct {
ID string
Text string
Metadata []promptMetadata
}
type promptMetadata struct {
Key string
Value string
}
var scenesPromptBundle = mustLoadPromptBundle()
func mustLoadPromptBundle() *prompt.Bundle {
bundle, err := prompt.LoadBundle(embeddedAssets, prompt.Definition{
PromptID: PromptID,
Version: ResponseSchemaVersion,
EmbeddedPath: "assets/prompts",
SystemPath: "assets/prompts/system.md",
UserPath: "assets/prompts/user.md",
})
if err != nil {
panic(err)
}
return bundle
}
func buildPromptData(req contracts.ChunkRequest) (promptData, error) {
if req.Source == nil {
return promptData{}, fmt.Errorf("dnd scenes prompt: source must not be nil")
}
data := promptData{
SourceID: req.Source.ID,
Units: make([]promptUnit, 0, len(req.Source.Units)),
}
for _, unit := range req.Source.Units {
data.Units = append(data.Units, promptUnit{
ID: unit.ID,
Text: unit.Text,
Metadata: selectedMetadata(unit),
})
}
return data, nil
}
func renderPrompt(req contracts.ChunkRequest) (system string, user string, metadata prompt.Metadata, err error) {
data, err := buildPromptData(req)
if err != nil {
return "", "", prompt.Metadata{}, err
}
system, user, metadata, err = scenesPromptBundle.RenderUserSystem(data)
if err != nil {
return "", "", prompt.Metadata{}, fmt.Errorf("dnd scenes prompt: %w", err)
}
return system, user, metadata, nil
}
func selectedMetadata(unit source.SourceUnit) []promptMetadata {
if len(unit.Metadata) == 0 {
return nil
}
keys := []string{"speaker", "start", "end"}
metadata := make([]promptMetadata, 0, len(keys))
for _, key := range keys {
value, ok := unit.Metadata[key]
if !ok {
continue
}
rendered := strings.TrimSpace(fmt.Sprint(value))
if rendered == "" {
continue
}
metadata = append(metadata, promptMetadata{
Key: key,
Value: rendered,
})
}
return metadata
}

View File

@@ -0,0 +1,155 @@
package scenes
import (
"encoding/json"
"reflect"
"strings"
"testing"
"gitea.maximumdirect.net/eric/notarius/internal/core/source"
"gitea.maximumdirect.net/eric/notarius/internal/framework/contracts"
"gitea.maximumdirect.net/eric/notarius/internal/framework/prompt"
)
func TestBuildPromptDataFromSourceDocument(t *testing.T) {
req := promptChunkRequest()
data, err := buildPromptData(req)
if err != nil {
t.Fatalf("buildPromptData() error = %v, want nil", err)
}
if data.SourceID != "session-alpha" {
t.Fatalf("SourceID = %q, want session-alpha", data.SourceID)
}
if len(data.Units) != 2 {
t.Fatalf("len(Units) = %d, want 2", len(data.Units))
}
first := data.Units[0]
if first.ID != "seg-001" || first.Text != "Aria and Bram discuss whether to enter the ruins." {
t.Fatalf("first unit = %#v, want source unit data", first)
}
wantMetadata := []promptMetadata{
{Key: "speaker", Value: "Alice"},
{Key: "start", Value: "1.25"},
{Key: "end", Value: "3.5"},
}
if !reflect.DeepEqual(first.Metadata, wantMetadata) {
t.Fatalf("first.Metadata = %#v, want %#v", first.Metadata, wantMetadata)
}
if len(data.Units[1].Metadata) != 0 {
t.Fatalf("second.Metadata = %#v, want no selected metadata", data.Units[1].Metadata)
}
}
func TestBuildPromptDataDoesNotMutateRequest(t *testing.T) {
req := promptChunkRequest()
beforeSource := mustJSON(t, req.Source)
beforeRequest := mustJSON(t, req)
if _, err := buildPromptData(req); err != nil {
t.Fatalf("buildPromptData() error = %v, want nil", err)
}
afterSource := mustJSON(t, req.Source)
afterRequest := mustJSON(t, req)
if beforeSource != afterSource || beforeRequest != afterRequest {
t.Fatalf(
"request mutated:\nsource before: %s\nsource after: %s\nrequest before: %s\nrequest after: %s",
beforeSource,
afterSource,
beforeRequest,
afterRequest,
)
}
}
func TestRenderPromptIncludesSourceUnitsAndMetadata(t *testing.T) {
system, user, metadata, err := renderPrompt(promptChunkRequest())
if err != nil {
t.Fatalf("renderPrompt() error = %v, want nil", err)
}
if !strings.Contains(system, prompt.HardeningText()) {
t.Fatalf("system prompt = %q, want hardening text", system)
}
for _, want := range []string{
"session-alpha",
"seg-001",
"seg-002",
"Aria and Bram discuss whether to enter the ruins.",
"The goblins rush out and initiative begins.",
"speaker: Alice",
"start: 1.25",
"end: 3.5",
"start_unit_id",
"end_unit_id",
"primary_mode",
"boundary_confidence",
"Recap, Discussion, Combat, or Narrative",
"High, Medium, or Low",
"no gaps",
"do not overlap",
} {
if !strings.Contains(user, want) {
t.Fatalf("user prompt = %q, want substring %q", user, want)
}
}
if metadata.PromptID != PromptID {
t.Fatalf("metadata.PromptID = %q, want %q", metadata.PromptID, PromptID)
}
if metadata.PromptVersion != ResponseSchemaVersion {
t.Fatalf("metadata.PromptVersion = %q, want %q", metadata.PromptVersion, ResponseSchemaVersion)
}
if metadata.EmbeddedPath != "assets/prompts" {
t.Fatalf("metadata.EmbeddedPath = %q, want assets/prompts", metadata.EmbeddedPath)
}
}
func TestBuildPromptDataRejectsMissingSourceContext(t *testing.T) {
if _, err := buildPromptData(contracts.ChunkRequest{}); err == nil || !strings.Contains(err.Error(), "source") {
t.Fatalf("buildPromptData() error = %v, want source error", err)
}
}
func promptChunkRequest() contracts.ChunkRequest {
return contracts.ChunkRequest{
Source: promptSourceDocument(),
}
}
func promptSourceDocument() *source.SourceDocument {
return &source.SourceDocument{
ID: "session-alpha",
Kind: "transcript",
Format: "application/vnd.seriatim.minimal+json",
Digest: "sha256:test",
Units: []source.SourceUnit{
{
ID: "seg-001",
Kind: "transcript_segment",
Text: "Aria and Bram discuss whether to enter the ruins.",
Metadata: map[string]any{
"speaker": "Alice",
"start": json.Number("1.25"),
"end": json.Number("3.5"),
"ignored": "not rendered",
},
},
{
ID: "seg-002",
Kind: "transcript_segment",
Text: "The goblins rush out and initiative begins.",
Metadata: map[string]any{"ignored": "not rendered"},
},
},
}
}
func mustJSON(t *testing.T, value any) string {
t.Helper()
encoded, err := json.Marshal(value)
if err != nil {
t.Fatalf("Marshal() error = %v, want nil", err)
}
return string(encoded)
}

View File

@@ -0,0 +1,21 @@
package scenes
import "gitea.maximumdirect.net/eric/notarius/internal/framework/llm"
const (
PromptID = "dnd.scenes"
ResponseSchemaKey = llm.ResponseSchemaKey("dnd_scenes")
ResponseSchemaID = "notarius.dnd.scenes"
ResponseSchemaVersion = "v1"
ResponseSchemaName = "notarius_dnd_scenes_v1"
)
func loadResponseSchema() (llm.ResponseSchema, error) {
return llm.LoadResponseSchema(embeddedAssets, llm.ResponseSchemaDefinition{
Key: ResponseSchemaKey,
ID: ResponseSchemaID,
Version: ResponseSchemaVersion,
Name: ResponseSchemaName,
AssetPath: "assets/schemas/dnd_scenes.v1.json",
})
}

View File

@@ -0,0 +1,138 @@
package scenes
import (
"encoding/json"
"strings"
"testing"
)
func TestLoadResponseSchemaForScenes(t *testing.T) {
schema, err := loadResponseSchema()
if err != nil {
t.Fatalf("loadResponseSchema() error = %v, want nil", err)
}
if schema.Key != ResponseSchemaKey {
t.Fatalf("schema.Key = %q, want %q", schema.Key, ResponseSchemaKey)
}
if schema.ID != ResponseSchemaID {
t.Fatalf("schema.ID = %q, want %q", schema.ID, ResponseSchemaID)
}
if schema.Version != ResponseSchemaVersion {
t.Fatalf("schema.Version = %q, want %q", schema.Version, ResponseSchemaVersion)
}
if schema.Name != ResponseSchemaName {
t.Fatalf("schema.Name = %q, want %q", schema.Name, ResponseSchemaName)
}
if !strings.HasPrefix(schema.SHA256, "sha256:") {
t.Fatalf("schema.SHA256 = %q, want sha256 prefix", schema.SHA256)
}
if !json.Valid(schema.JSONSchema) {
t.Fatalf("schema.JSONSchema is invalid JSON: %s", schema.JSONSchema)
}
}
func TestResponseSchemaShapeUsesSourceUnitBoundaries(t *testing.T) {
schema, err := loadResponseSchema()
if err != nil {
t.Fatalf("loadResponseSchema() error = %v, want nil", err)
}
var decoded map[string]any
if err := json.Unmarshal(schema.JSONSchema, &decoded); err != nil {
t.Fatalf("Unmarshal() error = %v, want nil", err)
}
if decoded["$id"] != ResponseSchemaID {
t.Fatalf("$id = %#v, want %q", decoded["$id"], ResponseSchemaID)
}
if decoded["additionalProperties"] != false {
t.Fatalf("additionalProperties = %#v, want false", decoded["additionalProperties"])
}
properties := decoded["properties"].(map[string]any)
if _, ok := properties["artifact_type"]; ok {
t.Fatal("schema includes artifact_type, want only scene response fields")
}
if _, ok := properties["session_scope"]; ok {
t.Fatal("schema includes session_scope, want no session wrapper")
}
sceneProperties := properties["scenes"].(map[string]any)["items"].(map[string]any)["properties"].(map[string]any)
for _, field := range []string{"scene_id", "start_segment_id", "end_segment_id"} {
if _, ok := sceneProperties[field]; ok {
t.Fatalf("scene schema includes old field %q", field)
}
}
for _, field := range []string{"start_unit_id", "end_unit_id"} {
property := sceneProperties[field].(map[string]any)
if property["type"] != "string" {
t.Fatalf("%s type = %#v, want string", field, property["type"])
}
}
modeEnum := sceneProperties["primary_mode"].(map[string]any)["enum"].([]any)
if !sameStrings(modeEnum, []string{"Recap", "Discussion", "Combat", "Narrative"}) {
t.Fatalf("primary_mode enum = %#v, want Recap/Discussion/Combat/Narrative", modeEnum)
}
confidenceEnum := sceneProperties["boundary_confidence"].(map[string]any)["enum"].([]any)
if !sameStrings(confidenceEnum, []string{"High", "Medium", "Low"}) {
t.Fatalf("boundary_confidence enum = %#v, want High/Medium/Low", confidenceEnum)
}
}
func TestResponseStructRejectsIntegerBoundaries(t *testing.T) {
raw := []byte(`{
"scenes": [
{
"start_unit_id": 1,
"end_unit_id": 3,
"short_title": "Ambush",
"primary_mode": "Combat",
"main_participants": ["Aria"],
"summary": "The party fights.",
"boundary_note": "Combat starts and resolves.",
"boundary_confidence": "High"
}
],
"boundary_caveats": []
}`)
var response chunkResponse
err := json.Unmarshal(raw, &response)
if err == nil {
t.Fatalf("Unmarshal() error = nil, want integer boundary type error: %#v", response)
}
if !strings.Contains(err.Error(), "string") {
t.Fatalf("Unmarshal() error = %v, want string type error", err)
}
}
func TestResponseSchemaJSONIsMutationSafe(t *testing.T) {
first, err := loadResponseSchema()
if err != nil {
t.Fatalf("loadResponseSchema() error = %v, want nil", err)
}
first.JSONSchema[0] = '['
second, err := loadResponseSchema()
if err != nil {
t.Fatalf("loadResponseSchema() error = %v, want nil", err)
}
if !json.Valid(second.JSONSchema) {
t.Fatalf("schema JSON was mutated: %s", second.JSONSchema)
}
if len(second.JSONSchema) > 0 && second.JSONSchema[0] == '[' {
t.Fatalf("schema JSON did not use defensive copy")
}
}
func sameStrings(got []any, want []string) bool {
if len(got) != len(want) {
return false
}
for i := range want {
if got[i] != want[i] {
return false
}
}
return true
}