eric/notarius
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add a roadmap to implement a D&D-specific chunk module

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-04 07:48:36 -05:00
parent 11073b613c
commit b95af4f87d
5 changed files with 628 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

104
docs/roadmap/chunk.md Normal file
View File

@@ -0,0 +1,104 @@
# Chunk Module Roadmap
Current Notarius behavior is documented in the canonical README, CLI,
configuration, operations, internal, and integration docs. This roadmap records
future chunk-module behavior only.
## Goal
Chunk modules should be a clear module-author boundary, and LLM-backed chunking
should be a first-class capability.
The immediate target is a D&D-specific scene chunker that divides transcript
source units into coherent scenes before extraction. The broader target is that
any chunk module can be implemented as a black box when it satisfies the
framework chunk contract.
## Target Chunk Contract
The framework chunk contract should support deterministic and LLM-backed
chunkers through the same module interface.
Chunkers should receive runtime dependencies from the runner, including the
structured LLM client when a chunker needs model calls. Chunkers should not
construct provider clients internally.
The framework should validate these result invariants for every chunk module:
- chunk IDs are non-empty and unique within a run;
- each chunk references the input source document ID;
- chunk indexes are deterministic and sequential in returned order;
- each chunk contains at least one source unit;
- each chunk source unit comes from the source document;
- source units within each chunk appear in source-document order.
The framework should not require complete source-unit coverage and should not
forbid overlap between chunks. Individual chunk modules may enforce stricter
policies, such as full coverage or non-overlap, when those policies are part of
the module's own contract.
Chunk metadata should remain flexible and module-owned. Framework code should
preserve chunk metadata and pass it to downstream modules, but it should not
adopt transcript-specific or D&D-specific metadata fields.
## D&D Scene Chunker Target
The D&D scene chunker should live under
`internal/modules/chunk/dnd/scenes` and use the module key `dnd/scenes`.
It should require transcript source capabilities and provide the generic
`chunks` capability plus a scene-specific chunk capability. D&D scene-boundary
prompt logic, response-schema interpretation, and stricter scene policies belong
inside the module.
The module should use the framework structured LLM client for scene-boundary
detection. The model response should describe source-unit boundaries and useful
scene metadata; the Go module should validate the response and convert it into
`contracts.SourceChunk` values.
For `dnd/scenes`, the module-owned policy should be:
- cover the full source document from first source unit to last source unit;
- return sequential, contiguous, non-overlapping scenes;
- use exact source-unit IDs for boundaries;
- fail with actionable errors for malformed model output rather than silently
falling back to a generic chunker.
Scene chunk IDs and indexes should be assigned by the module, not trusted from
model output. Useful scene information should be stored in chunk metadata, such
as title, primary mode, participants, summary, boundary note, and boundary
confidence. Overall boundary caveats should be surfaced as chunker warnings.
## Draft Asset Target
Initial D&D scene chunker prompt and schema drafts exist under
`internal/modules/chunk/dnd/scenes/assets`. They should be revised before the
module is implemented.
The response schema should be versioned and named consistently with existing
module-owned response schemas, such as `dnd_scenes.v1.json`, with a schema key,
schema ID, schema version, and OpenAI-compatible response schema name.
Boundary fields should use source-unit ID strings, not integer segment IDs.
The schema should focus on boundary and metadata decisions rather than final
framework chunk fields. Prompt terminology and schema terminology should match
exactly, including primary mode enum values and boundary field names.
The user prompt should be a Go template that includes source document ID,
chunking scope, ordered source units, and selected metadata such as speaker and
timestamps when available. It may contain D&D-specific scene guidance, but it
should not imply that the framework itself is transcript-specific.
## Documentation Target
Current-behavior docs should be updated only after the corresponding behavior is
implemented.
Internal module-author documentation should eventually define the chunk module
API, including `Chunker`, `ChunkRequest`, `ChunkResult`, `SourceChunk`,
validation invariants, warning semantics, LLM-backed chunker expectations,
module specs, capability guidance, and option parsing expectations.
When `dnd/scenes` becomes production behavior, configuration, CLI, internal
module, and troubleshooting docs should describe the implemented module and its
failure modes.

312
docs/roadmap/implementation.md Normal file
View File

@@ -0,0 +1,312 @@
# Chunk Module Implementation Plan
This plan implements the accepted target state in
[Chunk Module Roadmap](chunk.md). It is written for an LLM coding agent that
will implement each stage in order.
Before beginning any stage, review:
- `docs/policy/architecture.md`
- `docs/policy/development.md`
- `docs/policy/documentation.md`
- `docs/roadmap/chunk.md`
Do not move planned behavior into non-roadmap docs until the corresponding code
is implemented. Do not revert unrelated user changes.
## Stage 1: Framework Chunk Contract
Goal: make LLM-backed chunking first-class and enforce generic chunk result
invariants without adding any D&D-specific framework behavior.
Code changes:
- Add `LLMClient contracts.StructuredLLMClient` to `contracts.ChunkRequest` in
`internal/framework/contracts/contracts.go`.
- Update `internal/framework/pipeline/runner.go` so the runner passes
`input.LLMClient` to the chunker in `contracts.ChunkRequest`.
- Add framework-level chunk result validation after `chunker.Chunk` returns and
before lanes execute.
- Keep validation source-generic. The validator should reject:
- empty chunk ID;
- duplicate chunk ID;
- chunk `SourceID` that does not match the source document ID;
- chunk `Index` that does not match returned order;
- empty chunk units;
- repeated source unit inside one chunk;
- source unit not found in the source document;
- chunk units that do not appear in source-document order.
- The validator must not require complete coverage and must not reject overlap
between different chunks.
- Preserve existing warning behavior: append chunker warnings before returning
chunk errors, as the runner does today.
Documentation changes:
- Update implemented internal docs under `docs/internal/` to define the chunk
module API and validation invariants once the code exists.
- Keep examples and user docs unchanged in this stage unless an existing doc
becomes inaccurate.
Tests:
- Update contract tests for the new `ChunkRequest.LLMClient` field where useful.
- Add focused pipeline runner tests for each invalid chunk result case listed
above.
- Add runner tests proving partial coverage and overlapping chunks remain
accepted.
- Run:
```sh
go test ./internal/framework/contracts ./internal/framework/pipeline
```
Stage completion criteria:
- Existing generic chunking still works.
- A fake chunker can receive the structured LLM client through `ChunkRequest`.
- Framework tests prove the accepted generic chunk invariants.
## Stage 2: D&D Scene Assets And Module Skeleton
Goal: revise the draft D&D scene prompt and schema into module-owned assets and
add load/render plumbing without registering production behavior.
Asset decisions:
- Rename `internal/modules/chunk/dnd/scenes/assets/schemas/scene_map.schema.json`
to `internal/modules/chunk/dnd/scenes/assets/schemas/dnd_scenes.v1.json`.
- Use these schema constants unless a code-local naming conflict requires a
mechanical adjustment:
- prompt ID: `dnd.scenes`
- response schema key: `dnd_scenes`
- response schema ID: `notarius.dnd.scenes`
- response schema version: `v1`
- response schema name: `notarius_dnd_scenes_v1`
- Use this structured response shape:
```json
{
"scenes": [
{
"start_unit_id": "seg-001",
"end_unit_id": "seg-010",
"short_title": "Ambush at the gate",
"primary_mode": "Combat",
"main_participants": ["Aria", "Bandit mage"],
"summary": "The party fights the bandit mage at the gate.",
"boundary_note": "The scene begins when combat starts and ends when the immediate threat is resolved.",
"boundary_confidence": "High"
}
],
"boundary_caveats": []
}
```
- Required top-level fields: `scenes`, `boundary_caveats`.
- Required scene fields: `start_unit_id`, `end_unit_id`, `short_title`,
`primary_mode`, `main_participants`, `summary`, `boundary_note`,
`boundary_confidence`.
- Boundary fields are source-unit ID strings, not integers.
- `primary_mode` enum: `Recap`, `Discussion`, `Combat`, `Narrative`.
- `boundary_confidence` enum: `High`, `Medium`, `Low`.
- Keep `additionalProperties: false` throughout the schema.
- Do not include model-authored final chunk IDs or chunk indexes in the schema.
The Go module assigns deterministic chunk IDs and indexes.
Prompt decisions:
- Keep D&D-specific scene guidance in the D&D scene module.
- Make the user prompt a Go template similar to the spell extractor prompt.
- Include source document ID and ordered source units.
- Include selected source-unit metadata when present: `speaker`, `start`, and
`end`.
- Align prompt terms exactly with schema field names and enum values.
- Keep `dnd/scenes` module policy explicit in the prompt: full coverage,
sequential scenes, no gaps, no overlap, exact source-unit IDs.
Code changes:
- Add `assets.go` with an `embed.FS` for prompts and schemas.
- Add `schema.go` with the constants and a `loadResponseSchema` function using
`llm.LoadResponseSchema`, following the pattern in
`internal/modules/extract/dnd/spells/schema.go`.
- Add prompt rendering code using `framework/prompt.Bundle`, following the
pattern in `internal/modules/extract/dnd/spells/prompt.go`.
- Add internal response structs for the schema shape.
- Do not register the module in `internal/cli/catalog.go` in this stage.
Tests:
- Add tests that the schema loads, is valid JSON, has the expected metadata, and
rejects the old integer-boundary assumption through Go-side type expectations.
- Add prompt rendering tests that source unit IDs and selected metadata appear
in the rendered user prompt.
- Run:
```sh
go test ./internal/modules/chunk/dnd/scenes
```
Stage completion criteria:
- The scene schema and prompts are loadable embedded assets.
- The prompt/schema terminology is internally consistent.
- No production catalog behavior changes yet.
## Stage 3: D&D Scene Chunker Implementation
Goal: implement `dnd/scenes` as a contract-compliant chunk module with strict
module-owned validation.
Module decisions:
- Package path: `internal/modules/chunk/dnd/scenes`.
- Package name: `scenes`.
- Module key: `dnd/scenes`.
- `ModuleSpec`:
- `Stage`: `pipeline.StageChunk`
- `Requires`: `source.transcript`
- `Provides`: `chunks`, `chunks.scenes`
- Constructor: `New() *Chunker`.
- Registration function: `Register(registry *pipeline.ChunkerRegistry) error`.
- No module options initially. Reject non-empty options with an actionable
module-prefixed error unless a clear option is implemented in the same stage.
Chunking behavior:
- Validate `context.Context`, source document, non-empty source units, and
non-nil `LLMClient`.
- Render the scene prompt over the full source document.
- Call `LLMClient.CompleteStructured` with:
- `StageName`: `dnd/scenes`
- response schema name and schema JSON from the module schema loader.
- Validate the decoded response before producing chunks:
- `scenes` must be present and non-empty;
- every boundary ID must exist in the source document;
- each scene start must be at or before its end;
- the first scene starts at the first source unit;
- the final scene ends at the final source unit;
- scenes are contiguous in source order;
- scenes do not overlap;
- required metadata fields are non-empty after trimming;
- `main_participants` entries are trimmed and empty entries rejected.
- Assign deterministic chunk fields:
- `ID`: `scene-000001`, `scene-000002`, and so on;
- `SourceID`: source document ID;
- `Index`: zero-based returned order;
- `Units`: defensive copies of the source units in the scene range.
- Store per-scene metadata on each chunk:
- `scene_title`
- `primary_mode`
- `main_participants`
- `summary`
- `boundary_note`
- `boundary_confidence`
- `start_unit_id`
- `end_unit_id`
- `unit_count`
- Convert each `boundary_caveats` entry into a `contracts.Warning` with:
- `Scope`: `dnd/scenes`
- `ReasonCode`: `scene_boundary_caveat`
- `Message`: the caveat text.
- Fail explicitly for malformed model output. Do not fall back to `generic`.
- Implement `contracts.ManifestMetadataProvider` and include prompt and
response-schema provenance without raw prompts, raw schemas, source text, or
secrets.
Tests:
- Registration and `ModuleSpec`.
- Successful chunking from a fake LLM response.
- Prompt request uses the expected schema name and schema JSON.
- Caveats become warnings.
- Defensive copy behavior for source units and metadata.
- Errors for nil context, nil source, invalid source, nil LLM client, empty
model scenes, unknown boundary ID, out-of-order boundaries, gaps, overlap,
incomplete coverage, empty metadata fields, and non-empty unsupported options.
- Manifest metadata contains prompt/schema provenance.
- Run:
```sh
go test ./internal/modules/chunk/dnd/scenes
go test ./internal/framework/pipeline
```
Stage completion criteria:
- `dnd/scenes` works in focused tests with fake LLM clients.
- It is still not production-registered unless Stage 4 is completed.
## Stage 4: Production Registration And Implemented Docs
Goal: make `dnd/scenes` available in production configuration and document only
the behavior that now exists.
Code changes:
- Register `dnd/scenes` in `internal/cli/catalog.go`.
- Add or update catalog/default module tests so the production catalog exposes
the new chunk module.
- Add CLI/config validation tests proving a pipeline can select
`chunk: dnd/scenes`.
- Do not change the existing maintained example config unless the related CLI
fixture tests are updated to keep it loadable and useful.
Documentation changes:
- Update `docs/config.md` implemented production module tables and chunk module
notes.
- Update `docs/cli.md` implemented production module list.
- Update `docs/internal/modules.md` with `dnd/scenes` behavior, capabilities,
metadata, and failure policy.
- Update or add internal chunk-module documentation if Stage 1 did not already
create a clear API reference.
- Update `docs/troubleshooting.md` for common scene chunker failures:
malformed model output, invalid boundaries, incomplete coverage, and provider
failures during chunking.
- Keep roadmap docs for any deferred options or future prompt tuning.
Tests:
```sh
go test ./internal/cli
go test ./internal/core/config
go test ./internal/framework/pipeline
go test ./internal/modules/chunk/dnd/scenes
```
Stage completion criteria:
- Config resolution can bind `dnd/scenes`.
- User and internal docs describe the implemented module accurately.
- Existing examples and CLI docs remain truthful.
## Stage 5: Full Verification
Goal: verify the complete feature across contracts, production wiring, docs, and
the command entry point.
Run:
```sh
go test ./...
go vet ./...
go build ./cmd/notarius
```
Inspect diagnostics-sensitive output manually in tests or fixtures where
relevant:
- no raw prompts, source text, provider payloads, API keys, or secrets in
manifest metadata;
- errors name the module and operation;
- warnings are preserved in `RunOutput.Warnings`;
- run manifests record the `dnd/scenes` chunker when selected.
Stage completion criteria:
- Full validation commands pass.
- The feature is documented as implemented only where code supports it.
- `docs/roadmap/chunk.md` retains target-state context and does not duplicate
current-behavior reference material.

5
internal/modules/chunk/dnd/scenes/assets/prompts/system.md Normal file
View File

@@ -0,0 +1,5 @@
You are an expert in Dungeons & Dragons gameplay, writing, and analysis. You will be provided with an audio transcript of a D&D gameplay session. Then, the user will ask you to perform a task based upon the transcript.
Note that the transcript may contain transcription errors, repeated lines, incomplete sentences, and occasional misheard proper nouns. You may assume that speaker identity is accurate.
Pay careful attention to the task assigned to you by the user, and follow the provided instructions precisely.

85
internal/modules/chunk/dnd/scenes/assets/prompts/user.md Normal file
View File

@@ -0,0 +1,85 @@
TASK: Generate a structured JSON scene map for the provided D&D session transcript.
This is an intermediate artifact for an automated analysis pipeline. Your task is to divide the transcript into sequential, contiguous, non-overlapping scenes.
Return only valid JSON matching the provided JSON Schema.
Next, you will be provided with detailed instructions with respect to preparation of the strucured JSON scene map.
INSTRUCTIONS: Your task is to divide the provided transcript into sequential, contiguous, non-overlapping scenes.
A scene is a coherent unit of play. A new scene should usually begin when there is a meaningful change in location, objective, threat, activity, encounter, or mode of play.
Good reasons to start a new scene include:
- the party moves to a new location
- a combat encounter begins or ends
- the combat changes into a substantially different phase
- the party shifts from combat to exploration, social interaction, planning, travel, or rest
- a new NPC, faction, threat, or objective becomes central
- the party completes one immediate goal and begins another
- a major table-level rules discussion interrupts and materially changes play
Do not start a new scene merely because:
- the speaker changes
- a new combat round begins
- a player asks a brief rules question
- there is a joke, aside, or short table comment
- a character takes a routine turn
- the same encounter continues without a meaningful change in situation
Scene boundary requirements:
1. Cover the transcript from the first relevant segment to the last relevant segment.
2. Do not leave gaps between scenes.
3. Do not overlap scenes.
4. Preserve transcript order.
5. Use exact segment IDs from the transcript.
6. Each scene must have a start_segment_id and end_segment_id.
7. Scene IDs must be sequential: 1, 2, 3, and so on.
8. Prefer coherent scenes over excessive fragmentation.
9. Do not merge unrelated scenes just to keep the scene count low.
10. If a boundary is uncertain, choose the most practical boundary and explain the uncertainty in boundary_note.
For each scene:
- short_title should be brief and factual.
- primary_mode must use one of the allowed schema values.
- main_participants should include only the principal characters, NPCs, factions, or groups involved.
- summary should be factual and compact, usually one to three sentences.
- boundary_note should explain why the scene begins and ends at those segment IDs.
- boundary_confidence should be High, Medium, or Low.
Primary mode guidance:
- Use Recap for opening recap, initiative setup, session framing, or immediate continuation from prior events.
- Use Discussion when the party is primarily discussing options or choosing a course of action.
- Use Combat when active combat or combat-resolution mechanics dominate.
- Use Narrative for all other types of non-combat gameplay, including exploration, social interactions, shopping, preparing for battle, downtime, and so on.
Session scope requirements:
- In session_scope.source_description, briefly describe the transcript reviewed.
- In session_scope.first_segment_id, provide the first segment ID covered by the scene map.
- In session_scope.last_segment_id, provide the last segment ID covered by the scene map.
Boundary caveats:
In boundary_caveats, list any overall caveats about the scene divisions. Include scenes that could reasonably be split differently, combat phases that were kept together, gradual transitions, or places where map context would have helped.
Important restrictions:
- Do not invent missing events.
- Do not include comments.
- Do not include explanatory text outside the JSON object.
- Do not wrap the JSON in a code block.
- Return exactly one JSON object.
Before finalizing, check:
- The first scene starts at the first covered transcript segment.
- The final scene ends at the last covered transcript segment.
- Scene ranges are sequential and non-overlapping.
- Every scene has a start_segment_id and end_segment_id.
- Every summary is factual and compact.
- Boundary uncertainty is noted rather than hidden.
- The output is valid JSON and conforms to the schema.

122
internal/modules/chunk/dnd/scenes/assets/schemas/scene_map.schema.json Normal file
View File

@@ -0,0 +1,122 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "D&D Session Scene Map",
"description": "Structured scene segmentation for a D&D session transcript.",
"type": "object",
"additionalProperties": false,
"required": [
"artifact_type",
"session_scope",
"scenes",
"boundary_caveats"
],
"properties": {
"artifact_type": {
"type": "string",
"const": "current_session_scene_map"
},
"session_scope": {
"type": "object",
"additionalProperties": false,
"required": [
"source_description",
"first_segment_id",
"last_segment_id"
],
"properties": {
"source_description": {
"type": "string",
"description": "Brief description of the transcript or source material reviewed."
},
"first_segment_id": {
"type": "integer",
"description": "The first transcript segment covered by the scene map."
},
"last_segment_id": {
"type": "integer",
"description": "The final transcript segment covered by the scene map."
}
}
},
"scenes": {
"type": "array",
"description": "Sequential, contiguous, non-overlapping scene divisions covering the transcript.",
"minItems": 1,
"items": {
"type": "object",
"additionalProperties": false,
"required": [
"scene_id",
"start_segment_id",
"end_segment_id",
"short_title",
"primary_mode",
"main_participants",
"summary",
"boundary_note",
"boundary_confidence"
],
"properties": {
"scene_id": {
"type": "string",
"description": "Stable sequential scene identifier.",
"pattern": "[0-9]$"
},
"start_segment_id": {
"type": "integer",
"description": "Exact segment ID where this scene begins."
},
"end_segment_id": {
"type": "integer",
"description": "Exact segment ID where this scene ends."
},
"short_title": {
"type": "string",
"description": "Compact descriptive title for the scene."
},
"primary_mode": {
"type": "string",
"enum": [
"Recap",
"Planning",
"Combat",
"Narrative"
],
"description": "The dominant mode of play in the scene."
},
"main_participants": {
"type": "array",
"description": "Principal player characters, NPCs, factions, or groups involved in the scene.",
"items": {
"type": "string"
}
},
"summary": {
"type": "string",
"description": "One- to three-sentence factual summary of what happened in the scene."
},
"boundary_note": {
"type": "string",
"description": "Brief explanation of why the scene starts and ends at these segment boundaries."
},
"boundary_confidence": {
"type": "string",
"enum": [
"High",
"Medium",
"Low"
],
"description": "Confidence that the chosen scene boundaries are natural and useful."
}
}
}
},
"boundary_caveats": {
"type": "array",
"description": "Overall caveats about scene segmentation, ambiguous transitions, map-dependent context, or alternative reasonable divisions.",
"items": {
"type": "string"
}
}
}
}