D&D Module Harmonization And Prompt Reuse

This roadmap records proposed improvements to the spell, NPC, and combat-turn pipeline lanes. Current implemented behavior remains documented in the module internals and LLM runtime internals. The work below is not yet an implemented contract.

The three lanes already share the same overall decomposition: a typed extractor, durable codec, merger, normalizer, shape and provenance validators, domain-specific validation, embedded prompt assets, and central production registration. Spell catalog resolution, NPC identity and registry support, and combat normalization invariants are intentional domain differences. The goal is to remove mechanical drift without hiding those differences behind a broad generic abstraction.

Target State

Prompt Reuse Is A Rendered-Request Invariant

Provider prompt caching depends on the request prefix being byte-for-byte identical. Similar prose, duplicated files, or equivalent structured values are not sufficient. Treat message role, content bytes, ordering, cache-control metadata, input rendering, and any provider-visible separators as part of the cache identity.

The target prompt layout has the longest valid common prefix before any module-specific message:

Put shared static system, extraction-evidence, and in-world-identity policy before dynamic transcript and campaign-reference messages so reuse does not depend on identical transcript content.
Keep the shared transcript and campaign-reference messages in the same roles and order in every D&D extractor.
Keep cache-control declarations identical on corresponding messages.
Define one canonical ordering and serialization for shared named inputs. Continue resolving the deprecated roster input into the canonical party input before prompt rendering.
Keep genuinely universal extraction rules in shared message assets. These rules include using only transcript units as evidence, treating references as disambiguation, returning schema-conforming JSON only, using integer start_unit_id and end_unit_id values, omitting source_id for the mapper to assign, citing all factual claims, and preferring narrow ranges over broad ranges that bridge unrelated conversation.
Use one shared in-world identity message for lanes whose artifacts name actors or participants. It contains only rules that spells, NPCs, and combat turns can follow verbatim; artifact-specific identity, alias, target, and relationship rules remain local.
Keep rules shared by only a subset in subset-specific shared assets. The immediate-declaration-and-resolution boundary and NPC-registry grounding are shared by spells and combat. The NPC extractor does not consume a prior NPC registry.
Place subset-specific and module-specific messages only after the longest useful all-lane prefix. Place schema-specific task instructions last.
Avoid inserting a nominally shared message when its rendered input or wording differs by lane. Factor the common bytes into one message and leave the differences in later messages instead.

Tests inspect the fully rendered Scriptorium request boundary. Source-level message identity alone is not treated as proof of cache identity.

Exact Prompt And Input Identity Is Protected

Focused tests at the narrowest stable boundary expose the fully prepared provider-neutral request.

Render each extractor prompt with the same transcript and campaign references and assert that the intended common message prefix has identical roles, content bytes, ordering, and cache-control metadata.
Assert that spells and combat turns render identical NPC-registry messages for the same bound or unbound registry.
Verify common input material identity, including name, media type, content, digest, origin URI, size, empty-value representation, reference ordering, and roster fallback behavior.
Test both identical and intentionally different chunks and reference sets so the test proves the cache boundary rather than merely snapshotting one request.
Add an explicit assertion for the length of the common prefix. A new module-specific message inserted inside that prefix should require deliberate review.
Test that every prompt fingerprint includes exactly the assets actually rendered by that prompt. The current grouped reference hash helper should be replaced or refined so a module does not fingerprint an unused shared asset, while no used asset is omitted.
Prefer semantic assertions over complete prompt snapshots, except for the common rendered prefix whose exact bytes are the behavior under protection.

Stable Extraction Preparation Is Centralized

Move the three identical chunk-input preparation implementations into a D&D shared helper. The helper should clone the supplied material, fall back to the chunk content, verify byte equality, and fill the canonical name, media type, and size without retaining mutable request data.

Nil context, cancellation, source, chunk, and empty-unit checks remain local so typed result handling and module error context stay explicit. Do not wrap the complete LLM call, response DTO mapping, catalog preparation, registry preparation, or manifest metadata in a generic extractor framework.

Use the shared helper as the single source of the transcript prompt material so identical input requests cannot drift between lanes.

Validator Checkpoint And Composition Policy Is Aligned

Add explicit versioned policy checkpoint fingerprints to the spell shape, source-reference, and source-relatedness validators. Bump a policy version whenever acceptance, rejection, warning, or diagnostic-selection behavior changes.
Shape validators own malformed artifact shape; later validators defer when shape is invalid.
Make all source-reference validators collect bounded diagnostics through the shared D&D diagnostics package rather than mixing first-error and aggregate behavior.
Use artifact-qualified reason codes consistently unless a reason code is intentionally a stable cross-artifact contract.
Add a compact cross-lane validator contract test covering deterministic execution class, strict empty options, registration, policy fingerprinting, prerequisite behavior, and diagnostic bounds.

Source-Evidence Traversal Is Shared

Extract a D&D helper that validates source ranges and returns cited units or text once, in source-document order, without repeating units covered by overlapping ranges. Use it from spell, NPC, and combat relatedness validators.

Keep matching policy artifact-specific:

spell matching may use canonical catalog names and aliases;
NPC matching may use NPC comparison keys and aliases; and
combat matching may apply separate actor and declaration heuristics.

Review Unicode normalization, apostrophe handling, word boundaries, short-name false positives, multiword identities, and overlapping ranges with shared table-driven fixtures. Relatedness remains a warning heuristic and should not be presented as proof that every semantic claim is supported.

Structural Cleanup

Lane Registration

Reduce repetition in the central D&D registrar with focused registration helpers grouped by modules, validators, prompt assets, and default chains. Retain artifact-specific append and deep-clone behavior in the registrar, central ownership of validator ordering, and explicit typed registration. Avoid reflection and heterogeneous erased lane descriptors.

Naming And Package Conventions

Adopt consistent conventions for package aliases, module keys, artifact kinds, prompt IDs, schema IDs, reason codes, policy IDs, metadata fields, and checkpoint fingerprint names. Compatibility-sensitive identifiers should change only through an explicit migration; internal aliases can be harmonized independently.

Schemas And Assets

Keep private LLM response schemas distinct from durable artifact codec schemas. They represent different trust and compatibility boundaries.
Keep source-reference schema definitions package-owned in this work; do not add schema composition or generation machinery solely to deduplicate them.
Prefer one embedded shared asset over synchronized copies whenever content must be identical for prompt caching.
Make prompt fingerprints derive from an explicit prompt asset manifest, or from the prepared prompt definition, so message composition and provenance cannot drift independently.

Prompt factoring must not be accepted solely because cache reuse improves. Retain or restore module-specific wording when evaluation shows a meaningful quality regression. Record cache observations using non-secret request and usage metadata rather than prompt or transcript payloads.

Completion Criteria

This roadmap is complete when:

the three lanes expose the same stable structural conventions while keeping documented domain differences local;
shared provider-visible messages and inputs are produced from one source and verified byte-for-byte at the rendered-request boundary;
the common prompt prefix is deliberate, tested, and as long as extraction quality permits;
every validator policy that affects reusable results participates in checkpoint identity;
source-reference traversal and diagnostics no longer drift between lanes;
registration remains explicit and type-safe; and
focused D&D tests plus repository-wide tests, vetting, and the CLI build pass.

9.3 KiB Raw Blame History