Use shared assets instead of prompt prefix change detectors

2026-07-21 13:59:28 +00:00
parent 3d70920f3d
commit e01b8d1b6d
2 changed files with 49 additions and 74 deletions
--- a/docs/roadmap/dnd.md
+++ b/docs/roadmap/dnd.md
@@ -56,34 +56,28 @@ module-specific message:
  wording differs by lane. Factor the common bytes into one message and leave
  the differences in later messages instead.

-Tests inspect the fully rendered Scriptorium request boundary. Source-level
-message identity alone is not treated as proof of cache identity.
+Shared messages are embedded once and referenced by every applicable prompt.
+This makes reuse of the canonical asset the easiest way to preserve cache
+identity and avoids parallel module-specific copies that can drift.

-### Exact Prompt And Input Identity Is Protected
+### Shared Prompt Ownership And Input Behavior Are Protected

-Focused tests at the narrowest stable boundary expose the fully prepared
-provider-neutral request.
+Focused tests protect shared asset composition and input behavior without
+snapshotting the length or exact boundary of the common message prefix.

- Render each extractor prompt with the same transcript and campaign
-  references and assert that the intended common message prefix has identical
-  roles, content bytes, ordering, and cache-control metadata.
- Assert that spells and combat turns render identical NPC-registry messages
-  for the same bound or unbound registry.
+- Verify that each applicable prompt mounts and renders the canonical shared
+  assets rather than module-specific copies.
 - Verify common input material identity, including name, media type, content,
  digest, origin URI, size, empty-value representation, reference ordering,
  and `roster` fallback behavior.
- Test both identical and intentionally different chunks and reference sets so
-  the test proves the cache boundary rather than merely snapshotting one
-  request.
- Add an explicit assertion for the length of the common prefix. A new
-  module-specific message inserted inside that prefix should require deliberate
-  review.
 - Test that every prompt fingerprint includes exactly the assets actually
  rendered by that prompt. The current grouped reference hash helper should be
  replaced or refined so a module does not fingerprint an unused shared asset,
  while no used asset is omitted.
- Prefer semantic assertions over complete prompt snapshots, except for the
-  common rendered prefix whose exact bytes are the behavior under protection.
+- Document the expected shared message ordering and cache-control policy so
+  later changes are reviewed against the caching intent.
+- Prefer behavioral and ownership assertions over message-count, prefix-length,
+  or full-prompt snapshots.

 ### Stable Extraction Preparation Is Centralized

@@ -175,10 +169,10 @@ This roadmap is complete when:

 - the three lanes expose the same stable structural conventions while keeping
  documented domain differences local;
- shared provider-visible messages and inputs are produced from one source and
-  verified byte-for-byte at the rendered-request boundary;
- the common prompt prefix is deliberate, tested, and as long as extraction
-  quality permits;
+- shared provider-visible messages are produced from canonical shared assets,
+  and shared inputs have one deterministic implementation;
+- the common prompt ordering is deliberate, documented, and as cache-friendly
+  as extraction quality permits;
 - every validator policy that affects reusable results participates in
  checkpoint identity;
 - source-reference traversal and diagnostics no longer drift between lanes;
--- a/docs/roadmap/implementation.md
+++ b/docs/roadmap/implementation.md
@@ -33,17 +33,17 @@ The following decisions are complete and are not implementation-time choices:
  registration, or erase typed artifact relationships outside existing
  framework boundaries.
 - Exact cache identity means equal ordered message roles, content bytes, and
-  cache-control values after Scriptorium has rendered the prompt. Tests must
-  use `scriptorium.PreparedRun.Messages`; comparing source Markdown is
-  insufficient.
+  cache-control values after Scriptorium has rendered the prompt. Enforce this
+  primarily through one canonical shared asset per shared message and document
+  the expected order and cache-control policy.
 - Scriptorium and provider-specific types remain confined to the LLM runtime,
  prompt-asset wiring, and their tests. Production extractors continue to use
  only Notarius contracts.
 - Default tests remain offline, deterministic, and credential-free. Live model
  evaluation is an explicit manual acceptance activity, not part of
  `go test ./...`.
- Avoid broad snapshots. Exact byte assertions are warranted only for the
-  rendered common message prefixes because byte identity is the feature.
+- Do not add message-count, common-prefix-length, or complete rendered-prompt
+  snapshots. They are change detectors rather than durable behavioral tests.
 - Update current-behavior documentation only in the stage that changes that
  behavior. Do not describe a partially implemented later stage as complete.

@@ -100,34 +100,21 @@ spell/combat breakpoint. Use no more than these four markers so the request
 remains portable across the configured backends; do not add module-specific
 markers in this work.

-## Stage 1: Characterize The Rendered Cache Boundary
+## Stage 1: Protect Shared Input Behavior And Baseline Prompt Wiring

 ### Goal

-Protect the current three-message common prefix and establish reusable test
-support before changing prompt composition.
+Protect deterministic shared input rendering and confirm the existing prompt
+assets prepare successfully before changing prompt composition.

 ### Implementation

-1. Add a cross-lane prompt test under
-   `internal/modules/dnd/shared`, using the external test package
-   `shared_test` so it can import all three extractor packages without an
-   import cycle.
-2. In one `llm.AssetRegistry`, register spell, NPC, and combat prompt assets.
-   Build one offline Scriptorium engine with one test profile and use
-   `Engine.Prepare` for every prompt. Supply identical transcript, players,
-   party, and glossary inputs. Supply the same unbound NPC JSON to spell and
-   combat, plus the spell catalog required only by spells.
-3. Add a small test helper that compares an expected prefix of
-   `[]scriptorium.RenderedMessage` field by field: role, exact content string,
-   and a canonical JSON representation of cache control. Do not compare prompt
-   IDs, rendered prompt hashes, output contracts, or schemas across modules;
-   those are intentionally different.
-4. Assert that the first three messages are identical across all lanes and that
-   spell/combat NPC grounding is identical at their current corresponding
-   position. Assert the existing message counts separately so Stage 3 must
-   update them deliberately.
-5. Extend `internal/modules/dnd/shared/prompt_inputs_test.go` with one
+1. Retain one offline prompt preparation test in each extractor package. Each
+   test should prove registration succeeds, the prompt selects its package-owned
+   response schema, required dynamic inputs render, and no provider call or
+   credentials are required. Do not assert total message count, shared-prefix
+   length, or the complete rendered prompt.
+2. Extend `internal/modules/dnd/shared/prompt_inputs_test.go` with one
   table-driven identity test covering:
   - identical source material and references produce deeply equal common input
     materials;
@@ -135,11 +122,9 @@ support before changing prompt composition.
   - `roster` fallback produces the canonical `party` material;
   - an explicit non-empty `party` wins over `roster`; and
   - missing optional slots render the existing single-space placeholder.
-6. Do not add a second full HTTP/provider test. The Scriptorium engine's
-   prepared messages are the stable boundary owned by this repository;
-   Scriptorium owns provider request serialization. Existing LLM adapter tests
-   continue to prove that prepared message order and content reach the engine
-   request.
+3. Do not add a full HTTP/provider test for cross-module prompt equality.
+   Scriptorium owns provider request serialization, and shared asset ownership
+   removes the duplicated content that would otherwise need equality testing.

 ### Verification

@@ -151,10 +136,10 @@ go test ./internal/modules/dnd/shared ./internal/modules/dnd/extract/spells ./in

 ### Completion Criteria

- The new cross-lane test fails if any role, content byte, cache-control value,
-  or order within the intended common prefix differs.
- Tests distinguish intended prompt identity from intentionally different
-  prompt IDs, schemas, and module-specific suffixes.
+- Shared input tests protect deterministic serialization and canonical slot
+  handling.
+- Each extractor's existing behavior-level asset test prepares successfully
+  without pinning message count or prefix boundaries.
 - No production behavior changes in this stage.

 ## Stage 2: Make Prompt Asset Manifests Exact
@@ -233,8 +218,8 @@ go test ./internal/framework/promptfs ./internal/modules/dnd/shared ./internal/m

 ### Goal

-Implement the exact target prompt layout defined above and maximize the
-byte-identical all-lane and spell/combat prefixes.
+Implement the target prompt layout defined above using canonical shared assets
+and maximize the reusable all-lane and spell/combat content.

 ### Implementation

@@ -257,17 +242,12 @@ byte-identical all-lane and spell/combat prefixes.
   `Return exactly one JSON object` rule through the common evidence message.
 6. Keep the spell catalog input and NPC registry input byte generation
   unchanged. Keep the NPC extractor free of an `npcs` input.
-7. Update package asset tests for new message counts and positions. Replace
-   content-substring assertions for moved common policy with assertions in the
-   cross-lane rendered-prefix test.
-8. Update the cross-lane test to assert:
-   - indices 0 through 4 are identical for all three lanes when common inputs
-     are identical;
-   - indices 0 through 2 remain identical when transcript and references differ;
-   - spell and combat indices 5 and 6 are identical for the same NPC registry;
-   - the shared prefixes end at the exact indices in the table; and
-   - module-specific suffixes are not accidentally identical or reordered.
-9. Update `docs/internal/modules.md` and `docs/internal/llm.md` in the same
+7. Update package asset tests only for behavioral wiring: prompt preparation,
+   response schema selection, and required dynamic input rendering. Do not add
+   assertions for total message count, exact shared-prefix length, or the end
+   index of a shared section. The shared manifest tests own canonical asset
+   selection.
+8. Update `docs/internal/modules.md` and `docs/internal/llm.md` in the same
   change to describe the implemented common-prefix composition, exact prompt
   manifest fingerprinting, and cache-control placement. Describe current
   behavior, not this staged plan.
@@ -283,8 +263,9 @@ go test ./internal/framework/llm

 ### Completion Criteria

- Prepared messages match the target table exactly.
 - Shared rules exist in one embedded file and have no local paraphrased copy.
+- Every applicable prompt references those shared assets in the documented
+  order and with the documented cache-control policy.
 - Prompt fingerprints change for the intentional prompt contract change and
  include every newly rendered asset.
 - No artifact schema, durable representation, module selection, or reference
@@ -339,7 +320,7 @@ go test ./internal/modules/dnd/shared ./internal/modules/dnd/extract/spells ./in

 - Only one production implementation prepares chunk prompt material.
 - Extractor behavior and public errors retain useful module context.
- The rendered prompt identity tests still pass.
+- Prompt preparation and shared input behavior tests still pass.

 ## Stage 5: Share Cited Source Traversal

@@ -534,7 +515,7 @@ default test suite.
   the fixed decision for this plan is to leave them package-owned and separate.
 4. Consolidate redundant tests created by intermediate stages. Retain:
   - one shared manifest test suite;
-   - one exact rendered-prefix suite;
+   - behavior-level prompt preparation tests without prefix-length snapshots;
   - focused package tests for artifact-specific prompts and validators; and
   - existing production registration contract coverage.
 5. Update `docs/internal/modules.md` and `docs/internal/llm.md` so they are the