Plan combat scene validation

docs/roadmap/implementation.md

@@ -0,0 +1,488 @@
+# Implementation Plan
+
+## Objective
+
+Implement [D&D Combat Scene Semantic Validation](combat-scene-validation.md):
+add the first production D&D LLM-backed validator as an optional extract-stage
+validator for `dnd/scene-descriptions`, while preserving the existing chunk
+plan, durable artifacts, downstream combat gate, maintained examples, and
+production default validator chains.
+
+This plan is ordered and decision-complete. Each numbered stage is scoped for
+one gpt-5.6-terra implementation prompt. Complete and validate each stage
+before beginning the next one.
+
+## Decisions Shared By All Stages
+
+- Scene classification remains owned by the per-chunk
+  `dnd/scene-descriptions` extractor. Do not annotate `dnd/scenes` plans or add
+  a chunk-stage validator.
+- Add the typed validator package
+  `internal/modules/dnd/validate/scenedescriptions/combat_semantics` with key
+  `extract/dnd/scene-descriptions/combat_semantics`, registered for
+  `dnd.SceneDescriptionListKind` with execution class `llm_backed`.
+- The validator is selectable through the existing validator override but is
+  absent from all production default chains. It reviews only combat versus
+  non-combat, not title, summary, non-combat subtype, or boundary quality.
+- Reuse the injected scheduled `StructuredLLMClient`, existing profile
+  precedence, PromptKit structural-repair budget, validator execution retries,
+  producer semantic retries, and validation failure policy. Add no parallel
+  provider, scheduler, retry, or configuration mechanism.
+- Use prompt ID `dnd.scene_descriptions.validate_combat`, default profile
+  `dnd-extraction`, and private prompt/schema version `v1`.
+- The LLM response has exactly two required fields: `verdict` and
+  `explanation`. Verdict is one of `approved`, `combat_should_be_added`, or
+  `combat_should_be_removed`. Reject unknown fields, optional properties, and
+  `uniqueItems`. Require a trimmed, nonblank explanation of at most 512 Unicode
+  code points in both schema and deterministic interpretation.
+- Map rejections to application-owned reason codes
+  `scene_active_combat_not_classified` and
+  `scene_combat_classification_unsupported`. Codes, validator keys, opaque IDs,
+  and source coordinates are never model instructions.
+- Transcript unit IDs may remain in ordinary source presentation, but the
+  model must not reproduce them. Candidate presentation contains only the
+  proposed kind; application code owns IDs, ranges, mapping, and provenance.
+- PromptKit may repair structural validator output. A final malformed,
+  inconsistent, blank, or oversized validator response is an execution
+  failure, not a semantic rejection. Do not recursively validate or provide
+  semantic feedback to the validator's own LLM call.
+- Rejection guidance tells the producer either to return `kind: combat` or to
+  choose the appropriate `narrative`, `recap`, or `meta` kind, and includes the
+  bounded transcript-grounded explanation. It contains no internal code.
+- Keep durable schemas at their current versions and retain the exact `combat`
+  gates in combat-turn and enemy-event extraction.
+- Default tests are deterministic, offline, and credential-free. Use fakes at
+  the structured LLM boundary. Do not add exact prompt-length, token-count, or
+  complete prose snapshot tests.
+- No ADR is required. This feature applies the architecture and ADRs 0012,
+  0014, and 0015 without changing stage ownership or a durable contract.
+
+## Stage 1: Extract The Shared Combat Policy
+
+### Goal
+
+Give the existing extractor and future validator one byte-identical owner for
+the combat decision policy without changing the extractor contract or the
+common extraction prompt prefix.
+
+### Work
+
+1. Add `assets/dnd/shared/prompts/common-dnd-scene-combat-policy.md` and move
+   into it the current combat definition, mixed-chunk precedence, and explicit
+   non-examples: planning, threats, hostile dialogue, aftermath, recollection,
+   and rules discussion without active encounter play.
+2. Remove those rules from the local scene-description instructions. Retain
+   the exactly-one-scene task, all four supported kind names, the
+   `narrative`/`recap`/`meta` definitions and precedence, and title/summary
+   policy. The rendered prompt must remain complete and non-repetitive.
+3. Add the shared file to the scene extractor's `PromptAssetManifest` and
+   `prompt.yaml`. Preserve the extraction-wide prefix through the cached
+   transcript. Place the combat-policy message after the transcript and before
+   local instructions so other extraction lanes retain an identical prefix.
+4. Update prompt asset and composition tests to prove the shared policy appears
+   once in the correct relative position. Use semantic sentinels rather than a
+   full prompt snapshot or exact size assertion.
+5. Do not change the prompt ID/version, response schema, extractor logic,
+   artifact, or validator chains.
+
+### Inspect
+
+- `assets/dnd/shared/prompts/`
+- `assets/dnd/scene-descriptions/prompts/`
+- `internal/modules/dnd/extract/scenedescriptions/prompt_assets.go`
+- `internal/modules/dnd/extract/scenedescriptions/prompt_assets_test.go`
+- `internal/modules/dnd/register/prompt_cache_test.go`
+
+### Acceptance And Validation
+
+- One embedded shared asset owns the combat policy.
+- The scene prompt still prepares with complete classification instructions.
+- All extraction prompts retain the same prefix through the transcript.
+- No brittle change-detector test is added.
+
+Run:
+
+```sh
+go test ./internal/modules/dnd/extract/scenedescriptions
+go test ./internal/modules/dnd/register
+git diff --check
+```
+
+This stage is small enough for one implementation prompt.
+
+## Stage 2: Add Validator Prompt And Schema Assets
+
+### Goal
+
+Create and locally verify the private PromptKit contract before implementing
+semantic logic or global registration.
+
+### Work
+
+1. Create
+   `assets/dnd/scene-descriptions/validate/combat-semantics/` with:
+
+   - `prompts/prompt.yaml`;
+   - `prompts/instructions.md`;
+   - `prompts/proposed-kind.md`; and
+   - `schemas/dnd_scene_combat_semantics_llm.v1.json`.
+
+2. Define required inputs `transcript` (`application/json`) and
+   `proposed_kind` (`text/plain`). Order messages as system, shared combat
+   policy, stable local validator instructions, variable proposed kind, and
+   final variable shared chunk transcript. Mark only the two variable messages
+   ephemeral. Do not select campaign references or the identity fragment.
+3. Define the verdict semantics precisely. `approved` approves only the
+   proposed combat status, not the whole scene description.
+4. Define a strict object schema with `additionalProperties: false`, both
+   fields in `required`, the three-value verdict enum, and explanation
+   `minLength: 1` and `maxLength: 512`. Include no confidence, ID, range,
+   diagnostics, or correction field.
+5. Add package-local asset/schema plumbing in `combat_semantics`, following
+   existing D&D manifests. Use:
+
+   - schema key `dnd_scene_combat_semantics_llm`;
+   - schema ID `notarius.dnd.scene_descriptions.combat_semantics.llm`; and
+   - schema name `notarius_dnd_scene_combat_semantics_llm_v1`.
+
+   Expose `RegisterPromptAssets`, cached asset metadata, and mutation-safe
+   schema loading. Do not register them at the D&D composition root yet.
+6. Add focused tests that prepare the embedded prompt, require both inputs,
+   accept all valid verdicts, and reject missing, extra, mistyped, blank, and
+   oversized fields. Metadata tests must prove hashes are available without
+   exposing raw assets.
+
+### Inspect
+
+- existing scene-description extractor asset and schema code
+- analogous D&D semantic-reconciliation prompt assets
+- `internal/framework/llm`
+- `internal/framework/promptfs`
+
+### Acceptance And Validation
+
+- The package-local embedded prompt and schema prepare offline.
+- All schema fields are required and no `uniqueItems` appears.
+- The prompt contains only the policy, proposed kind, and transcript needed for
+  this decision and selects rather than copies the shared policy.
+- No global registration changes occur yet.
+
+Run:
+
+```sh
+go test ./internal/modules/dnd/validate/scenedescriptions/combat_semantics
+go test ./internal/modules/dnd/extract/scenedescriptions
+git diff --check
+```
+
+This stage is small enough for one implementation prompt.
+
+## Stage 3: Implement The Typed Validator
+
+### Goal
+
+Implement package-local validator construction, preconditions, structured
+completion, verdict interpretation, corrections, fingerprints, and behavioral
+tests.
+
+### Work
+
+1. Define the module key, two reason codes, zero-field strict `Options`, private
+   response type, and validator holding only the injected client and immutable
+   prompt/schema/policy metadata. Construction fails contextually for a nil LLM
+   dependency.
+2. Register through a typed builder consuming `request.Dependencies.LLM`.
+   Retain no unrelated references or mutable request maps.
+3. Before calling the model, require extract stage, non-nil source and chunk,
+   valid chunk-scoped source input, exactly one scene satisfying the existing
+   extract shape contract, and scene ID/range equal to the current chunk.
+   Violations are execution errors, not approval or skip.
+4. Call `CompleteStructured` once per execution with validator key as stage
+   name; the selected prompt/version; request profile, session ID, and repair
+   pointer; cloned chunk source input as `transcript`; and only the scene kind
+   string as `proposed_kind`. Do not forward references, scene prose,
+   identifiers, ranges, reason codes, or semantic correction.
+5. Interpret output deterministically:
+
+   - trim and enforce the 512-code-point explanation bound;
+   - approve `approved` for either proposed combat status;
+   - accept `combat_should_be_added` only for a non-combat proposal;
+   - accept `combat_should_be_removed` only for a combat proposal; and
+   - treat every unknown or inconsistent result as execution failure.
+
+6. Build bounded `ValidationResult` messages and semantic correction guidance.
+   Approved results carry no rejection message or quality diagnostic.
+7. Implement `Name`, `ExecutionClass`, `Spec`, manifest metadata, checkpoint
+   fingerprints, and strict option validation. A validator spec does not
+   declare a producer correction protocol.
+8. Add consolidated table-driven tests with a recording fake LLM for approvals,
+   both rejection directions, inconsistent results, explanation bounds, input
+   preconditions, completion error/cancellation, request-field forwarding,
+   model-visible input exclusions, option rejection, and metadata ownership.
+
+### Inspect
+
+- `internal/framework/contracts/{contracts,typed_pipeline}.go`
+- `internal/framework/pipeline/{construction,validator_registry}.go`
+- `internal/modules/dnd/validate/scenedescriptions/{shape,source_refs}`
+- existing LLM-backed D&D extractor and normalizer packages
+
+### Acceptance And Validation
+
+- The package is usable without provider-specific types or global state.
+- It cannot judge a wrong-stage, aggregate, malformed, or wrong-chunk value.
+- Semantic rejection and execution failure remain distinct.
+- Guidance is meaningful and contains no opaque identifiers or internal codes.
+- Consequential contracts are tested offline through the package boundary.
+
+Run:
+
+```sh
+go test ./internal/modules/dnd/validate/scenedescriptions/...
+go vet ./internal/modules/dnd/validate/scenedescriptions/...
+git diff --check
+```
+
+This stage is small enough for one implementation prompt.
+
+## Stage 4: Register And Resolve The Optional Validator
+
+### Goal
+
+Expose the module through production D&D composition and prove explicit
+configuration works without changing defaults.
+
+### Work
+
+1. Register the validator builder in
+   `internal/modules/dnd/register/validators.go` and its assets in
+   `internal/modules/dnd/register/modules.go`, preserving deterministic
+   registration and contextual errors.
+2. Extend registrar/catalog tests to prove the typed kind, execution class,
+   production prompt/schema availability, and absence from both scene default
+   chains.
+3. Add one focused resolution/preparation case whose scene extract override
+   repeats the current default validators in order and appends the semantic
+   validator last. Prove validator `llm_profile`,
+   `structured_output_repair_attempts`, and `retries` reach the prepared
+   LLM-backed validator through existing machinery.
+4. Add a negative case for selecting this typed validator against an
+   incompatible artifact kind if existing typed-registry tests do not already
+   exercise the concrete production key. Rely on generic tests for all other
+   validator-binding rules.
+5. Do not alter the minimal or complete D&D example configurations.
+
+### Inspect
+
+- `internal/modules/dnd/register/{validators,modules,chains}.go`
+- `internal/modules/dnd/register/register_test.go`
+- `internal/modules/dnd/register/prompt_cache_test.go`
+- typed validator resolution/preparation tests under
+  `internal/framework/pipeline`
+
+### Acceptance And Validation
+
+- Production registration exposes the validator and prompt.
+- An explicit compatible override resolves before execution.
+- Default chain membership/order and maintained examples are unchanged.
+- Incompatible selection still fails during resolution or preparation.
+
+Run:
+
+```sh
+go test ./internal/modules/dnd/register
+go test ./internal/framework/pipeline
+go test ./internal/cli
+git diff --check
+```
+
+This stage is small enough for one implementation prompt.
+
+## Stage 5: Prove Feedback-Aware Producer Correction
+
+### Goal
+
+Prove the assembled D&D path can reject a well-formed wrong scene kind, guide
+the scene producer, and accept a corrected replacement without leaking
+internal identifiers.
+
+### Work
+
+1. Add one focused test under `internal/modules/integration` (or extend the
+   narrowest assembled scene test) with a stateful fake structured client that
+   distinguishes scene producer and semantic validator prompt IDs.
+2. Configure the current deterministic scene extract chain followed by the
+   semantic validator, with a positive producer retry budget and no unnecessary
+   validator execution retries.
+3. Use a chunk containing clear active combat. Return a valid non-combat scene
+   first, `combat_should_be_added` from the validator, a corrected combat scene
+   after feedback, and approval on revalidation.
+4. Assert outcomes rather than private choreography:
+
+   - the accepted normalized scene is `combat`;
+   - superseded attempts leave no rejection or process warning;
+   - the producer receives the exact latest defective model candidate and one
+     semantic correction request;
+   - guidance directs `kind: combat` and includes the bounded explanation;
+   - guidance excludes validator key, reason code, chunk ID, and source range;
+     and
+   - the corrected candidate traverses the complete validator chain again.
+
+5. Keep unsupported-combat verdict mapping owned by Stage 3 package tests
+   unless adding it to this integration test is materially cheaper than a
+   duplicate assembled setup.
+6. Rely on existing framework tests for aggregate guidance, exhaustion,
+   `warn_continue`, validator retries, and checkpoint ineligibility. Change
+   generic framework code only if this test exposes a real defect, and report
+   the scope expansion explicitly.
+
+### Inspect
+
+- `internal/modules/integration/`
+- framework extract-correction and semantic-correction tests
+- `internal/cli/assembled_spell_pipeline_contract_test.go`
+- `internal/modules/dnd/extract/scenedescriptions/`
+
+### Acceptance And Validation
+
+- One deterministic assembled test proves D&D semantic correction and
+  revalidation.
+- Only the latest defective response and semantic guidance reach the producer.
+- No internal code or deterministic identifier reaches correction text.
+- Existing framework budgets and policies remain the only orchestration path.
+
+Run:
+
+```sh
+go test ./internal/modules/integration
+go test ./internal/framework/pipeline
+go test ./internal/modules/dnd/extract/scenedescriptions
+git diff --check
+```
+
+This stage is small enough for one implementation prompt.
+
+## Stage 6: Add Evaluation Material And Documentation
+
+### Goal
+
+Provide accurate opt-in configuration guidance and a bounded human-reviewed
+corpus for evaluation before any future default-chain proposal.
+
+### Work
+
+1. Add a synthetic, non-sensitive corpus at
+   `internal/modules/dnd/validate/scenedescriptions/combat_semantics/testdata/evaluation_cases.json`.
+   Each case has a human-readable unique name, transcript units, proposed kind,
+   expected verdict, and reviewer rationale. Cover active combat, setup and
+   aftermath around combat, multi-phase combat, rules interruption, planning,
+   threats, hostile dialogue, aftermath without combat, recap/recollection,
+   and sustained out-of-character discussion.
+2. Add one cheap fixture-contract test for strict decoding, unique/nonblank
+   names, nonblank input, supported proposed kinds/verdicts, and representation
+   of all verdict classes. Do not assert the exact fixture count or treat fake
+   responses as a measurement of model quality.
+3. Document a manual or explicitly opt-in provider evaluation procedure in
+   `docs/internal/dnd.md`. Require recording false acceptance, false rejection,
+   producer correction success, added calls, latency, and token use before
+   default-chain promotion. Do not add live calls to the default suite.
+4. Update `docs/config.md`, the canonical configuration owner, to list the key,
+   state its LLM-backed opt-in status, and show the smallest illustrative
+   override that repeats the existing scene extract chain and appends the
+   semantic validator. Emphasize that an override replaces the chain and that
+   a positive producer retry budget is required if rejection should trigger a
+   corrected scene-description attempt. Link to the existing binding table
+   instead of repeating profile, structural-repair, producer-retry, or
+   validator-retry rules.
+5. Update `docs/internal/dnd.md` to describe the module boundary, prompt-policy
+   ownership, verdict mapping, deterministic correction mapping, and difference
+   from deferred boundary-coherence review. Link to configuration and generic
+   retry/failure owners rather than duplicating volatile contracts.
+6. Update `docs/internal/overview.md` or `docs/internal/modules.md` only if its
+   implemented inventory would otherwise become false. Do not add an ADR, a
+   third example, or a durable integration contract change.
+
+### Inspect
+
+- `docs/{config,internal/dnd,internal/modules,internal/overview}.md`
+- all files under `docs/policy/`
+- the two maintained D&D example configurations
+
+### Acceptance And Validation
+
+- The corpus is bounded, synthetic, strict, and covers the intended cases
+  without masquerading as a provider quality test.
+- Configuration documents one correct opt-in path and preserves replacement
+  semantics.
+- Canonical documentation owners describe only behavior implemented in Stages
+  1–5, with no volatile duplication or default-enabled claim.
+- All new relative Markdown links resolve manually; the repository currently
+  has no automated documentation link checker.
+
+Run:
+
+```sh
+go test ./internal/modules/dnd/validate/scenedescriptions/combat_semantics
+go test ./internal/modules/dnd/register ./internal/framework/pipeline ./internal/cli
+git diff --check
+```
+
+This stage is small enough for one implementation prompt.
+
+## Stage 7: Final Verification And Scope Audit
+
+### Goal
+
+Verify the feature as one coherent change and leave it ready for a separate
+post-implementation review.
+
+### Work
+
+1. Compare the complete diff with the feature roadmap and every stage above.
+   Implement or report every unmet acceptance criterion; do not silently defer
+   required work.
+2. Confirm by inspection that the validator is registered but absent from
+   default chains/examples; both consumers select the shared combat policy; no
+   duplicate policy remains; its schema has all fields required and no
+   `uniqueItems`; model-facing content requests no opaque IDs or internal
+   codes; it uses the scheduled client and existing budgets; and downstream
+   exact-combat gates are unchanged.
+3. Format changed Go files. Remove only redundant tests, stale comments, unused
+   helpers, or roadmap-inconsistent documentation found during this review; do
+   not perform unrelated refactoring.
+4. Run the full validation suite. If an environmental limitation blocks a
+   command, run unaffected checks and record the exact limitation rather than
+   weakening tests.
+5. Leave both roadmap documents in place for the post-implementation audit. Do
+   not promote the validator or retire the roadmaps.
+
+### Acceptance And Validation
+
+- Focused and repository-wide checks pass.
+- No durable schema, chunk plan, default chain, example, or downstream gate
+  changed outside the roadmap.
+- The worktree contains no binary, credential, private transcript, ignored
+  debris, or unrelated modification.
+- Documentation paths and links resolve, and the repository is ready for
+  completeness and code-quality review.
+
+Run:
+
+```sh
+gofmt -w <changed-go-files>
+go test ./...
+go test -race ./...
+go vet ./...
+go build ./cmd/notarius
+git diff --check
+git status --short --untracked-files=all
+```
+
+This stage is small enough for one implementation prompt.
+
+## Open Questions
+
+None. The feature roadmap and this plan fix the stage, scope, schema, prompt
+ownership, reason-code mapping, configuration status, failure semantics,
+evaluation gate, and documentation ownership. Default-chain promotion and
+chunk-boundary semantic review are explicitly outside this plan.