notarius/docs/roadmap/implementation.md

Implementation Plan

Objective

Implement D&D Combat Scene Semantic Validation: add the first production D&D LLM-backed validator as an optional extract-stage validator for dnd/scene-descriptions, while preserving the existing chunk plan, durable artifacts, downstream combat gate, maintained examples, and production default validator chains.

This plan is ordered and decision-complete. Each numbered stage is scoped for one gpt-5.6-terra implementation prompt. Complete and validate each stage before beginning the next one.

Decisions Shared By All Stages

• Scene classification remains owned by the per-chunk dnd/scene-descriptions extractor. Do not annotate dnd/scenes plans or add a chunk-stage validator.
• Add the typed validator package internal/modules/dnd/validate/scenedescriptions/combat_semantics with key extract/dnd/scene-descriptions/combat_semantics, registered for dnd.SceneDescriptionListKind with execution class llm_backed.
• The validator is selectable through the existing validator override but is absent from all production default chains. It reviews only combat versus non-combat, not title, summary, non-combat subtype, or boundary quality.
• Reuse the injected scheduled StructuredLLMClient, existing profile precedence, PromptKit structural-repair budget, validator execution retries, producer semantic retries, and validation failure policy. Add no parallel provider, scheduler, retry, or configuration mechanism.
• Use prompt ID dnd.scene_descriptions.validate_combat, default profile dnd-extraction, and private prompt/schema version v1.
• The LLM response has exactly two required fields: verdict and explanation. Verdict is one of approved, combat_should_be_added, or combat_should_be_removed. Reject unknown fields, optional properties, and uniqueItems. Require a trimmed, nonblank explanation of at most 512 Unicode code points in both schema and deterministic interpretation.
• Map rejections to application-owned reason codes scene_active_combat_not_classified and scene_combat_classification_unsupported. Codes, validator keys, opaque IDs, and source coordinates are never model instructions.
• Transcript unit IDs may remain in ordinary source presentation, but the model must not reproduce them. Candidate presentation contains only the proposed kind; application code owns IDs, ranges, mapping, and provenance.
• PromptKit may repair structural validator output. A final malformed, inconsistent, blank, or oversized validator response is an execution failure, not a semantic rejection. Do not recursively validate or provide semantic feedback to the validator's own LLM call.
• Rejection guidance tells the producer either to return kind: combat or to choose the appropriate narrative, recap, or meta kind, and includes the bounded transcript-grounded explanation. It contains no internal code.
• Keep durable schemas at their current versions and retain the exact combat gates in combat-turn and enemy-event extraction.
• Default tests are deterministic, offline, and credential-free. Use fakes at the structured LLM boundary. Do not add exact prompt-length, token-count, or complete prose snapshot tests.
• No ADR is required. This feature applies the architecture and ADRs 0012, 0014, and 0015 without changing stage ownership or a durable contract.

Stage 1: Extract The Shared Combat Policy

✅ Complete

Goal

Give the existing extractor and future validator one byte-identical owner for the combat decision policy without changing the extractor contract or the common extraction prompt prefix.

Work

1. Add assets/dnd/shared/prompts/common-dnd-scene-combat-policy.md and move into it the current combat definition, mixed-chunk precedence, and explicit non-examples: planning, threats, hostile dialogue, aftermath, recollection, and rules discussion without active encounter play.
2. Remove those rules from the local scene-description instructions. Retain the exactly-one-scene task, all four supported kind names, the narrative/recap/meta definitions and precedence, and title/summary policy. The rendered prompt must remain complete and non-repetitive.
3. Add the shared file to the scene extractor's PromptAssetManifest and prompt.yaml. Preserve the extraction-wide prefix through the cached transcript. Place the combat-policy message after the transcript and before local instructions so other extraction lanes retain an identical prefix.
4. Update prompt asset and composition tests to prove the shared policy appears once in the correct relative position. Use semantic sentinels rather than a full prompt snapshot or exact size assertion.
5. Do not change the prompt ID/version, response schema, extractor logic, artifact, or validator chains.

Inspect

• assets/dnd/shared/prompts/
• assets/dnd/scene-descriptions/prompts/
• internal/modules/dnd/extract/scenedescriptions/prompt_assets.go
• internal/modules/dnd/extract/scenedescriptions/prompt_assets_test.go
• internal/modules/dnd/register/prompt_cache_test.go

Acceptance And Validation

• One embedded shared asset owns the combat policy.
• The scene prompt still prepares with complete classification instructions.
• All extraction prompts retain the same prefix through the transcript.
• No brittle change-detector test is added.

Run:

go test ./internal/modules/dnd/extract/scenedescriptions
go test ./internal/modules/dnd/register
git diff --check

This stage is small enough for one implementation prompt.

Stage 2: Add Validator Prompt And Schema Assets

✅ Complete

Goal

Create and locally verify the private PromptKit contract before implementing semantic logic or global registration.

Work

1. Create assets/dnd/scene-descriptions/validate/combat-semantics/ with:

   • prompts/prompt.yaml;
   • prompts/instructions.md;
   • prompts/proposed-kind.md; and
   • schemas/dnd_scene_combat_semantics_llm.v1.json.

2. Define required inputs transcript (application/json) and proposed_kind (text/plain). Order messages as system, shared combat policy, stable local validator instructions, variable proposed kind, and final variable shared chunk transcript. Mark only the two variable messages ephemeral. Do not select campaign references or the identity fragment.

3. Define the verdict semantics precisely. approved approves only the proposed combat status, not the whole scene description.

4. Define a strict object schema with additionalProperties: false, both fields in required, the three-value verdict enum, and explanation minLength: 1 and maxLength: 512. Include no confidence, ID, range, diagnostics, or correction field.

5. Add package-local asset/schema plumbing in combat_semantics, following existing D&D manifests. Use:

   • schema key dnd_scene_combat_semantics_llm;
   • schema ID notarius.dnd.scene_descriptions.combat_semantics.llm; and
   • schema name notarius_dnd_scene_combat_semantics_llm_v1.

   Expose RegisterPromptAssets, cached asset metadata, and mutation-safe schema loading. Do not register them at the D&D composition root yet.

6. Add focused tests that prepare the embedded prompt, require both inputs, accept all valid verdicts, and reject missing, extra, mistyped, blank, and oversized fields. Metadata tests must prove hashes are available without exposing raw assets.

Inspect

• existing scene-description extractor asset and schema code
• analogous D&D semantic-reconciliation prompt assets
• internal/framework/llm
• internal/framework/promptfs

Acceptance And Validation

• The package-local embedded prompt and schema prepare offline.
• All schema fields are required and no uniqueItems appears.
• The prompt contains only the policy, proposed kind, and transcript needed for this decision and selects rather than copies the shared policy.
• No global registration changes occur yet.

Run:

go test ./internal/modules/dnd/validate/scenedescriptions/combat_semantics
go test ./internal/modules/dnd/extract/scenedescriptions
git diff --check

This stage is small enough for one implementation prompt.

Stage 3: Implement The Typed Validator

Goal

Implement package-local validator construction, preconditions, structured completion, verdict interpretation, corrections, fingerprints, and behavioral tests.

Work

1. Define the module key, two reason codes, zero-field strict Options, private response type, and validator holding only the injected client and immutable prompt/schema/policy metadata. Construction fails contextually for a nil LLM dependency.

2. Register through a typed builder consuming request.Dependencies.LLM. Retain no unrelated references or mutable request maps.

3. Before calling the model, require extract stage, non-nil source and chunk, valid chunk-scoped source input, exactly one scene satisfying the existing extract shape contract, and scene ID/range equal to the current chunk. Violations are execution errors, not approval or skip.

4. Call CompleteStructured once per execution with validator key as stage name; the selected prompt/version; request profile, session ID, and repair pointer; cloned chunk source input as transcript; and only the scene kind string as proposed_kind. Do not forward references, scene prose, identifiers, ranges, reason codes, or semantic correction.

5. Interpret output deterministically:

   • trim and enforce the 512-code-point explanation bound;
   • approve approved for either proposed combat status;
   • accept combat_should_be_added only for a non-combat proposal;
   • accept combat_should_be_removed only for a combat proposal; and
   • treat every unknown or inconsistent result as execution failure.

6. Build bounded ValidationResult messages and semantic correction guidance. Approved results carry no rejection message or quality diagnostic.

7. Implement Name, ExecutionClass, Spec, manifest metadata, checkpoint fingerprints, and strict option validation. A validator spec does not declare a producer correction protocol.

8. Add consolidated table-driven tests with a recording fake LLM for approvals, both rejection directions, inconsistent results, explanation bounds, input preconditions, completion error/cancellation, request-field forwarding, model-visible input exclusions, option rejection, and metadata ownership.

Inspect

• internal/framework/contracts/{contracts,typed_pipeline}.go
• internal/framework/pipeline/{construction,validator_registry}.go
• internal/modules/dnd/validate/scenedescriptions/{shape,source_refs}
• existing LLM-backed D&D extractor and normalizer packages

Acceptance And Validation

• The package is usable without provider-specific types or global state.
• It cannot judge a wrong-stage, aggregate, malformed, or wrong-chunk value.
• Semantic rejection and execution failure remain distinct.
• Guidance is meaningful and contains no opaque identifiers or internal codes.
• Consequential contracts are tested offline through the package boundary.

Run:

go test ./internal/modules/dnd/validate/scenedescriptions/...
go vet ./internal/modules/dnd/validate/scenedescriptions/...
git diff --check

This stage is small enough for one implementation prompt.

Stage 4: Register And Resolve The Optional Validator

Goal

Expose the module through production D&D composition and prove explicit configuration works without changing defaults.

Work

1. Register the validator builder in internal/modules/dnd/register/validators.go and its assets in internal/modules/dnd/register/modules.go, preserving deterministic registration and contextual errors.
2. Extend registrar/catalog tests to prove the typed kind, execution class, production prompt/schema availability, and absence from both scene default chains.
3. Add one focused resolution/preparation case whose scene extract override repeats the current default validators in order and appends the semantic validator last. Prove validator llm_profile, structured_output_repair_attempts, and retries reach the prepared LLM-backed validator through existing machinery.
4. Add a negative case for selecting this typed validator against an incompatible artifact kind if existing typed-registry tests do not already exercise the concrete production key. Rely on generic tests for all other validator-binding rules.
5. Do not alter the minimal or complete D&D example configurations.

Inspect

• internal/modules/dnd/register/{validators,modules,chains}.go
• internal/modules/dnd/register/register_test.go
• internal/modules/dnd/register/prompt_cache_test.go
• typed validator resolution/preparation tests under internal/framework/pipeline

Acceptance And Validation

• Production registration exposes the validator and prompt.
• An explicit compatible override resolves before execution.
• Default chain membership/order and maintained examples are unchanged.
• Incompatible selection still fails during resolution or preparation.

Run:

go test ./internal/modules/dnd/register
go test ./internal/framework/pipeline
go test ./internal/cli
git diff --check

This stage is small enough for one implementation prompt.

Stage 5: Prove Feedback-Aware Producer Correction

Goal

Prove the assembled D&D path can reject a well-formed wrong scene kind, guide the scene producer, and accept a corrected replacement without leaking internal identifiers.

Work

1. Add one focused test under internal/modules/integration (or extend the narrowest assembled scene test) with a stateful fake structured client that distinguishes scene producer and semantic validator prompt IDs.

2. Configure the current deterministic scene extract chain followed by the semantic validator, with a positive producer retry budget and no unnecessary validator execution retries.

3. Use a chunk containing clear active combat. Return a valid non-combat scene first, combat_should_be_added from the validator, a corrected combat scene after feedback, and approval on revalidation.

4. Assert outcomes rather than private choreography:

   • the accepted normalized scene is combat;
   • superseded attempts leave no rejection or process warning;
   • the producer receives the exact latest defective model candidate and one semantic correction request;
   • guidance directs kind: combat and includes the bounded explanation;
   • guidance excludes validator key, reason code, chunk ID, and source range; and
   • the corrected candidate traverses the complete validator chain again.

5. Keep unsupported-combat verdict mapping owned by Stage 3 package tests unless adding it to this integration test is materially cheaper than a duplicate assembled setup.

6. Rely on existing framework tests for aggregate guidance, exhaustion, warn_continue, validator retries, and checkpoint ineligibility. Change generic framework code only if this test exposes a real defect, and report the scope expansion explicitly.

Inspect

• internal/modules/integration/
• framework extract-correction and semantic-correction tests
• internal/cli/assembled_spell_pipeline_contract_test.go
• internal/modules/dnd/extract/scenedescriptions/

Acceptance And Validation

• One deterministic assembled test proves D&D semantic correction and revalidation.
• Only the latest defective response and semantic guidance reach the producer.
• No internal code or deterministic identifier reaches correction text.
• Existing framework budgets and policies remain the only orchestration path.

Run:

go test ./internal/modules/integration
go test ./internal/framework/pipeline
go test ./internal/modules/dnd/extract/scenedescriptions
git diff --check

This stage is small enough for one implementation prompt.

Stage 6: Add Evaluation Material And Documentation

Goal

Provide accurate opt-in configuration guidance and a bounded human-reviewed corpus for evaluation before any future default-chain proposal.

Work

1. Add a synthetic, non-sensitive corpus at internal/modules/dnd/validate/scenedescriptions/combat_semantics/testdata/evaluation_cases.json. Each case has a human-readable unique name, transcript units, proposed kind, expected verdict, and reviewer rationale. Cover active combat, setup and aftermath around combat, multi-phase combat, rules interruption, planning, threats, hostile dialogue, aftermath without combat, recap/recollection, and sustained out-of-character discussion.
2. Add one cheap fixture-contract test for strict decoding, unique/nonblank names, nonblank input, supported proposed kinds/verdicts, and representation of all verdict classes. Do not assert the exact fixture count or treat fake responses as a measurement of model quality.
3. Document a manual or explicitly opt-in provider evaluation procedure in docs/internal/dnd.md. Require recording false acceptance, false rejection, producer correction success, added calls, latency, and token use before default-chain promotion. Do not add live calls to the default suite.
4. Update docs/config.md, the canonical configuration owner, to list the key, state its LLM-backed opt-in status, and show the smallest illustrative override that repeats the existing scene extract chain and appends the semantic validator. Emphasize that an override replaces the chain and that a positive producer retry budget is required if rejection should trigger a corrected scene-description attempt. Link to the existing binding table instead of repeating profile, structural-repair, producer-retry, or validator-retry rules.
5. Update docs/internal/dnd.md to describe the module boundary, prompt-policy ownership, verdict mapping, deterministic correction mapping, and difference from deferred boundary-coherence review. Link to configuration and generic retry/failure owners rather than duplicating volatile contracts.
6. Update docs/internal/overview.md or docs/internal/modules.md only if its implemented inventory would otherwise become false. Do not add an ADR, a third example, or a durable integration contract change.

Inspect

• docs/{config,internal/dnd,internal/modules,internal/overview}.md
• all files under docs/policy/
• the two maintained D&D example configurations

Acceptance And Validation

• The corpus is bounded, synthetic, strict, and covers the intended cases without masquerading as a provider quality test.
• Configuration documents one correct opt-in path and preserves replacement semantics.
• Canonical documentation owners describe only behavior implemented in Stages 1–5, with no volatile duplication or default-enabled claim.
• All new relative Markdown links resolve manually; the repository currently has no automated documentation link checker.

Run:

go test ./internal/modules/dnd/validate/scenedescriptions/combat_semantics
go test ./internal/modules/dnd/register ./internal/framework/pipeline ./internal/cli
git diff --check

This stage is small enough for one implementation prompt.

Stage 7: Final Verification And Scope Audit

Goal

Verify the feature as one coherent change and leave it ready for a separate post-implementation review.

Work

1. Compare the complete diff with the feature roadmap and every stage above. Implement or report every unmet acceptance criterion; do not silently defer required work.
2. Confirm by inspection that the validator is registered but absent from default chains/examples; both consumers select the shared combat policy; no duplicate policy remains; its schema has all fields required and no uniqueItems; model-facing content requests no opaque IDs or internal codes; it uses the scheduled client and existing budgets; and downstream exact-combat gates are unchanged.
3. Format changed Go files. Remove only redundant tests, stale comments, unused helpers, or roadmap-inconsistent documentation found during this review; do not perform unrelated refactoring.
4. Run the full validation suite. If an environmental limitation blocks a command, run unaffected checks and record the exact limitation rather than weakening tests.
5. Leave both roadmap documents in place for the post-implementation audit. Do not promote the validator or retire the roadmaps.

Acceptance And Validation

• Focused and repository-wide checks pass.
• No durable schema, chunk plan, default chain, example, or downstream gate changed outside the roadmap.
• The worktree contains no binary, credential, private transcript, ignored debris, or unrelated modification.
• Documentation paths and links resolve, and the repository is ready for completeness and code-quality review.

Run:

gofmt -w <changed-go-files>
go test ./...
go test -race ./...
go vet ./...
go build ./cmd/notarius
git diff --check
git status --short --untracked-files=all

This stage is small enough for one implementation prompt.

Open Questions

None. The feature roadmap and this plan fix the stage, scope, schema, prompt ownership, reason-code mapping, configuration status, failure semantics, evaluation gate, and documentation ownership. Default-chain promotion and chunk-boundary semantic review are explicitly outside this plan.