notarius/docs/roadmap/implementation.md

D&D Validation-Boundary Alignment Implementation Plan

Status: Proposed.

Implement this plan in order. It repairs the immediate combat extraction failure, makes D&D validation diagnostics consistently domain-owned, and then aligns the spell and NPC private response boundaries with the same policy.

Do not change durable artifact schemas, artifact kinds, public Go types, framework retry behavior, checkpoint formats, or the scene chunker. Existing v1 private-schema identities may be corrected in place because Notarius has not been run in production. Changed prompt and schema content will invalidate development checkpoints through existing fingerprints.

Cross-Stage Decisions

Private LLM schemas own the transport envelope: required fields, JSON types, nullability, array/object shape, and unknown-field rejection. Deterministic domain validators own semantic rules: supported enum values, nonblank values, required non-empty collections, positive and resolvable source units, catalog/identity policy, and normalized invariants.

For typed D&D artifacts, generic/valid_json remains first as a representation sanity check. All deterministic domain validators that can reject a candidate run next. generic/valid_json_schema runs after them as a durable-schema backstop, followed by warning-only relatedness validators. This order gives expected candidate failures bounded domain reason codes while retaining a final check that typed encoding conforms to the durable contract.

Do not expose raw JSON Schema errors or candidate values through the generic validator. Do not add tests that require particular words or phrases to remain in prompt prose.

Stage 1: Repair Combat Extraction

Changes

• Update the combat extraction instructions to enumerate the complete allowed values:
  • turn_kind: turn, reaction, legendary_action, lair_action, other;
  • action category: attack, spell, movement, item, ability_check, saving_throw, condition, other.
• Keep the private combat schema structurally permissive and the durable schema strict. Do not restore enum, minimum, minLength, or minItems constraints to the private schema.
• Reorder the combat extraction default chain to: generic/valid_json, combat shape, combat source references, generic/valid_json_schema, combat source relatedness.
• Reorder the combat normalization default chain to: generic/valid_json, combat shape, normalized invariants, combat source references, generic/valid_json_schema, combat source relatedness.
• Preserve configured validator overrides as authoritative; only production default composition changes.

Tests

• Add an assembled combat pipeline case whose raw LLM response contains an unsupported turn kind or action category. Exhausted retries must produce a non-fatal invalid_combat_turn_shape rejection owned by the combat shape validator, not json_schema_invalid or a framework error.
• Retain coverage that a later valid retry succeeds and discarded-attempt warnings/rejections do not become durable.
• Update registrar contract tests to assert the new extraction and normalization order.
• Rely on behavioral enum/schema tests and prompt fingerprint coverage; do not add prompt-word change-detector tests.

Completion Check

Run go test ./internal/modules/dnd/extract/combatturns ./internal/modules/dnd/validate/combatturns/... ./internal/modules/dnd/register ./internal/modules/integration and git diff --check.

Stage 2: Make D&D Validator Ordering Consistent

Changes

• Move generic/valid_json_schema behind all rejecting domain validators in every spell and NPC extraction and normalization default chain:
  • spell extraction/normalization: shape, catalog, source references, schema, then source relatedness;
  • NPC extraction: shape, source references, schema, then source relatedness;
  • NPC normalization: shape, identity, source references, schema, then source relatedness.
• Keep generic/valid_json first and warning-only source relatedness last.
• Do not change validator implementations, reason codes, warning promotion, retry counts, or user-provided chain order.
• Document the default-chain policy in the pipeline/module internals: domain validators diagnose expected semantic failures and the generic schema validator is the final rejecting representation backstop.

Tests

• Update production registrar tests for every affected chain.
• Add one representative spell and NPC assembled rejection proving that a domain-invalid but encodable candidate is attributed to the owning domain validator rather than the generic schema validator. Do not duplicate each validator package's existing case matrix at integration level.
• Confirm explicitly configured validator chains retain their exact configured order.

Completion Check

Run go test ./internal/modules/dnd/register ./internal/modules/integration ./internal/framework/pipeline and git diff --check.

Stage 3: Align Spell and NPC Private Response Boundaries

Changes

• Revise the existing v1 spell and NPC private schemas in place:
  • retain required fields, JSON types, array/object structure, additionalProperties: false, and omission of framework-assigned fields;
  • remove minLength, minItems, and positive-number minimum constraints;
  • leave the durable spell and NPC schemas unchanged.
• Replace shared.UnitRef in the private spell and NPC response DTOs with integer candidates so zero and negative unit IDs survive decoding and mapping into source.SourceRef for deterministic source validation.
• Update canonicalization and ordering helpers to operate on candidate integers without repairing invalid values. Valid positive IDs retain current output, ordering, and exact-deduplication behavior; invalid ranges remain available to validators.
• Keep malformed JSON, missing/unknown fields, wrong JSON types, and non-integer source IDs as LLM-boundary errors.
• Update LLM/module internals and the spell/NPC integration contracts to state the structural-private/semantic-validator ownership boundary.

Tests

• For each private schema, prove structurally valid candidates with blank strings, empty required collections, and nonpositive unit IDs pass the private schema, while missing fields, unknown fields, and wrong JSON types do not.
• Through raw-JSON LLM fakes, prove semantic values survive decoding and mapping without repair.
• Add representative assembled cases showing:
  • blank or empty spell/NPC fields are rejected by the appropriate shape validator;
  • nonpositive or nonexistent unit IDs are rejected by the appropriate source validator; and
  • exhausted validation retries remain non-fatal rejected outputs.
• Preserve existing valid mapping, source-position ordering, deduplication, catalog, identity, checkpoint-fingerprint, and durable codec tests.

Completion Check

Run go test ./internal/modules/dnd/extract/spells ./internal/modules/dnd/extract/npcs ./internal/modules/dnd/validate/spells/... ./internal/modules/dnd/validate/npcs/... ./internal/modules/integration and git diff --check.

Final Verification

Run:

git diff --check
go test ./...
go vet ./...
go build ./cmd/notarius
go test -race ./internal/modules/dnd/... ./internal/framework/pipeline ./internal/cli ./internal/modules/integration

Review current-behavior documentation for stale statements that private spell, NPC, or combat schemas own semantic validation. Confirm the scene schema and prompt remain unchanged: their enumerations are explicitly communicated and scene-plan construction has a distinct structural mapping boundary.

Open Questions

None. The stages above define the validation ownership, default ordering, compatibility policy, diagnostic behavior, and test boundaries required for implementation.