eric/notarius
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Plan simpler D&D extraction contracts

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-22 18:35:25 +00:00
parent ab0b4e350c
commit 14991cf58b
4 changed files with 665 additions and 135 deletions
Download Patch File Download Diff File Expand all files Collapse all files

469
docs/roadmap/implementation.md
View File

@@ -1,174 +1,373 @@
# D&D Validation-Boundary Alignment Implementation Plan
# Minimal D&D Extraction Contracts Implementation Plan
Status: Proposed.
**Status:** Ready for implementation
Implement this plan in order. It repairs the immediate combat extraction
failure, makes D&D validation diagnostics consistently domain-owned, and then
aligns the spell and NPC private response boundaries with the same policy.
## Objective
Do not change durable artifact schemas, artifact kinds, public Go types,
framework retry behavior, checkpoint formats, or the scene chunker. Existing
v1 private-schema identities may be corrected in place because Notarius has not
been run in production. Changed prompt and schema content will invalidate
development checkpoints through existing fingerprints.
Implement the durable contract redesign defined by
[Minimal D&D Extraction Contracts](minimal-dnd-extraction-contracts.md) and
[ADR-0009](../adr/0009-minimal-evidence-grounded-extraction-artifacts.md).
The result is a coordinated in-place redesign of NPC, spell-cast, and combat-turn
extraction. Each lane must emit only its narrow, evidence-grounded facts; no
removed rich-schema field may survive as an optional field, placeholder, compatibility
shim, or deterministic synthesis.
## Cross-Stage Decisions
This plan deliberately starts with NPCs because the NPC registry is a generated
reference consumed by spell and combat extraction. Spell and combat then change
independently, followed by one repository-wide contract and documentation
pass.
Private LLM schemas own the transport envelope: required fields, JSON types,
nullability, array/object shape, and unknown-field rejection. Deterministic
domain validators own semantic rules: supported enum values, nonblank values,
required non-empty collections, positive and resolvable source units,
catalog/identity policy, and normalized invariants.
## Fixed Decisions And Guardrails
For typed D&D artifacts, `generic/valid_json` remains first as a representation
sanity check. All deterministic domain validators that can reject a candidate
run next. `generic/valid_json_schema` runs after them as a durable-schema
backstop, followed by warning-only relatedness validators. This order gives
expected candidate failures bounded domain reason codes while retaining a final
check that typed encoding conforms to the durable contract.
The implementing agent must treat these as decisions, not open design choices:
Do not expose raw JSON Schema errors or candidate values through the generic
validator. Do not add tests that require particular words or phrases to remain
in prompt prose.
- Keep the artifact kinds `dnd/npc-list`, `dnd/spell-list`, and
`dnd/combat-turn-list`, their module keys and capabilities, their media types,
and their prompt IDs stable.
- Retain exactly the existing v1 schema keys, IDs, names, versions, filenames,
prompt IDs, and prompt versions listed in the feature roadmap. Change their
unpublished shapes and content in place; do not harmonize the private spell
identity as part of this work.
- Do not add v2 assets, runtime version negotiation, migration code, dual-write
behavior, a second module registration, or compatibility fixtures for the
superseded pre-release shapes. Existing fixtures should be rewritten or
deleted according to whether they still protect current behavior.
- Require the top-level list field and every record field. A list may be empty,
but every returned record must have at least one source reference. Use strict
JSON objects with unknown fields rejected and no nullable or optional legacy
fields.
- Keep private source references limited to `start_unit_id` and `end_unit_id`.
Mapping assigns the current source document ID; campaign references and an
NPC registry never become event evidence.
- Keep the shared prompt ordering policy: stable shared instructions first,
stable campaign and generated references next, module task material after
those references, and the chunk-variable transcript last.
- Preserve the existing production validator-chain order and the rule that a
configured chain is authoritative. Remove or simplify validators; do not
silently reorder chains.
- Preserve the existing NPC ID derivation algorithm and comparison/display
normalization for canonical names. Retain existing semantic-policy
identifiers as well as schema and prompt versions; pre-redesign development
state is disposable. Unchanged canonical names must not receive new IDs.
- Use deterministic code only for display normalization, known catalog or NPC
canonicalization, application-owned IDs, evidence canonicalization, ordering,
and exact duplicate collapse. Do not introduce fuzzy matching or inferred
enrichment.
- Follow the testing policy: protect schemas and behavior at their owning
boundaries, delete obsolete tests, and avoid tests that snapshot prompt prose
or detect exact shared-prefix length. No live or paid model call is a stage
completion requirement.
- Each stage must leave the repository compiling and `go test ./...` passing.
Update all affected fixtures and callers within the stage that changes a
public Go type; do not leave an intentionally broken intermediate commit.
## Stage 1: Repair Combat Extraction
## Stage 1: Cut Over The NPC Contract And Name Projection
### Changes
This stage establishes the redesigned generated reference on which later stages
depend.
- Update the combat extraction instructions to enumerate the complete allowed
values:
- `turn_kind`: `turn`, `reaction`, `legendary_action`, `lair_action`,
`other`;
- action `category`: `attack`, `spell`, `movement`, `item`,
`ability_check`, `saving_throw`, `condition`, `other`.
- Keep the private combat schema structurally permissive and the durable schema
strict. Do not restore enum, minimum, `minLength`, or `minItems` constraints
to the private schema.
- Reorder the combat extraction default chain to:
`generic/valid_json`, combat shape, combat source references,
`generic/valid_json_schema`, combat source relatedness.
- Reorder the combat normalization default chain to:
`generic/valid_json`, combat shape, normalized invariants, combat source
references, `generic/valid_json_schema`, combat source relatedness.
- Preserve configured validator overrides as authoritative; only production
default composition changes.
### 1.1 Replace the durable and private data shapes
### Tests
- Reduce `dnd.NPC` in `internal/modules/dnd/types.go` to exactly `ID`, `Name`,
and `SourceRefs`. Delete `NPCRelationship` and every alias, description, and
relationship field or helper that becomes unused.
- Rewrite the existing strict durable schema `dnd_npcs.v1.json` with required
top-level `npcs`, and records containing only required `id`, `name`, and
`source_refs`. Keep the NPC codec at schema version `v1`, name
`notarius_dnd_npcs_v1`, and its existing schema ID and media type.
- Rewrite the strict private schema `dnd_npcs_llm.v1.json`. Its records contain
only required `name` and model-facing source ranges. Keep the extractor schema
identity and prompt ID/version unchanged.
- Reduce the private response DTO and mapping accordingly. Preserve candidate
data at the mapping boundary so deterministic validators, rather than mapping
defaults, reject blank names or invalid ranges. Assign the current source ID
and derive the NPC ID in application code.
- Rewrite the NPC task and instruction assets to ask only for individually
identifiable NPC names and supporting transcript ranges. Explicitly exclude
anonymous groups, generic roles, invented labels, descriptions, aliases, and
relationships. Keep transcript material last in the manifest.
- Add an assembled combat pipeline case whose raw LLM response contains an
unsupported turn kind or action category. Exhausted retries must produce a
non-fatal `invalid_combat_turn_shape` rejection owned by the combat shape
validator, not `json_schema_invalid` or a framework error.
- Retain coverage that a later valid retry succeeds and discarded-attempt
warnings/rejections do not become durable.
- Update registrar contract tests to assert the new extraction and
normalization order.
- Rely on behavioral enum/schema tests and prompt fingerprint coverage; do not
add prompt-word change-detector tests.
### 1.2 Simplify NPC identity, normalization, and validation
### Completion Check
- Keep the identity policy at `dnd.npcs.identity.v1`. Retain display
normalization, comparison keys, ID syntax, and deterministic ID derivation.
Validate nonblank canonical names, exact ID/name agreement, duplicate
canonical names, and duplicate IDs. Delete alias validation and alias-specific
issue locations/codes.
- Keep NPC normalization at `dnd.npcs.normalize.v1`. Normalize the retained
display name, derive its ID, canonicalize and deduplicate source references,
consolidate records only by the canonical-name comparison key, preserve the
first stable record, and union exact evidence. Remove alias promotion,
relationship merge/rewrites, and their warning codes.
- Keep the existing NPC shape and source-relatedness policy identifiers. The
shape validator checks only list presence, nonblank `id` and `name`, and
nonempty source references; source-relatedness grounds a record only through
its retained name. The existing source-ref validator remains the owner of
range validity.
- Keep the default validation order domain-first, then JSON Schema, then the
advisory relatedness validator. Simplify only the validator implementations
and selections whose owned behavior changed.
Run `go test ./internal/modules/dnd/extract/combatturns
./internal/modules/dnd/validate/combatturns/... ./internal/modules/dnd/register
./internal/modules/integration` and `git diff --check`.
### 1.3 Separate durable registry provenance from model input
## Stage 2: Make D&D Validator Ordering Consistent
- Continue to canonicalize and retain the complete redesigned NPC artifact for
registry validation, cache identity, manifests, and provenance.
`Registry.Digest()` remains the digest of that complete canonical artifact
when bound and remains empty when no registry was supplied.
- Build a second structural JSON projection for model grounding with the exact
shape `{"npcs":[{"name":"Mira Thorn"}]}` in normalized registry order. It
contains names only: no IDs, source references, origin URI, aliases, or other
provenance. Generate it with typed values and `json.Marshal`, not string
concatenation. The empty projection is exactly `{"npcs":[]}`.
- Add an explicit projection digest accessor. The digest is SHA-256 over the
exact projected bytes, including for an absent or empty registry, and
`PromptInput().Digest` must equal it. Keep full-artifact and projection digests
distinct even when their content happens to coincide.
- Index `Registry.Lookup` by canonical name only. Retain immutable return values
and defensive copies. Remove alias indexing and alias-aware comments.
- In spell extraction, combat extraction, and combat normalization, use the NPC
name-projection digest for the component-local checkpoint fingerprint because
it exactly describes the names that affect those operations. Keep the full
registry digest and count in manifest metadata for provenance. Framework-owned
generated-reference fingerprints may still invalidate a run when any upstream
artifact byte changes; do not broaden this stage into a framework cache
redesign.
### Changes
### 1.4 Update owners, consumers, and tests
- Move `generic/valid_json_schema` behind all rejecting domain validators in
every spell and NPC extraction and normalization default chain:
- spell extraction/normalization: shape, catalog, source references, schema,
then source relatedness;
- NPC extraction: shape, source references, schema, then source relatedness;
- NPC normalization: shape, identity, source references, schema, then source
relatedness.
- Keep `generic/valid_json` first and warning-only source relatedness last.
- Do not change validator implementations, reason codes, warning promotion,
retry counts, or user-provided chain order.
- Document the default-chain policy in the pipeline/module internals: domain
validators diagnose expected semantic failures and the generic schema
validator is the final rejecting representation backstop.
- Update NPC codec, schema, extractor, identity, validator, normalizer, registry,
registration, pipeline-integration, and CLI fixtures to the new shape. Adapt
spell and combat tests that construct `dnd.NPC` values so the repository
remains buildable, but do not change their own artifact contracts yet.
- Add or rewrite focused tests for strict schema acceptance/rejection,
codec round trips, unchanged ID derivation for known names, name-only
consolidation, evidence union, registry immutability, and canonical-name-only
lookup. Rewrite the existing fixture for the minimal current contract; do not
retain the superseded rich fixture solely to test backwards incompatibility.
- Test the projection as a data contract: it contains only ordered names,
equivalent normalized registries produce identical bytes and digest,
evidence/ID-only changes do not change the projection digest, and name/order
changes do. This is not authorization to snapshot assembled prompt prose or
prefix lengths.
- Update `docs/integrations/dnd-npc-artifacts.md` to own the redesigned v1
durable schema. Update `docs/internal/modules.md` and
`docs/internal/llm.md` only for current NPC behavior that lands in this stage.
### Tests
### Stage 1 completion criteria
- Update production registrar tests for every affected chain.
- Add one representative spell and NPC assembled rejection proving that a
domain-invalid but encodable candidate is attributed to the owning domain
validator rather than the generic schema validator. Do not duplicate each
validator package's existing case matrix at integration level.
- Confirm explicitly configured validator chains retain their exact configured
order.
- NPC durable and private schemas expose no removed enrichment fields.
- The generated NPC prompt input contains names only while manifests retain
full registry provenance.
- Spell and combat consumers accept generated or external registries in the
redesigned v1 shape; rich pre-redesign registry JSON fails strict decoding.
- Focused NPC, spell-wiring, combat-wiring, integration, and CLI tests pass, and
`go test ./...` passes.
### Completion Check
## Stage 2: Cut Over Spell-Cast Extraction
Run `go test ./internal/modules/dnd/register ./internal/modules/integration
./internal/framework/pipeline` and `git diff --check`.
### 2.1 Replace the spell contract and prompt
## Stage 3: Align Spell and NPC Private Response Boundaries
- Reduce `dnd.SpellCast` to exactly `Caster`, `Spell`, and `SourceRefs`.
- Rewrite `dnd_spells.v1.json` in place. Keep the durable codec at version v1,
name `notarius_dnd_spells_v1`, and its existing schema ID. Its strict record
contains only required `caster`, `spell`, and `source_refs`.
- Rewrite `dnd_spells_llm.v1.json` with the same logical fields and model-facing
ranges. Keep the existing private key, ID, name, prompt version, and schema
path unchanged.
- Reduce the private DTO, canonicalization, and mapping to the retained fields.
Continue assigning current source IDs, stable-ordering candidates by evidence,
and preserving semantically invalid candidates for deterministic validation.
- Rewrite spell task and instruction assets around the narrow casting-event
boundary. Retain the spell catalog and name-only NPC projection as
disambiguation inputs, never evidence. Remove effect, outcome, and narrative
duties and remove the `common-dnd-immediate-resolution.md` message from the
spell manifest and spell asset registration. Do not delete the shared file in
this stage because combat still uses it.
### Changes
### 2.2 Simplify spell policies
- Revise the existing v1 spell and NPC private schemas in place:
- retain required fields, JSON types, array/object structure,
`additionalProperties: false`, and omission of framework-assigned fields;
- remove `minLength`, `minItems`, and positive-number `minimum` constraints;
- leave the durable spell and NPC schemas unchanged.
- Replace `shared.UnitRef` in the private spell and NPC response DTOs with
integer candidates so zero and negative unit IDs survive decoding and mapping
into `source.SourceRef` for deterministic source validation.
- Update canonicalization and ordering helpers to operate on candidate integers
without repairing invalid values. Valid positive IDs retain current output,
ordering, and exact-deduplication behavior; invalid ranges remain available
to validators.
- Keep malformed JSON, missing/unknown fields, wrong JSON types, and
non-integer source IDs as LLM-boundary errors.
- Update LLM/module internals and the spell/NPC integration contracts to state
the structural-private/semantic-validator ownership boundary.
- Keep the spell shape policy identifier unchanged and validate only list
presence, nonblank caster/spell, and nonempty source references.
- Keep catalog validation, source-reference validation, and source-relatedness
behavior and policy identities unchanged where their actual semantics are
already limited to spell name, caster, and evidence.
- Simplify normalization and duplicate comparison to caster, catalog-canonical
spell name, and the complete valid evidence set. Remove all prose selection,
copying, fixture fields, and assertions. Preserve catalog fingerprints and
metadata.
- Keep the exact inclusion rule from the feature roadmap: an actual casting or
unambiguous declared attempt is included; mentions, plans, rules discussion,
and catalog matches are not.
### Tests
### 2.3 Update owners and tests
- For each private schema, prove structurally valid candidates with blank
strings, empty required collections, and nonpositive unit IDs pass the
private schema, while missing fields, unknown fields, and wrong JSON types do
not.
- Through raw-JSON LLM fakes, prove semantic values survive decoding and mapping
without repair.
- Add representative assembled cases showing:
- blank or empty spell/NPC fields are rejected by the appropriate shape
validator;
- nonpositive or nonexistent unit IDs are rejected by the appropriate source
validator; and
- exhausted validation retries remain non-fatal rejected outputs.
- Preserve existing valid mapping, source-position ordering, deduplication,
catalog, identity, checkpoint-fingerprint, and durable codec tests.
- Update spell codec, private schema, extractor, validators, normalizer, merge
and registration tests, pipeline integration, CLI output fixtures, and any
maintained examples to the minimal shape. Rewrite or delete rich-schema
fixtures rather than retaining them as compatibility cases.
- Replace tests of effects and narrative descriptions with focused tests of the
retained contract: strict unknown-field rejection, codec round trips, current
source-ID assignment, catalog canonicalization, evidence ordering, duplicate
collapse, and NPC name-projection wiring.
- Update `docs/integrations/dnd-spell-artifacts.md` as the canonical redesigned
v1 contract and update current internal module/LLM documentation for the
smaller prompt and response. Do not duplicate the spell catalog contract
owned by its existing integration document.
### Completion Check
### Stage 2 completion criteria
Run `go test ./internal/modules/dnd/extract/spells
./internal/modules/dnd/extract/npcs ./internal/modules/dnd/validate/spells/...
./internal/modules/dnd/validate/npcs/... ./internal/modules/integration` and
`git diff --check`.
- No production spell type, schema, prompt, validator, normalizer, fixture, or
documentation contract refers to effect or narrative description.
- Spell prompt/schema identities remain exactly their existing v1 values.
- Focused spell and pipeline tests pass, and `go test ./...` passes.
## Final Verification
## Stage 3: Cut Over Combat-Turn Extraction
Run:
### 3.1 Replace the combat contract and prompt
```text
git diff --check
- Reduce `dnd.CombatTurn` to exactly `Actor`, `TurnKind`, and `SourceRefs`.
Delete `CombatAction`, `CombatActionCategory`, their constants, and all
now-unused helpers. Retain the existing five `CombatTurnKind` values.
- Rewrite `dnd_combat_turns.v1.json` in place. Keep the durable codec at version
v1, name `notarius_dnd_combat_turns_v1`, and its existing schema ID. Use a
strict required record with `actor`, `turn_kind`, and `source_refs` only.
- Simplify the durable codec to direct strict encoding/decoding if its custom
wire representation exists only to distinguish removed nullable fields.
Preserve presence semantics for the top-level list and strict unknown-field
rejection.
- Rewrite `dnd_combat_turns_llm.v1.json` while retaining its private schema and
prompt identities, and reduce the DTO/mapping to actor, turn kind, and
model-facing ranges. Keep the semantic mapping policy identifier unchanged.
- Rewrite combat task/instruction assets around detecting ordered turns and
discrete interrupting events. Remove round, action, target, declaration,
resolution, outcome, and summary duties. Remove the shared immediate-
resolution message from the combat manifest.
- Once both spell and combat manifests no longer use it, delete
`common-dnd-immediate-resolution.md` and its shared and module asset
registrations. Retain all other shared evidence, identity, reference, NPC,
and transcript assets in their cache-friendly order.
### 3.2 Simplify combat normalization and validation
- Keep combat normalization at `dnd.combat_turns.normalize.v1`. Continue to
display-normalize and registry-canonicalize actors, canonicalize evidence,
order records chronologically, and collapse exact duplicates by actor, turn
kind, and complete valid evidence. Delete action/target/prose normalization
and warning codes.
- Keep the invariant policy identifier unchanged. It checks canonical actor
display, canonical and chronological evidence, stable event ordering, and
absence of duplicate event identities; it performs no nested-action checks.
- Keep the shape policy identifier unchanged and validate only list presence,
nonblank actor, allowed turn kind, and nonempty source references.
- Keep the source-relatedness policy identifier unchanged and compare only the
actor against cited transcript material. Remove declaration-token heuristics
and their now-unused helpers. Keep source-reference validation unchanged if
its semantics did not change.
- Simplify merge/clone behavior to copy only retained values and source refs.
Preserve stable ordering and defensive ownership.
### 3.3 Update owners and tests
- Update codec, private schema, extractor, normalizer, validators, merge,
registration, pipeline integration, CLI fixtures, and examples to the minimal
shape. Rewrite or delete rich-schema fixtures rather than retaining them as
compatibility cases.
- Delete tests whose sole policy was round/action/summary handling. Add or
rewrite focused tests for strict schemas, enum validation, mapping and
source-ID assignment, actor canonicalization through the redesigned NPC
registry, chronology, exact duplicate collapse, and invariant validation.
- Update `docs/integrations/dnd-combat-turn-artifacts.md` as the canonical
redesigned v1 contract and update current internal module/LLM documentation
for the implemented behavior.
### Stage 3 completion criteria
- No production combat type, prompt, schema, policy, normalizer, fixture, or
current documentation refers to rounds, actions, summaries, declarations,
targets, resolutions, or action categories.
- The immediate-resolution shared asset has no remaining registration or file.
- Focused combat and pipeline tests pass, and `go test ./...` passes.
## Stage 4: Complete The Repository-Wide Cutover
### 4.1 Audit contract identity and stale surface area
- Search code, embedded assets, tests, examples, and current documentation for
all removed field names and unintended v2 schema/prompt names. Removed fields
may remain only in historical ADR or roadmap context; active schema, prompt,
fixture, and current-behavior surfaces must describe the minimal v1 contract.
- Verify the three artifact registrations still bind their original kinds and
exact redesigned Go types through extract, merge, normalize, codec, and
validators.
- Verify all prompt manifests still point at their v1 private schemas, stable
reference material precedes module-variable material, and transcript content
remains last. Do not add a change-detector test for prompt message count,
prose, shared-prefix content, or prefix length.
- Verify pre-redesign development checkpoints naturally miss through changed
prompt/schema content and generated-dependency fingerprints. Do not add
migration or compatibility handling for disposable pre-release state.
### 4.2 Exercise representative assembled behavior
- Keep unit case matrices at their owning schema, validator, normalizer, and
registry boundaries. At the assembled-pipeline boundary, retain only
representative tests proving each redesigned lane is registered, strict
JSON/schema failures are attributed to the correct validator, semantic
failures are attributed to the correct domain validator, and generated NPC
output is accepted by later spell/combat stages.
- Cover an ordered multi-step run in which NPC extraction produces the minimal
artifact and spell/combat consume its name projection. Assert that downstream
event source references point only to the current transcript and not to NPC
registry evidence.
- Verify CLI logical output and manifests retain the v1 schema/prompt identities,
full NPC registry provenance, and the relevant component fingerprints without
exposing prompt bodies or reference payloads.
### 4.3 Finish documentation and lifecycle state
- Reconcile the three integration documents, `docs/internal/modules.md`, and
`docs/internal/llm.md` with the final code. Correct any stale validator-order
descriptions while doing so; current docs must describe the registered order,
not preserve an older generic ordering example.
- Mark ADR-0009 `Accepted` and the feature roadmap `Implemented` when the code,
tests, and current-behavior documentation all land. Remove the completed item
from `docs/roadmap/future.md`; retain the feature roadmap and ADR as design
rationale unless the repository's normal roadmap-retirement practice calls
for moving the completed roadmap later.
- Record any human-reviewed rich/minimal model evaluation separately from CI
results. Evaluation may motivate later prompt tuning but does not reopen the
approved durable minimal field set within this implementation.
### 4.4 Final verification
Run, in order:
```sh
gofmt -w <changed Go files>
go test ./...
go vet ./...
go build ./cmd/notarius
go test -race ./internal/modules/dnd/... ./internal/framework/pipeline ./internal/cli ./internal/modules/integration
go test -race ./internal/modules/dnd/...
git diff --check
```
Review current-behavior documentation for stale statements that private spell,
NPC, or combat schemas own semantic validation. Confirm the scene schema and
prompt remain unchanged: their enumerations are explicitly communicated and
scene-plan construction has a distinct structural mapping boundary.
If the repository-wide race command exposes an unrelated, pre-existing failure,
document it with the narrower affected package result; do not weaken or skip
ordinary tests for the changed D&D packages.
### Stage 4 completion criteria
- All three lanes use only their minimal v1 contracts from model response
through durable output.
- No compatibility shim, removed-field policy, unintended v2 asset, or stale
current-behavior documentation remains.
- Generated NPC references are name-only for LLM input and remain full-fidelity
for durable provenance.
- Repository tests, vet, build, race checks for the changed domain, and diff
hygiene checks pass.
## Open Questions
None. The stages above define the validation ownership, default ordering,
compatibility policy, diagnostic behavior, and test boundaries required for
implementation.
None. The feature roadmap, ADR, and fixed decisions above define the cutover,
pre-release schema, evidence, projection, validation, testing, and documentation
policies needed to implement each stage without further product decisions.