Audit the D&D modules

2026-07-24 13:43:53 +00:00
parent 8adcf6840d
commit 6dc9d522b1
2 changed files with 1191 additions and 0 deletions
--- a/docs/roadmap/implementation.md
+++ b/docs/roadmap/implementation.md
@@ -0,0 +1,425 @@
+# D&D Extraction Module Audit Execution Plan
+
+Status: Ready for audit execution
+
+This document divides the audit defined in
+[D&D Extraction Module Refactoring Audit Strategy](audit.md) into five prompts.
+Execute the stages in order. Each stage is a read-only code audit: it may update
+`docs/roadmap/audit.md`, but it must not change production code, tests, prompts,
+schemas, examples, current-behavior documentation, or configuration.
+
+The word "implementation" in this filename refers to implementing the audit
+strategy, not implementing refactors.
+
+## Global Instructions
+
+Every stage must:
+
+1. Read `docs/roadmap/audit.md` in full before beginning.
+2. Follow `AGENTS.md`, `docs/development.md`, and all policies under
+   `docs/policy/`.
+3. Prefer the codebase knowledge graph for code discovery and call tracing.
+   Use direct file and text searches for prompt assets, JSON Schemas,
+   configuration, documentation, and exact string comparison.
+4. Inspect all five extractors where the stage calls for comparison:
+   `spells`, `npcs`, `combatturns`, `npcinteractions`, and
+   `scenedescriptions`.
+5. Cite exact files and symbols for every observation written to `audit.md`.
+6. Distinguish confirmed facts, provisional interpretations, intentional
+   specialization, and actionable findings.
+7. Avoid treating visual similarity, file count, or line count as sufficient
+   evidence for sharing.
+8. Preserve findings from earlier stages unless new evidence disproves them.
+   When revising an earlier conclusion, edit it in place and record the reason;
+   do not append contradictory conclusions.
+9. Keep implementation out of scope. Do not modify code to test whether a
+   proposed refactor is convenient.
+10. Leave the repository otherwise unchanged and report any pre-existing dirty
+    worktree state before proceeding.
+
+## Working Results Structure
+
+Stages 1 through 4 maintain one working area at the end of `audit.md`:
+
+```markdown
+## Audit Results
+
+Status: In progress
+
+### Baseline And Module Matrix
+### Divergence Register
+### Prompt, Schema, And LLM Review
+### Extraction And Canonicalization Review
+### Duplication Register
+### Contextual Architecture And Ownership Review
+### Candidate Decisions
+```
+
+Create headings when their owning stage begins. Later stages may refine earlier
+sections but must not duplicate them. Use compact tables where they improve
+five-way comparison. Keep detailed evidence in the relevant review section and
+keep the divergence and duplication registers concise.
+
+Each divergence-register entry must have:
+
+- a stable identifier such as `D-01`;
+- affected modules;
+- observed difference;
+- provisional classification from the audit strategy;
+- evidence links;
+- impact or reason it may be harmless; and
+- status: open, confirmed, intentional, rejected, or superseded.
+
+Each duplication-register entry must have:
+
+- a stable identifier such as `R-01`;
+- participating modules;
+- repeated responsibility;
+- exact, structural, or policy-duplication classification;
+- meaningful differences;
+- candidate owner, if any;
+- evidence links; and
+- provisional outcome: extract, harmonize without sharing, retain separately,
+  or defer.
+
+Do not assign final severity until Stage 5 has traced ownership and evaluated
+impact.
+
+## Stage 1: Inventory And Convention Matrix
+
+### Objective
+
+Establish the complete, factual five-module baseline and identify convention
+differences without yet recommending shared abstractions.
+
+### Required review
+
+For every extractor, inspect:
+
+- production and test file inventory;
+- module key, artifact kind, capabilities, and execution class;
+- constructor inputs and retained prepared state;
+- option decoding and unknown-option behavior;
+- `ModuleSpec`, registration builder, and reference slots;
+- manifest metadata and checkpoint fingerprint providers;
+- exported versus package-private surface;
+- embedded asset registration;
+- principal extractor entry point and result type; and
+- package-local test organization.
+
+Trace registration into production composition far enough to confirm that the
+declared contract is the one actually selected. Inspect neighboring packages
+only as required to verify an identity or ownership fact.
+
+### Write to `audit.md`
+
+Create `## Audit Results`, mark it `Status: In progress`, and add:
+
+1. `### Baseline And Module Matrix`
+   - one row per module;
+   - columns for every comparison dimension in the audit strategy;
+   - concise facts with links rather than judgments; and
+   - an explicit note where a dimension is not applicable.
+2. `### Divergence Register`
+   - record every observed organizational, naming, construction, registration,
+     provenance, or test-layout difference;
+   - classify only as required specialization, permitted variation, convention
+     drift, architectural divergence, or undetermined; and
+   - do not propose helper extraction in this stage.
+
+Also add a short baseline-validation note recording:
+
+```sh
+go test -count=1 ./...
+go vet ./...
+go build ./cmd/notarius
+gofmt -l .
+git diff --check
+```
+
+If a command fails, record the exact command, affected package, and concise
+failure classification. Do not fix it during the audit.
+
+### Completion gate
+
+Stage 1 is complete when every matrix cell is populated or marked not
+applicable, every observed baseline divergence has a stable register entry, and
+the repository's starting validation state is recorded.
+
+## Stage 2: Prompts, Schemas, And LLM Boundaries
+
+### Objective
+
+Determine whether the five modules consistently present stable context,
+references, lane instructions, transcripts, and structured-output contracts to
+the LLM, and identify exact or near-duplicate assets without changing them.
+
+### Required review
+
+For every extractor:
+
+- read the complete prompt manifest and every referenced local and shared
+  asset;
+- record the exact ordered message sequence, role, input, and cache-control
+  boundary;
+- compare shared message files by identity and bytes, not by paraphrased
+  meaning;
+- verify stable-to-variable ordering against `docs/internal/llm.md`;
+- map declared prompt inputs to module reference slots and generated inputs;
+- trace optional, required, empty, and generated reference projections;
+- inspect prompt and schema registration, hashing, and diagnostic redaction;
+- inspect the complete private response schema for identity, required fields,
+  nullability, strict objects, and semantic constraints;
+- map schema fields to private DTOs and response mapping; and
+- compare prompt and schema tests at their behavioral boundaries.
+
+Treat the scene-description whole-chunk evidence model as a specialization to
+explain, not a presumption of drift. Likewise, treat catalog and NPC registry
+inputs as subset-specific responsibilities unless evidence shows inconsistent
+handling of the same contract.
+
+### Write to `audit.md`
+
+Add `### Prompt, Schema, And LLM Review` containing:
+
+- a five-way prompt-order and cache-boundary table;
+- a prompt-input and reference-projection table;
+- a private-schema and DTO ownership table;
+- exact shared-asset usage;
+- local assets with identical or near-identical content;
+- content-safety and diagnostic observations; and
+- evidence-backed deviations from documented LLM conventions.
+
+Update the divergence register for confirmed or newly discovered prompt,
+schema, reference, provenance, and testing differences.
+
+Create `### Duplication Register` and add prompt-, schema-, metadata-, and
+asset-related candidates. For each prompt candidate, state whether the text is
+byte-identical, merely similar, or semantically different. Do not recommend a
+shared prompt asset unless all intended consumers should receive future edits
+atomically.
+
+### Completion gate
+
+Stage 2 is complete when every prompt message and input is accounted for, every
+private schema field has an owner, cache-prefix claims are based on exact
+message identity, and every asset-sharing candidate has a provisional keep or
+share outcome.
+
+## Stage 3: Extraction, Evidence, And Canonicalization
+
+### Objective
+
+Compare runtime extraction flow and identify repeated algorithms or policy
+without erasing artifact-specific semantics.
+
+### Required review
+
+For every extractor, inspect and trace:
+
+- request, context, source, chunk, and dependency validation;
+- construction-time state versus operation-time overrides;
+- structured request assembly and provider-error wrapping;
+- response-to-artifact mapping;
+- source identity attachment and unit-ID resolution;
+- evidence range validation assumptions;
+- canonical source-reference ordering and exact deduplication;
+- artifact ordering and deterministic tie-breakers;
+- enum or canonical-name handling;
+- warnings and diagnostics;
+- cloning and aliasing boundaries; and
+- focused extractor, model, canonicalization, and malformed-output tests.
+
+Compare complete algorithms rather than function names alone. For similar
+canonicalization helpers, identify which parts are:
+
+- identical source-reference mechanics;
+- artifact-specific mapping;
+- artifact-specific ordering policy; or
+- validation that belongs to a later validator rather than extraction.
+
+Use call traces and complexity data to inspect repeated scans, allocations,
+serialization, or high-cognitive-complexity paths. Report performance only when
+a plausible workload and complexity impact exist.
+
+### Write to `audit.md`
+
+Add `### Extraction And Canonicalization Review` containing:
+
+- a five-way extraction-flow table;
+- an evidence and ordering-policy table;
+- clone, mutation, error, and warning observations;
+- test-ownership comparisons; and
+- code-quality or performance candidates with concrete impact.
+
+Update the divergence register, revising earlier provisional classifications
+where runtime evidence explains or contradicts them.
+
+Expand the duplication register with request-validation, reference,
+canonicalization, ordering, deduplication, mapping, error, and test-support
+candidates. Give each candidate a provisional owner and explicitly identify the
+artifact-specific code that must remain local.
+
+### Completion gate
+
+Stage 3 is complete when the full extraction path of all five modules is
+accounted for, every similar canonicalization path has been decomposed into
+shared mechanics versus domain policy, and every runtime duplication candidate
+has a provisional ownership decision.
+
+## Stage 4: Contextual Architecture And Ownership Review
+
+### Objective
+
+Validate candidate findings against the surrounding D&D and framework
+architecture, and decide which apparent similarities should actually be shared.
+
+### Required review
+
+For every open divergence and duplication candidate:
+
+- trace callers and consumers;
+- inspect existing facilities in `internal/modules/dnd/shared`;
+- inspect focused D&D registry, identity, catalog, codec, normalize, and
+  validate packages as relevant;
+- verify the typed artifact and reference contracts;
+- inspect production registration, default validator composition, and
+  checkpoint fingerprint assembly;
+- verify documentation ownership and current durable contracts; and
+- check whether a proposed generic helper has a genuine domain-neutral owner.
+
+Apply the shared-code hierarchy from `audit.md`:
+
+1. module-owned artifact semantics;
+2. D&D-wide shared mechanics;
+3. focused subset-specific D&D packages; and
+4. framework-owned domain-neutral behavior.
+
+Reject or defer candidates whose API would require artifact-specific callbacks,
+type erasure, module-key branching, a broad configuration object, or speculative
+future consumers.
+
+### Write to `audit.md`
+
+Add `### Contextual Architecture And Ownership Review` containing:
+
+- traced ownership evidence for every open candidate;
+- dependency-direction and layer-boundary conclusions;
+- checkpoint, provenance, and reference compatibility conclusions;
+- documentation or test ownership implications; and
+- any product-contract questions that cannot be decided as refactors.
+
+Add `### Candidate Decisions`, with one row per divergence and duplication
+identifier. Choose exactly one outcome:
+
+- extract now;
+- harmonize without sharing;
+- retain intentionally separate;
+- reject as harmful abstraction; or
+- defer pending a named missing requirement or product decision.
+
+For extract or harmonize outcomes, specify:
+
+- target owner;
+- minimal responsibility and proposed API shape;
+- participating modules;
+- behavior that remains package-owned;
+- migration order;
+- relevant tests; and
+- principal risks.
+
+Update every register entry to confirmed, intentional, rejected, superseded, or
+explicitly deferred. No entry may remain merely open at the end of this stage.
+
+### Completion gate
+
+Stage 4 is complete when all candidates have traced ownership and a final
+keep/share/harmonize/defer decision, no proposed helper violates dependency
+direction, and all required product decisions are separated from executable
+refactoring recommendations.
+
+## Stage 5: Synthesis And Final Audit
+
+### Objective
+
+Turn the working evidence into one concise, internally consistent audit that
+can support roadmap decisions and a later implementation plan.
+
+### Required work
+
+1. Re-read the complete strategy and all working audit results.
+2. Recheck every cited file and symbol against the current working tree.
+3. Reconcile duplicate, overlapping, or contradictory observations.
+4. Assign severity only to confirmed actionable findings:
+   - **high:** correctness, security, data integrity, or architectural failure
+     with substantial impact;
+   - **medium:** meaningful drift, duplication, or design weakness likely to
+     cause defects or costly divergence;
+   - **low:** localized maintainability, clarity, test-quality, or
+     documentation issue with limited immediate impact.
+5. Keep optional improvements separate from findings.
+6. Confirm intentional specializations and rejected sharing candidates are
+   documented so future work does not repeatedly reopen them without evidence.
+7. Run the repository-wide validation commands from Stage 1 again and record
+   the final result. Do not modify code in response to failures.
+
+### Rewrite `audit.md`
+
+Preserve the strategy sections above `## Audit Results`, but replace the
+provisional working area with this final structure:
+
+```markdown
+## Audit Results
+
+Status: Complete
+
+### Executive Conclusion
+### Final Module Comparison Matrix
+### Prioritized Findings
+### Intentional Differences To Preserve
+### Shared-Code Decisions
+### Rejected Or Deferred Candidates
+### Recommended Refactoring Sequence
+### Validation And Residual Risks
+```
+
+Requirements for the final sections:
+
+- `Executive Conclusion` directly answers the user's three audit questions.
+- `Final Module Comparison Matrix` remains factual and compact.
+- `Prioritized Findings` follows the finding standard in `audit.md`, is ordered
+  by severity and impact, and cites exact evidence.
+- `Intentional Differences To Preserve` explains why harmonization would be
+  incorrect.
+- `Shared-Code Decisions` records extract and harmonize recommendations,
+  proposed ownership, and why sharing is better than continued separation.
+- `Rejected Or Deferred Candidates` records superficially attractive
+  abstractions and why they should not be pursued now.
+- `Recommended Refactoring Sequence` groups accepted recommendations into
+  independently safe, dependency-ordered scopes. It is a sequence, not a
+  decision-complete implementation plan.
+- `Validation And Residual Risks` records commands, results, limits of the
+  static audit, and any human or model-quality evaluation still needed.
+
+Remove the working divergence and duplication registers after their evidence
+has been incorporated into the final sections. Do not leave raw stage notes,
+provisional severities, or superseded conclusions in the completed audit.
+
+### Completion gate
+
+Stage 5 is complete when:
+
+- the final audit satisfies every deliverable and completion criterion in
+  `audit.md`;
+- the three user questions receive explicit answers;
+- every recommendation has evidence, ownership, scope, and rationale;
+- no finding relies only on similarity or stylistic preference;
+- validation results are current;
+- no code or current-behavior files changed; and
+- `git status --short` shows only the intended `audit.md` audit-result changes
+  plus any pre-existing user changes.
+
+## Open Questions
+
+None. The audit stages, working-document structure, decision rules, and final
+deliverable are fully specified.