notarius/docs/roadmap/implementation.md

Scene-Aware Combat Extraction Implementation Plan

Status

Ready for implementation. Execute each stage in order, with one implementation prompt per stage.

Feature Contract

The target behavior and policy choices are defined in Scene-Aware Combat Extraction. That document is authoritative when this plan and the feature contract appear to overlap.

The central rule is strict opt-in: dnd/combat-turns may call the LLM only for an accepted chunk with one exact scene-description record whose kind is combat. Exact non-combat matches and missing or mismatched coverage produce an accepted empty combat-turn list without an LLM call.

Required Reading

Before Stage 1, read:

• docs/development.md;
• every document under docs/policy/;
• docs/roadmap/scene-aware-combat-extraction.md;
• docs/internal/pipeline.md;
• docs/internal/modules.md;
• docs/integrations/dnd-scene-description-artifacts.md;
• docs/integrations/dnd-combat-turn-artifacts.md; and
• the focused code and tests named in the applicable stage.

Before changing a subsystem in a later stage, reread its focused current documentation and tests if they have changed since the earlier stage.

Global Implementation Constraints

• Keep scene interpretation under internal/modules/dnd; generic framework and source packages must not import D&D types or interpret scene kinds.
• Use the existing ordered generated-reference mechanism. Do not add implicit lane discovery, a new dependency syntax, a general DAG, or direct output-file reads.
• Add scene_descriptions only to the combat extractor. The combat normalizer continues to accept only the optional npcs registry.
• Keep both durable v1 schemas unchanged.
• Do not add the scene artifact or a derived classification to the combat prompt. Do not change combat prompt assets, prompt ordering, prompt inputs, response schema, or prompt fingerprint.
• Preserve existing combat extraction behavior for eligible combat chunks, including NPC grounding, retries, validators, candidate mapping, and normalization.
• Treat the scene artifact as control context, never as combat evidence. Never copy its references, title, or summary into combat turns.
• Use deterministic, offline tests and existing LLM fakes. Do not call a live provider or turn probabilistic model output into a correctness gate.
• Test stable behavior at the narrowest owning boundary. Do not add tests for private helper shape, cache implementation details, or exact diagnostic prose.
• Do not add a framework abstraction unless implementation proves the existing reference contracts cannot express this feature. No such framework gap is currently expected.

Stage 1 — Add the Domain Scene-Eligibility Reference Boundary

Goal

Create one domain-owned, immutable boundary that decodes an approved scene-description reference, retains only gating data, and supports both construction-time external references and operation-time generated references. This stage must not change combat extraction behavior.

Implementation

1. Add internal/modules/dnd/scenedescriptions/registry, following the useful external/generated resolution pattern in internal/modules/dnd/npcs/registry without copying NPC-specific projection or identity policy.
2. Define these package contracts:
   • reference slot: scene_descriptions;
   • maximum item size: 1 MiB (1048576 bytes);
   • accepted representation: exactly one application/json item decoded by codec/scenedescriptions;
   • eligibility projection: scene ID, exact source_ref, and kind only.
3. Decode through the approved scene-description codec. Reject:
   • more than one item;
   • invalid or non-JSON media types;
   • oversized content;
   • invalid approved scene JSON; and
   • duplicate scene IDs, even if duplicate records are otherwise identical.
4. Do not retain titles, summaries, original bytes, paths, or prompt material in the resolved eligibility view. Framework reference provenance remains the owner of raw reference identity.
5. Build an immutable lookup keyed by exact scene ID. Expose a chunk-match result with three stable states:
   • exact: ID, source ID, start unit ID, and end unit ID all match;
   • missing: no record has the current chunk ID;
   • mismatched: the ID exists but any source-range field differs. Only the exact state exposes the scene kind.
6. Compute a semantic eligibility digest from a canonical projection sorted by scene ID. The digest must:
   • ignore title, summary, and input array order; and
   • change when an ID, source-range field, or kind changes. Represent an unbound view with the canonical empty projection {"scenes":[]} and its sha256: digest so generated-reference construction has a stable sentinel without pretending the required reference was bound.
7. Provide a resolver that:
   • seeds and validates an external materialized item during construction;
   • permits an unbound seed because a generated required reference is not populated until the operation-time handoff;
   • resolves an operation-time generated item when present;
   • otherwise returns the seeded view; and
   • safely reuses resolved immutable views across concurrent extract jobs.
8. Expose only the accessors required by combat extraction: bound state, record count, eligibility digest, and exact chunk matching. Return defensive values rather than mutable maps or scene records.

Tests

Add focused package tests that own:

• approved decoding and exact match classification;
• missing and mismatched match states;
• strict item count, media type, size, codec, and duplicate-ID failures;
• semantic digest stability across prose and ordering changes;
• semantic digest changes for each gating field;
• construction-time seed versus operation-time generated override; and
• immutability and concurrent-safe reuse at the public package boundary where practical.

Do not duplicate the scene codec's exhaustive JSON-shape tests.

Validation

Run:

go test ./internal/modules/dnd/scenedescriptions/registry
go test ./internal/modules/dnd/codec/scenedescriptions
git diff --check

Stage 1 is complete when the new package is independently usable and tested but no production module selects it yet.

Stage 2 — Enforce the Required Combat Scene Gate

Goal

Make dnd/combat-turns require the scene-description artifact and invoke its existing LLM path only for exact combat scenes. Update directly affected configuration fixtures and contract tests in the same stage so the repository does not retain knowingly invalid combat configurations.

Module Contract

1. In internal/modules/dnd/extract/combatturns, add aliases for the scene-registry slot and size limit and declare a reference slot with:
   • name scene_descriptions;
   • Required: true;
   • media type application/json;
   • accepted artifact kind dnd/scene-description-list;
   • maximum size 1 MiB; and
   • a description stating that it gates combat eligibility and is not combat evidence.
2. Preserve sorted reference-slot ordering and the existing agreement between ReferenceSlots() and ModuleSpec().ReferenceSlots.
3. Add a scene resolver to the immutable extractor. Constructor behavior must:
   • validate a materialized external scene artifact before execution;
   • allow the construction reference set to lack an item for a configured generated binding; and
   • retain no scene prose.
4. Define a stable gating policy identity dnd.combat_turns.scene_gate.v1.
5. Extend manifest metadata with scene_gate_policy. For a seeded external reference only, also report scene_eligibility_digest and scene_description_count. Do not publish those operation-time values for a generated reference in singleton component metadata; framework provenance owns that handoff.
6. Extend prepared-component checkpoint fingerprints with:
   • scene_gate_policy; and
   • scene_eligibility, using the seeded external eligibility digest or the registry's stable unbound sentinel for a generated binding. Keep the existing prompt, response-schema, mapping-policy, and NPC-registry fingerprints unchanged.

Extraction Behavior

In Extractor.Extract, retain common request validation through shared.PrepareChunkExtraction, then resolve and apply the scene gate before building prompt inputs or resolving NPC prompt grounding:

1. If the operation-time scene reference is absent, return a contextual module error stating that the required reference is missing. Normal configured runs should have failed earlier during resolution or required-producer handoff; this check protects direct module use.
2. For an exact combat match, continue through the existing NPC resolution, prompt construction, LLM request, response mapping, and validation path without semantic changes.
3. For an exact narrative, recap, or meta match, return dnd.CombatTurnList{CombatTurns: []dnd.CombatTurn{}}. Make no LLM call and emit no warning.
4. For a missing or mismatched match, return the same non-nil empty list without an LLM call and attach exactly one content-safe warning:
   • scope: scene_descriptions;
   • reason code: scene_classification_unavailable;
   • message: explain that no exact classification was available and combat extraction was skipped, without scene prose or transcript content.
5. The deterministic result is an accepted extract result, not an error or rejection. It must pass the existing combat shape, source-reference, schema, merge, and normalization boundaries and must not consume retry attempts.
6. Do not pass scene_descriptions into shared.PromptInputs or add it to the structured completion request. The current shared helper already selects only campaign and transcript inputs; preserve that boundary explicitly.

Required Test And Fixture Migration

1. Update combat extractor test helpers so tests of the existing LLM behavior supply an exact combat scene artifact for their current chunk. Keep NPC-specific test intent unchanged.
2. Add a compact table-driven extractor test covering:
   • exact combat;
   • exact narrative, recap, and meta;
   • missing scene ID;
   • matching ID with each mismatched range identity; and
   • an operation-time missing required reference. Assert typed output, warning reason where applicable, and whether the fake LLM was called. Do not assert private helper calls.
3. Update module specification, registration, metadata, and fingerprint tests for the required slot and new policy identities. Assert that changing only scene prose does not change the eligibility fingerprint, while changing kind or range does.
4. Update every in-repository pipeline or programmatic test configuration that selects dnd/combat-turns:
   • the maintained complete example must bind the earlier scene-descriptions lane to the later scene_descriptions step reference;
   • CLI combat contract configurations must bind a compatible external scene artifact when testing a standalone one-step profile;
   • the ordered D&D integration fixture must produce scene descriptions in its first step and bind them in its consumer step; and
   • fake clients used by those integration tests must return an exact combat scene where the test expects the combat LLM path.
5. Add or update a configuration-resolution assertion proving that a selected combat extractor without scene_descriptions is rejected as an unbound required slot.
6. Update existing expected lane counts, schemas, request sets, fingerprints, and provenance counts only where the new scene producer or consumer materially changes the contract. Preserve the original NPC-grounding assertions.

Validation

Run:

go run ./cmd/notarius config validate \
  --config examples/dnd-complete.config.yml \
  --pipeline dnd-session
go test ./internal/modules/dnd/...
go test ./internal/modules/integration
go test ./internal/cli
git diff --check

Stage 2 is complete when required-slot resolution is enforced, every combat extract path obeys the eligibility rule, all affected configurations are valid, and focused suites pass.

Stage 3 — Prove Ordered Handoff, Provenance, And Retry Behavior

Goal

Protect the cross-component behavior that cannot be established by the scene-registry and extractor package tests alone. Keep this stage lean: reuse the real codecs, resolver, pipeline, and validators, with a deterministic fake only at the LLM boundary.

Integration Coverage

1. Extend or refactor the focused ordered D&D integration test so it proves, in one representative assembled workflow:
   • scene descriptions and NPCs normalize in the first step;
   • their accepted artifacts are supplied to the intended second-step targets;
   • an exact combat scene causes exactly one combat completion request;
   • the combat request contains transcript, campaign, and NPC inputs as configured but no scene_descriptions input;
   • the combat output and existing NPC canonicalization remain correct;
   • the manifest records one generated scene-description reference for the combat extract target with the expected kind, schema, media type, digest, and producer identity; and
   • manifest and component metadata contain no scene title, summary, transcript text, or generated artifact payload.
2. Add one focused non-combat ordered-run case using an exact narrative classification. Configure combat retries to a value greater than zero, then prove:
   • no combat completion request occurs;
   • one accepted empty combat lane reaches normalized output;
   • no retry or rejection is recorded for the deterministic skip; and
   • no unavailable-classification warning is emitted for an exact non-combat match.
3. Add one focused missing-or-mismatched case at the narrowest stable boundary not already covered by Stage 2. Prove the accepted empty result and scene_classification_unavailable warning reach the runner's durable warning collection without a combat LLM call. Do not repeat every mismatch variant at integration scope.

Checkpoint And Identity Coverage

1. Confirm the existing framework-generated-reference dependency fingerprint already includes the canonical generated scene artifact. Reuse that mechanism; do not add a second framework fingerprint path.
2. Add focused D&D coverage proving:
   • a seeded external reference's component fingerprint changes when eligibility kind or exact range changes;
   • title-only or summary-only changes leave that semantic component fingerprint unchanged; and
   • the generated reference appears in the combat extract checkpoint dependencies, so changing the generated artifact prevents reuse.
3. Prefer an existing checkpoint identity or prepared-pipeline test boundary. Do not add a full CLI resume test if the same dependency invalidation is already credibly protected by the framework checkpoint tests plus the D&D prepared-pipeline assertion.

Validation

Run:

go test ./internal/modules/integration
go test ./internal/framework/pipeline
go test ./internal/cli
go test ./internal/modules/dnd/extract/combatturns
go test ./internal/modules/dnd/scenedescriptions/registry
git diff --check

Stage 3 is complete when the ordered generated handoff, no-call behavior, warning propagation, provenance privacy, and checkpoint invalidation are protected at their stable owning boundaries without redundant end-to-end cases.

Stage 4 — Update Canonical Documentation And Finish Validation

Goal

Move the implemented behavior out of future-only documentation and into each canonical current-behavior owner, then perform repository-wide verification.

Documentation

1. Update docs/config.md to own:
   • the combat extractor's required scene_descriptions slot;
   • accepted artifact kind, JSON media type, single-item and 1 MiB limits;
   • configuration-time rejection when the slot is unbound; and
   • the explicit generated binding used by the complete example. Update illustrative combat configuration fragments so none show an invalid unbound combat extractor.
2. Update docs/operations.md to explain that the first step produces NPC and scene-description artifacts and the second step runs combat extraction only for exact combat scenes.
3. Update docs/integrations/dnd-combat-turn-artifacts.md to own the externally observable eligibility, empty-result, warning, retry, prompt-exclusion, and provenance behavior.
4. Add a concise downstream-consumer link or summary to docs/integrations/dnd-scene-description-artifacts.md without duplicating the combat contract.
5. Update docs/internal/modules.md with the scene eligibility registry, construction-time versus operation-time resolution, exact matching, fingerprint, and content-retention boundaries. Keep generic ordered-handoff mechanics canonical in docs/internal/pipeline.md; update that document only if implementation changed or clarified a generic mechanism.
6. Update docs/internal/overview.md only if its component inventory requires the new domain package.
7. Remove the completed combat-gating bullet from docs/roadmap/future.md. Preserve the remaining scene-chunking evaluation and ordered-pipeline guidance.
8. Review docs/roadmap/scene-aware-combat-extraction.md for consistency with the final implementation. Keep it as feature history until a later explicit retirement request; do not duplicate its implementation sequence there.
9. Verify that README and CLI documentation need no changes. Update them only if an existing command or linked example became inaccurate.

Final Verification

Run all repository checks:

go run ./cmd/notarius config validate \
  --config examples/dnd-minimal.config.yml \
  --pipeline dnd-session
go run ./cmd/notarius config validate \
  --config examples/dnd-complete.config.yml \
  --pipeline dnd-session
go test ./...
go vet ./...
go build ./cmd/notarius
git diff --check

Also verify:

• only the two maintained D&D configuration examples remain;
• every current documentation link points to an existing file;
• no current-behavior document describes combat extraction without the required scene gate;
• no prompt asset or combat response schema changed; and
• no generated build artifact is left in the working tree.

Stage 4 is complete when documentation ownership is correct, future work no longer lists the implemented item, both examples validate, and all repository checks pass.

Open Questions

None. The feature contract and this plan resolve the required behavior, reference ownership, matching semantics, failure policy, warning policy, checkpoint identity, documentation ownership, and test boundaries.