eric/notarius
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Plan scene-aware combat extraction

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-25 18:59:57 +00:00
parent 9614469b45
commit 24238d249e
2 changed files with 601 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

412
docs/roadmap/implementation.md Normal file
View File

@@ -0,0 +1,412 @@
# Scene-Aware Combat Extraction Implementation Plan
## Status
Ready for implementation. Execute each stage in order, with one implementation
prompt per stage.
## Feature Contract
The target behavior and policy choices are defined in
[Scene-Aware Combat Extraction](scene-aware-combat-extraction.md). That document
is authoritative when this plan and the feature contract appear to overlap.
The central rule is strict opt-in: `dnd/combat-turns` may call the LLM only for
an accepted chunk with one exact scene-description record whose `kind` is
`combat`. Exact non-combat matches and missing or mismatched coverage produce an
accepted empty combat-turn list without an LLM call.
## Required Reading
Before Stage 1, read:
- `docs/development.md`;
- every document under `docs/policy/`;
- `docs/roadmap/scene-aware-combat-extraction.md`;
- `docs/internal/pipeline.md`;
- `docs/internal/modules.md`;
- `docs/integrations/dnd-scene-description-artifacts.md`;
- `docs/integrations/dnd-combat-turn-artifacts.md`; and
- the focused code and tests named in the applicable stage.
Before changing a subsystem in a later stage, reread its focused current
documentation and tests if they have changed since the earlier stage.
## Global Implementation Constraints
- Keep scene interpretation under `internal/modules/dnd`; generic framework and
source packages must not import D&D types or interpret scene kinds.
- Use the existing ordered generated-reference mechanism. Do not add implicit
lane discovery, a new dependency syntax, a general DAG, or direct output-file
reads.
- Add `scene_descriptions` only to the combat extractor. The combat normalizer
continues to accept only the optional `npcs` registry.
- Keep both durable v1 schemas unchanged.
- Do not add the scene artifact or a derived classification to the combat
prompt. Do not change combat prompt assets, prompt ordering, prompt inputs,
response schema, or prompt fingerprint.
- Preserve existing combat extraction behavior for eligible combat chunks,
including NPC grounding, retries, validators, candidate mapping, and
normalization.
- Treat the scene artifact as control context, never as combat evidence. Never
copy its references, title, or summary into combat turns.
- Use deterministic, offline tests and existing LLM fakes. Do not call a live
provider or turn probabilistic model output into a correctness gate.
- Test stable behavior at the narrowest owning boundary. Do not add tests for
private helper shape, cache implementation details, or exact diagnostic prose.
- Do not add a framework abstraction unless implementation proves the existing
reference contracts cannot express this feature. No such framework gap is
currently expected.
## Stage 1 — Add the Domain Scene-Eligibility Reference Boundary
### Goal
Create one domain-owned, immutable boundary that decodes an approved
scene-description reference, retains only gating data, and supports both
construction-time external references and operation-time generated references.
This stage must not change combat extraction behavior.
### Implementation
1. Add `internal/modules/dnd/scenedescriptions/registry`, following the useful
external/generated resolution pattern in `internal/modules/dnd/npcs/registry`
without copying NPC-specific projection or identity policy.
2. Define these package contracts:
- reference slot: `scene_descriptions`;
- maximum item size: 1 MiB (`1048576` bytes);
- accepted representation: exactly one `application/json` item decoded by
`codec/scenedescriptions`;
- eligibility projection: scene ID, exact `source_ref`, and `kind` only.
3. Decode through the approved scene-description codec. Reject:
- more than one item;
- invalid or non-JSON media types;
- oversized content;
- invalid approved scene JSON; and
- duplicate scene IDs, even if duplicate records are otherwise identical.
4. Do not retain titles, summaries, original bytes, paths, or prompt material in
the resolved eligibility view. Framework reference provenance remains the
owner of raw reference identity.
5. Build an immutable lookup keyed by exact scene ID. Expose a chunk-match result
with three stable states:
- exact: ID, source ID, start unit ID, and end unit ID all match;
- missing: no record has the current chunk ID;
- mismatched: the ID exists but any source-range field differs.
Only the exact state exposes the scene kind.
6. Compute a semantic eligibility digest from a canonical projection sorted by
scene ID. The digest must:
- ignore title, summary, and input array order; and
- change when an ID, source-range field, or kind changes.
Represent an unbound view with the canonical empty projection
`{"scenes":[]}` and its `sha256:` digest so generated-reference construction
has a stable sentinel without pretending the required reference was bound.
7. Provide a resolver that:
- seeds and validates an external materialized item during construction;
- permits an unbound seed because a generated required reference is not
populated until the operation-time handoff;
- resolves an operation-time generated item when present;
- otherwise returns the seeded view; and
- safely reuses resolved immutable views across concurrent extract jobs.
8. Expose only the accessors required by combat extraction: bound state, record
count, eligibility digest, and exact chunk matching. Return defensive values
rather than mutable maps or scene records.
### Tests
Add focused package tests that own:
- approved decoding and exact match classification;
- missing and mismatched match states;
- strict item count, media type, size, codec, and duplicate-ID failures;
- semantic digest stability across prose and ordering changes;
- semantic digest changes for each gating field;
- construction-time seed versus operation-time generated override; and
- immutability and concurrent-safe reuse at the public package boundary where
practical.
Do not duplicate the scene codec's exhaustive JSON-shape tests.
### Validation
Run:
```sh
go test ./internal/modules/dnd/scenedescriptions/registry
go test ./internal/modules/dnd/codec/scenedescriptions
git diff --check
```
Stage 1 is complete when the new package is independently usable and tested but
no production module selects it yet.
## Stage 2 — Enforce the Required Combat Scene Gate
### Goal
Make `dnd/combat-turns` require the scene-description artifact and invoke its
existing LLM path only for exact combat scenes. Update directly affected
configuration fixtures and contract tests in the same stage so the repository
does not retain knowingly invalid combat configurations.
### Module Contract
1. In `internal/modules/dnd/extract/combatturns`, add aliases for the
scene-registry slot and size limit and declare a reference slot with:
- name `scene_descriptions`;
- `Required: true`;
- media type `application/json`;
- accepted artifact kind `dnd/scene-description-list`;
- maximum size 1 MiB; and
- a description stating that it gates combat eligibility and is not combat
evidence.
2. Preserve sorted reference-slot ordering and the existing agreement between
`ReferenceSlots()` and `ModuleSpec().ReferenceSlots`.
3. Add a scene resolver to the immutable extractor. Constructor behavior must:
- validate a materialized external scene artifact before execution;
- allow the construction reference set to lack an item for a configured
generated binding; and
- retain no scene prose.
4. Define a stable gating policy identity
`dnd.combat_turns.scene_gate.v1`.
5. Extend manifest metadata with `scene_gate_policy`. For a seeded external
reference only, also report `scene_eligibility_digest` and
`scene_description_count`. Do not publish those operation-time values for a
generated reference in singleton component metadata; framework provenance
owns that handoff.
6. Extend prepared-component checkpoint fingerprints with:
- `scene_gate_policy`; and
- `scene_eligibility`, using the seeded external eligibility digest or the
registry's stable unbound sentinel for a generated binding.
Keep the existing prompt, response-schema, mapping-policy, and NPC-registry
fingerprints unchanged.
### Extraction Behavior
In `Extractor.Extract`, retain common request validation through
`shared.PrepareChunkExtraction`, then resolve and apply the scene gate before
building prompt inputs or resolving NPC prompt grounding:
1. If the operation-time scene reference is absent, return a contextual module
error stating that the required reference is missing. Normal configured runs
should have failed earlier during resolution or required-producer handoff;
this check protects direct module use.
2. For an exact `combat` match, continue through the existing NPC resolution,
prompt construction, LLM request, response mapping, and validation path
without semantic changes.
3. For an exact `narrative`, `recap`, or `meta` match, return
`dnd.CombatTurnList{CombatTurns: []dnd.CombatTurn{}}`. Make no LLM call and
emit no warning.
4. For a missing or mismatched match, return the same non-nil empty list without
an LLM call and attach exactly one content-safe warning:
- scope: `scene_descriptions`;
- reason code: `scene_classification_unavailable`;
- message: explain that no exact classification was available and combat
extraction was skipped, without scene prose or transcript content.
5. The deterministic result is an accepted extract result, not an error or
rejection. It must pass the existing combat shape, source-reference, schema,
merge, and normalization boundaries and must not consume retry attempts.
6. Do not pass `scene_descriptions` into `shared.PromptInputs` or add it to the
structured completion request. The current shared helper already selects
only campaign and transcript inputs; preserve that boundary explicitly.
### Required Test And Fixture Migration
1. Update combat extractor test helpers so tests of the existing LLM behavior
supply an exact combat scene artifact for their current chunk. Keep
NPC-specific test intent unchanged.
2. Add a compact table-driven extractor test covering:
- exact combat;
- exact narrative, recap, and meta;
- missing scene ID;
- matching ID with each mismatched range identity; and
- an operation-time missing required reference.
Assert typed output, warning reason where applicable, and whether the fake LLM
was called. Do not assert private helper calls.
3. Update module specification, registration, metadata, and fingerprint tests
for the required slot and new policy identities. Assert that changing only
scene prose does not change the eligibility fingerprint, while changing
kind or range does.
4. Update every in-repository pipeline or programmatic test configuration that
selects `dnd/combat-turns`:
- the maintained complete example must bind the earlier
`scene-descriptions` lane to the later `scene_descriptions` step reference;
- CLI combat contract configurations must bind a compatible external scene
artifact when testing a standalone one-step profile;
- the ordered D&D integration fixture must produce scene descriptions in its
first step and bind them in its consumer step; and
- fake clients used by those integration tests must return an exact combat
scene where the test expects the combat LLM path.
5. Add or update a configuration-resolution assertion proving that a selected
combat extractor without `scene_descriptions` is rejected as an unbound
required slot.
6. Update existing expected lane counts, schemas, request sets, fingerprints,
and provenance counts only where the new scene producer or consumer
materially changes the contract. Preserve the original NPC-grounding
assertions.
### Validation
Run:
```sh
go run ./cmd/notarius config validate \
--config examples/dnd-complete.config.yml \
--pipeline dnd-session
go test ./internal/modules/dnd/...
go test ./internal/modules/integration
go test ./internal/cli
git diff --check
```
Stage 2 is complete when required-slot resolution is enforced, every combat
extract path obeys the eligibility rule, all affected configurations are valid,
and focused suites pass.
## Stage 3 — Prove Ordered Handoff, Provenance, And Retry Behavior
### Goal
Protect the cross-component behavior that cannot be established by the
scene-registry and extractor package tests alone. Keep this stage lean: reuse
the real codecs, resolver, pipeline, and validators, with a deterministic fake
only at the LLM boundary.
### Integration Coverage
1. Extend or refactor the focused ordered D&D integration test so it proves, in
one representative assembled workflow:
- scene descriptions and NPCs normalize in the first step;
- their accepted artifacts are supplied to the intended second-step targets;
- an exact combat scene causes exactly one combat completion request;
- the combat request contains transcript, campaign, and NPC inputs as
configured but no `scene_descriptions` input;
- the combat output and existing NPC canonicalization remain correct;
- the manifest records one generated scene-description reference for the
combat extract target with the expected kind, schema, media type, digest,
and producer identity; and
- manifest and component metadata contain no scene title, summary, transcript
text, or generated artifact payload.
2. Add one focused non-combat ordered-run case using an exact `narrative`
classification. Configure combat retries to a value greater than zero, then
prove:
- no combat completion request occurs;
- one accepted empty combat lane reaches normalized output;
- no retry or rejection is recorded for the deterministic skip; and
- no unavailable-classification warning is emitted for an exact non-combat
match.
3. Add one focused missing-or-mismatched case at the narrowest stable boundary
not already covered by Stage 2. Prove the accepted empty result and
`scene_classification_unavailable` warning reach the runner's durable warning
collection without a combat LLM call. Do not repeat every mismatch variant
at integration scope.
### Checkpoint And Identity Coverage
1. Confirm the existing framework-generated-reference dependency fingerprint
already includes the canonical generated scene artifact. Reuse that
mechanism; do not add a second framework fingerprint path.
2. Add focused D&D coverage proving:
- a seeded external reference's component fingerprint changes when eligibility
kind or exact range changes;
- title-only or summary-only changes leave that semantic component
fingerprint unchanged; and
- the generated reference appears in the combat extract checkpoint
dependencies, so changing the generated artifact prevents reuse.
3. Prefer an existing checkpoint identity or prepared-pipeline test boundary.
Do not add a full CLI resume test if the same dependency invalidation is
already credibly protected by the framework checkpoint tests plus the D&D
prepared-pipeline assertion.
### Validation
Run:
```sh
go test ./internal/modules/integration
go test ./internal/framework/pipeline
go test ./internal/cli
go test ./internal/modules/dnd/extract/combatturns
go test ./internal/modules/dnd/scenedescriptions/registry
git diff --check
```
Stage 3 is complete when the ordered generated handoff, no-call behavior,
warning propagation, provenance privacy, and checkpoint invalidation are
protected at their stable owning boundaries without redundant end-to-end cases.
## Stage 4 — Update Canonical Documentation And Finish Validation
### Goal
Move the implemented behavior out of future-only documentation and into each
canonical current-behavior owner, then perform repository-wide verification.
### Documentation
1. Update `docs/config.md` to own:
- the combat extractor's required `scene_descriptions` slot;
- accepted artifact kind, JSON media type, single-item and 1 MiB limits;
- configuration-time rejection when the slot is unbound; and
- the explicit generated binding used by the complete example.
Update illustrative combat configuration fragments so none show an invalid
unbound combat extractor.
2. Update `docs/operations.md` to explain that the first step produces NPC and
scene-description artifacts and the second step runs combat extraction only
for exact combat scenes.
3. Update `docs/integrations/dnd-combat-turn-artifacts.md` to own the externally
observable eligibility, empty-result, warning, retry, prompt-exclusion, and
provenance behavior.
4. Add a concise downstream-consumer link or summary to
`docs/integrations/dnd-scene-description-artifacts.md` without duplicating
the combat contract.
5. Update `docs/internal/modules.md` with the scene eligibility registry,
construction-time versus operation-time resolution, exact matching,
fingerprint, and content-retention boundaries. Keep generic ordered-handoff
mechanics canonical in `docs/internal/pipeline.md`; update that document only
if implementation changed or clarified a generic mechanism.
6. Update `docs/internal/overview.md` only if its component inventory requires
the new domain package.
7. Remove the completed combat-gating bullet from
`docs/roadmap/future.md`. Preserve the remaining scene-chunking evaluation
and ordered-pipeline guidance.
8. Review `docs/roadmap/scene-aware-combat-extraction.md` for consistency with
the final implementation. Keep it as feature history until a later explicit
retirement request; do not duplicate its implementation sequence there.
9. Verify that README and CLI documentation need no changes. Update them only
if an existing command or linked example became inaccurate.
### Final Verification
Run all repository checks:
```sh
go run ./cmd/notarius config validate \
--config examples/dnd-minimal.config.yml \
--pipeline dnd-session
go run ./cmd/notarius config validate \
--config examples/dnd-complete.config.yml \
--pipeline dnd-session
go test ./...
go vet ./...
go build ./cmd/notarius
git diff --check
```
Also verify:
- only the two maintained D&D configuration examples remain;
- every current documentation link points to an existing file;
- no current-behavior document describes combat extraction without the required
scene gate;
- no prompt asset or combat response schema changed; and
- no generated build artifact is left in the working tree.
Stage 4 is complete when documentation ownership is correct, future work no
longer lists the implemented item, both examples validate, and all repository
checks pass.
## Open Questions
None. The feature contract and this plan resolve the required behavior,
reference ownership, matching semantics, failure policy, warning policy,
checkpoint identity, documentation ownership, and test boundaries.

189
docs/roadmap/scene-aware-combat-extraction.md Normal file
View File

@@ -0,0 +1,189 @@
# Scene-Aware Combat Extraction
## Status
Proposed near-term scope.
## Purpose
Use an accepted D&D scene-description artifact as the eligibility boundary for
combat-turn extraction. The combat LLM should run only for a chunk whose exact
scene record has `kind: combat`; every other chunk should be ignored by the
combat extractor. This work should connect the existing scene-description lane,
ordered generated-reference handoff, and combat-turn lane without expanding the
minimal combat-turn artifact contract or introducing D&D policy into generic
pipeline code.
## Target Workflow
The complete D&D pipeline has two ordered steps:
1. The first step extracts and normalizes NPCs and scene descriptions for the
accepted scene chunks.
2. At the step barrier, the accepted NPC and scene-description artifacts become
generated references.
3. The second step supplies the NPC artifact to its existing consumers and the
scene-description artifact to combat-turn extraction.
4. For each chunk, combat-turn extraction performs its existing LLM extraction
only for an exact `combat` classification. Every other chunk produces a
deterministic empty artifact without an LLM call.
The scene-description dependency must remain explicit in pipeline
configuration. The combat extractor must not discover another lane implicitly,
read output files directly, inspect chunker-private state, or require generic
chunk materialization to interpret D&D scene classifications.
## Scene-Description Reference Contract
The combat extractor should declare a required structured reference slot named
`scene_descriptions`. The slot accepts one approved
`dnd/scene-description-list` artifact using the existing durable scene
description schema.
The reference may be supplied as:
- a generated artifact from an earlier ordered step; or
- an external artifact through the existing reference-materialization
boundary.
External artifacts must be decoded and validated before source parsing or LLM
execution. Generated artifacts must cross the existing typed step-handoff
boundary and be validated before use. A bound artifact that is malformed,
incompatible, or internally inconsistent is an error; it must not be treated as
though the slot were unbound.
Pipeline resolution must reject combat extraction when this slot is not bound.
In an ordered same-run workflow, failure of the configured scene-description
producer to yield an accepted normalized artifact must fail the run before the
combat consumer step starts, consistent with existing required generated
reference semantics.
The prepared reference view should be immutable and safe for concurrent
extract jobs. Its metadata and checkpoint identity should be content-safe and
must not expose scene titles, summaries, paths, or source text.
## Chunk Matching And Gating Policy
A scene record authorizes combat extraction for the current chunk only when all
of the following are true:
- the scene ID exactly equals the current accepted chunk ID;
- the scene source ID exactly equals the chunk source ID;
- the scene start and end unit IDs exactly equal the chunk's inclusive source
range; and
- the approved artifact contains exactly one such record.
The normalized scene-description contract already rejects conflicting IDs and
ranges. The combat extractor must nevertheless require the exact match above at
its own decision boundary rather than relying on array position, range overlap,
title, summary, or inferred chronology.
An exact match with `kind: combat` performs the existing combat-turn LLM
extraction. An exact match with `narrative`, `recap`, or `meta` returns a typed
`dnd/combat-turn-list` containing an empty `combat_turns` array without making
an LLM call.
A valid artifact with no exact match, incomplete coverage, or a chunk identity
or range mismatch also returns the deterministic empty artifact. It must not
infer combat eligibility from an overlapping or adjacent scene. Emit a bounded,
content-safe warning for missing or mismatched coverage so operators can
distinguish an intentional non-combat classification from an unavailable exact
classification. The warning must not include scene prose or transcript text.
This policy relies on the scene contract's mixed-scene precedence: any chunk in
which combat is a substantive central activity is classified as `combat`.
Scene-aware gating must not add a second classification policy.
## Extraction, Validation, And Provenance
The deterministic empty result follows the same typed extractor and validator
boundaries as an LLM-produced empty result. It is not a rejection, does not
consume retry budget, and continues through merge and normalization normally.
The durable combat-turn schema remains unchanged.
Existing combat extraction behavior—including prompt assets, NPC grounding,
candidate mapping, validators, retries, warnings, and normalization—remains
unchanged for chunks classified as combat. The scene-description artifact is
control context only:
- it must not be added to the combat prompt;
- its title or summary must not become combat evidence;
- its source references must not be copied into combat turns; and
- it must not create, repair, or classify a combat turn.
Generated-reference provenance and dependency fingerprints should cover the
scene artifact through the existing ordered-handoff machinery. External
references should contribute their existing materialization provenance plus a
component-local semantic fingerprint sufficient to invalidate combat extract
checkpoints when a classification or chunk identity changes. Checkpoint reuse
must never preserve a skipped result after the effective scene classification
changes.
Run manifests may report bounded module metadata such as the number of approved
scene records. They must not contain scene prose or duplicate the referenced
artifact payload.
## Configuration And Documentation
The maintained complete D&D example should bind the normalized
`scene-descriptions` lane from the first step to the `scene_descriptions` slot
in the second step. The minimal example should remain unchanged.
When implemented, current-behavior documentation should be updated in its
canonical locations:
- Configuration owns the new selectable reference slot and binding example.
- Operations owns the ordered scene-aware workflow.
- The combat-turn integration contract owns externally observable extraction,
empty-result, and provenance behavior.
- Internal pipeline and module documentation own preparation, handoff, matching,
and checkpoint mechanics.
- `future.md` should remove the completed scene-aware combat item.
## Quality Expectations
Tests should protect behavior and architectural boundaries rather than internal
helper shape. Coverage should demonstrate:
- exact non-combat matches produce accepted empty combat artifacts without an
LLM call;
- exact combat matches retain the existing LLM path;
- an unbound required reference is rejected during pipeline resolution;
- missing and mismatched chunk coverage produces an accepted empty combat
artifact without an LLM call and emits a bounded warning;
- malformed external and generated artifacts fail at the appropriate
preparation or handoff boundary;
- retries are neither consumed nor attempted for deterministic skips;
- scene artifact changes invalidate relevant checkpoint reuse;
- no scene prose is exposed through combat prompts, warnings, metadata, or
manifests; and
- the complete maintained configuration resolves and materializes the intended
ordered dependency.
Model-output fixtures should remain deterministic test doubles. This feature
does not require live-provider tests or assertions over probabilistic model
quality.
## Non-Goals
This scope does not:
- change the durable scene-description or combat-turn schemas;
- add scene fields to combat-turn artifacts;
- annotate generic chunks with D&D classifications;
- make the combat lane depend implicitly on the scene-description lane;
- skip spell, NPC, NPC-interaction, or scene-description extraction;
- infer combat from scene titles, summaries, overlap, or campaign references;
- introduce arbitrary DAG scheduling or concurrent cross-step execution;
- add prior-run artifact discovery or new reference syntax; or
- add an LLM-backed validator, repair pass, or generic deduplication stage.
## Completion Criteria
The scope is complete when an explicitly configured ordered D&D pipeline can
hand an approved scene-description artifact to combat extraction, exact
`combat` scene matches are the only chunks that invoke the combat LLM, every
other chunk produces a deterministic empty result, provenance and checkpoint
identity remain correct, the complete example demonstrates the workflow, and
the canonical current-behavior documentation reflects the implemented
contract.