diff --git a/docs/releases/v0.6.0.md b/docs/releases/v0.6.0.md new file mode 100644 index 00000000..c80a0e82 --- /dev/null +++ b/docs/releases/v0.6.0.md @@ -0,0 +1,84 @@ +# Notarius v0.6.0 + +This release improves the reliability and ergonomics of unattended, +subprocess-driven D&D extraction pipelines. + +## Summary + +Notarius now gives models more precise, semantically useful correction guidance +when deterministic or model-backed validation rejects an otherwise structured +candidate. Semantic reconciliation retries identify candidates with compact, +request-local numbers, preserve valid candidates when a proposal cannot be +repaired, and report bounded process warnings when reconciliation is +incomplete. D&D extraction also canonicalizes safely resolvable reversed source +ranges and applies more consistent schema, prompt, and validator constraints, +improving successful operation with smaller models without weakening evidence +validation. + +An optional LLM-backed combat-semantics validator can review whether scene +descriptions correctly consolidate combat and apply the `combat` kind. For +subprocess callers, command-line references now follow the pipeline-scoped +configuration model: a shared reference can be supplied once and automatically +reach every compatible selected target, while lane- and binding-specific forms +remain available for exceptions. + +## Compatibility + +- The meaning of an unqualified `--reference slot=path` or + `--without-reference slot` selector has changed. It now applies to every + selected pipeline target that declares the slot instead of requiring exactly + one matching target. Review callers that relied on ambiguity rejection or a + unique implicit target; the current selector contract is documented in the + [CLI reference](../cli.md). +- The stage-wide `merge.slot` CLI reference shorthand has been removed. Use + `lane.merge.slot` for an exact merge binding, `lane.slot` for all compatible + bindings in one lane, or an unqualified `slot` for pipeline scope. +- Command-line file references cannot replace, remove, or coexist with a + generated artifact handoff for the same concrete target and slot. Resolution + reports the conflict so the caller can narrow or remove the selector. +- The private LLM-facing semantic-reconciliation contract now uses contiguous + `candidate_number` values rather than application identities. This is not a + durable or operator-configurable contract and requires no operator action. +- Durable D&D artifact schemas, the `notarius.run-result.v2` receipt, + `notarius.warnings.v2`, and `notarius.diagnostics.v1` are unchanged from + `v0.5.0`. + +## Upgrade + +1. Update subprocess commands to provide shared inputs once with + `--reference slot=path`, and convert any stage-wide `merge.slot` selector to + an explicit supported scope. The [subprocess consumer guide](../consumers/subprocess.md) + and [complete D&D consumer guide](../consumers/dnd-pipeline.md) show the + current invocation pattern. +2. Review CLI reference overrides that overlap generated same-run references; + narrow or remove an external selector rather than attempting to replace the + generated handoff. +3. Run `notarius config validate --config --pipeline ` for every + deployed pipeline configuration. +4. Optionally add + `extract/dnd/scene-descriptions/combat_semantics` to the scene-description + validator chain where the deployment wants LLM-backed combat-scene review; + see the [configuration reference](../config.md). +5. Perform a representative run with the deployed model profile and inspect + the machine-readable receipt, warnings, and diagnostics before promotion. + +## Changes + +- Added feedback-aware, module-requested retry support for normalizers while + keeping retry budgets bounded and preserving attempt diagnostics. +- Reworked semantic reconciliation around compact, contiguous candidate + numbers, explicit namespace guidance, complete proposal validation, and + model-facing retry feedback that omits opaque internal identities. +- Preserved candidates separately when an exhausted semantic proposal cannot + be safely applied, with a bounded process warning describing the fallback. +- Canonicalized safely resolvable reversed D&D source-reference endpoints + before deterministic coverage validation and clarified the shared evidence + prompt. +- Improved D&D schemas, prompts, and validators with closed-value constraints, + contextual correction guidance, shared source-range coverage logic, and + clearer item-holder transition rules. +- Added the optional D&D scene-description combat-semantics validator, shared + combat policy assets, evaluation fixtures, and retry-path coverage. +- Made unqualified CLI references pipeline-scoped, added hierarchical lane and + exact-binding selectors, defined deterministic override precedence, and + documented first-class subprocess use. diff --git a/docs/roadmap/cli-pipeline-references.md b/docs/roadmap/cli-pipeline-references.md deleted file mode 100644 index 1510fda1..00000000 --- a/docs/roadmap/cli-pipeline-references.md +++ /dev/null @@ -1,258 +0,0 @@ -# Pipeline-Scoped CLI References - -## Purpose - -Make command-line reference binding match the pipeline-level mental model used -by configuration and by subprocess callers. A caller should be able to supply -each shared external reference once, while retaining explicit syntax for the -less common case in which one lane or one stage binding needs a different -source. - -## Motivation And Current Problem - -Notarius already accepts repeatable `--reference selector=path` and -`--without-reference selector` flags. The current unqualified form, -`--reference slot=path`, succeeds only when exactly one selected chunk, extract, -merge, or normalize target declares that slot. If several targets consume a -shared reference such as `party`, `glossary`, or `spell_catalog`, the command is -rejected as ambiguous and the caller must repeat stage-qualified bindings. - -That behavior is internally precise but does not match the public configuration -model. A reference declared at pipeline scope is shared with every compatible -target, while target-local configuration provides the exceptional override. -It is therefore surprising for the least-qualified CLI syntax to mean "find one -unique target" rather than "supply this reference to the pipeline." - -The mismatch is especially costly for subprocess use. The primary complete D&D -workflow expects an orchestrator to provide one transcript, one output root, -and a small collection of campaign reference files. Requiring the orchestrator -to know and enumerate every internal consumer of those shared files couples it -to lane composition, makes commands needlessly long, and creates maintenance -work whenever another compatible D&D module is added. - -## Target CLI Contract - -`--reference` remains repeatable, but selector qualification expresses scope: - -| Form | Target scope | -| --- | --- | -| `slot=path` | Every selected pipeline target that declares `slot`. | -| `chunk.slot=path` | The selected chunk binding. | -| `lane.slot=path` | Every extract, merge, or normalize binding in `lane` that declares `slot`. | -| `lane.extract.slot=path` | The extract binding in `lane`. | -| `lane.merge.slot=path` | The merge binding in `lane`. | -| `lane.normalize.slot=path` | The normalize binding in `lane`. | - -The current stage-wide `merge.slot=path` shorthand is removed. A merge-specific -override must name its lane as `lane.merge.slot=path`; this keeps the grammar -hierarchical and avoids another uniqueness-dependent selector. - -`--without-reference` uses the same selector forms without `=path`. It removes -matching external bindings only. It never removes a generated artifact handoff, -and resolution continues to reject a missing required reference. - -Broad selectors are expected to match multiple compatible targets. They fail -when they match no selected target, when the named lane is not selected, or -when a stage-qualified target does not declare the slot. Errors should identify -the selector and relevant scope without requiring callers to understand private -resolver structures. - -## Precedence And Conflict Policy - -Command-line references are operational overrides and take precedence over -external file references supplied at pipeline, step, lane, or binding scope in -configuration. Among CLI selectors that affect the same concrete target: - -1. an exact `lane.stage.slot` or `chunk.slot` selector wins over a lane-scoped - selector; -2. a lane-scoped selector wins over a pipeline-scoped selector; and -3. the last occurrence wins among selectors with equal scope and action. - -This ordering lets an orchestrator provide shared defaults once and express -only genuine exceptions: - -```sh ---reference party=/refs/party.txt \ ---reference npc-registry.party=/refs/npc-party-context.txt -``` - -Binding and unbinding the same concrete target at the same specificity is a -configuration error rather than an argument-order-dependent result. A -more-specific unbind may carve an exception out of a broader binding, and a -more-specific binding may restore an exception to a broader unbind. - -Generated artifact references remain a distinct source form. A CLI file -reference must not silently replace, remove, or coexist with a generated -handoff for the same concrete target and slot. Resolution fails with a -target-specific conflict and directs the caller to narrow or remove the CLI -selector. Same-run generated registry and eligibility handoffs in the complete -D&D pipeline therefore remain controlled by pipeline composition. - -CLI reference paths continue to resolve relative to the process working -directory and retain CLI provenance. Subprocess guidance must recommend -absolute paths. The reference source is auxiliary context and does not affect -the generated prompt session identifier. - -## Subprocess Target State - -Passing shared references is a first-class part of the documented subprocess -workflow. The complete D&D invocation should have this shape: - -```sh -notarius run dnd-session \ - --config /absolute/path/to/notarius.yml \ - --input /absolute/path/to/transcripts/final.trimmed.json \ - --output-dir /absolute/path/to/notarius-output \ - --reference party=/absolute/path/to/references/party.txt \ - --reference players=/absolute/path/to/references/players.txt \ - --reference glossary=/absolute/path/to/references/glossary.txt \ - --reference spell_catalog=/absolute/path/to/references/spells.json \ - --json -``` - -Only references actually available to and desired by the deployment need to be -passed. The selected modules continue to determine accepted slot names, media -types, requiredness, and size limits. The orchestrator passes references as -separate argument-vector elements rather than building a shell command, captures -standard output and standard error separately, checks exit status, and uses the -JSON receipt's `output_directory` to locate the generated run bundle. - -The documentation must clearly distinguish: - -- the input file supplied by `--input`; -- the output *root* supplied by `--output-dir` and the run-specific output path - returned in the receipt; -- external file references supplied by `--reference`; and -- generated references produced and consumed within the configured ordered - pipeline. - -## Required Implementation Changes - -### CLI selector model and parsing - -- Replace the unique-target interpretation of unqualified and lane-qualified - selectors with explicit pipeline, lane, chunk, and exact-binding scopes. -- Remove parsing and help text for the stage-wide `merge.slot` form. -- Preserve repeatable flag handling, non-empty selector/path checks, supported - stage names, and syntax-error exit classification. -- Represent selector scope explicitly enough that precedence and diagnostics do - not depend on inferring intent from empty fields. - -### Target expansion and precedence - -- Refactor CLI reference resolution to return every compatible selected target - for a broad selector rather than requiring uniqueness. -- Resolve overlapping bind and unbind requests into one deterministic action - per concrete target and slot using the precedence policy above. -- Produce exact `pipeline.ReferenceBinding` and `pipeline.ReferenceUnbind` - values after CLI scoping is resolved. Keep broad-selector policy at the CLI - boundary rather than adding CLI grammar or D&D knowledge to the generic - pipeline framework. -- Preserve CLI provenance and the existing rule that command-line paths are - materialized relative to the working directory. -- Detect generated-reference conflicts before file access or stage execution - and retain required-slot validation after effective bindings are known. -- Ensure lane selection is honored for compact pipelines. Explicit ordered - pipelines continue to use their complete configured lane set because they do - not support `--only`. - -### Public and internal documentation - -- Rewrite the reference-selector section of `docs/cli.md`, which is the - canonical owner of flag syntax, selector semantics, precedence, and command - errors. -- Update `docs/consumers/subprocess.md` so reference-bearing invocation is part - of the primary subprocess workflow rather than an unillustrated aside. -- Update `docs/consumers/dnd-pipeline.md` with the complete D&D subprocess - invocation and explain which references are external versus generated. -- Lightly update `docs/config.md` to link the configuration hierarchy to the - CLI override contract without duplicating CLI syntax. -- Update `docs/internal/cli.md` and, only where necessary, the reference - resolution discussion in `docs/internal/pipeline.md` to describe the - implemented expansion and precedence boundary. -- Do not expand the README quickstart or duplicate complete configuration - files in prose. The maintained example configuration remains the canonical - copyable pipeline definition. - -No ADR is required. This is a deliberate public CLI usability correction that -fits the existing architectural decisions: configuration remains centralized, -operational overrides remain explicit, generated handoffs remain part of -pipeline composition, and broad CLI syntax is translated into existing exact -framework bindings at the application boundary. The CLI and configuration -references are the durable owners of the resulting current behavior once it is -implemented. - -## Testing And Validation - -Add lean offline behavioral coverage at the CLI contract boundary for: - -- one pipeline-scoped reference reaching multiple compatible chunk and lane - targets; -- one lane-scoped reference reaching both extract and normalize bindings; -- exact-binding and lane-scoped exceptions overriding broader values; -- final-occurrence behavior at equal specificity; -- broad selectors with no matches and selectors naming unselected lanes or - undeclared slots; -- bind/unbind specificity and same-specificity conflicts; -- rejection of external/generated conflicts and protection of required - generated handoffs; -- working-directory resolution and CLI provenance for expanded bindings; and -- removal of the `merge.slot` shorthand. - -Prefer package-level resolution and representative `RunWithOptions` tests over -duplicating the full selector matrix through end-to-end fixtures. Existing -tests that encode unique-target ambiguity should be rewritten or removed rather -than retained as historical change detectors. Do not assert complete error -strings when stable classification and a concise semantic fragment provide -sufficient confidence. - -Validation for the completed change should include: - -```sh -go test ./internal/cli ./internal/core/config ./internal/framework/pipeline -go test ./... -go vet ./... -go build ./cmd/notarius -``` - -Also validate both maintained example configurations against their selected -pipelines, verify every changed documentation link, and manually review the -D&D subprocess command against the actual flag parser and complete example -configuration. - -## Non-Goals - -- Passing reference contents directly in command-line arguments or environment - variables. -- Adding a reference-manifest file, glob syntax, wildcard selectors, or a - second pipeline-reference flag. -- Allowing the CLI to define undeclared reference slots or bypass module media - type, size, UTF-8, requiredness, or materialization validation. -- Replacing generated same-run artifacts with external files or changing - ordered pipeline dependencies. -- Changing configuration-file reference path resolution, prompt-session - derivation, output directory allocation, or the run-result receipt. -- Adding D&D-specific selector behavior to generic CLI or pipeline packages. -- Preserving uniqueness-dependent `slot` or `lane.slot` behavior as an alias. - -## Acceptance Criteria - -- A subprocess caller can supply `party`, `players`, `glossary`, and - `spell_catalog` once each and have every compatible selected D&D target - receive the corresponding external file. -- `slot=path` has pipeline scope and never fails merely because several selected - targets declare the slot. -- Lane and exact-binding selectors provide deterministic, documented - exceptions with specificity-based precedence. -- A broad selector that matches nothing is rejected before source parsing or - LLM work. -- CLI external references override configured external paths but never silently - replace or remove generated artifact references. -- All effective CLI bindings retain correct provenance and absolute-path - materialization behavior. -- `docs/cli.md` owns the exact public syntax, and both subprocess guides present - shared CLI references as the primary orchestration workflow. -- Current behavior outside `docs/roadmap/` is not documented until the code - lands, and no complete example is duplicated outside `examples/`. -- Focused tests, repository-wide tests, vetting, build, maintained configuration - validation, and documentation-link review pass. diff --git a/docs/roadmap/dnd-validation-reliability.md b/docs/roadmap/dnd-validation-reliability.md deleted file mode 100644 index 668d510f..00000000 --- a/docs/roadmap/dnd-validation-reliability.md +++ /dev/null @@ -1,240 +0,0 @@ -# D&D Validation Reliability Hardening - -## Purpose - -Improve the correctness and efficiency of D&D validation retries by giving the -producer complete, contextual, semantically useful correction guidance; moving -simple closed-value checks into compatible LLM-facing schemas; and removing -duplicated source-range containment logic. The work should make one retry more -likely to repair every detectable defect without weakening deterministic -validation or exposing internal identifiers to a model. - -## Current State And Findings - -The validation architecture is coherent and the major safety boundaries are in -place. Validators run through the framework-managed retry protocol, model-facing -guidance is distinct from reason codes and operator diagnostics, schemas use -provider-compatible required fields, and D&D extraction adapters canonically -order safely resolvable source-reference endpoints before validation. The -review nevertheless found four opportunities to make that architecture more -consistent and effective. - -### Generic correction guidance - -Most rejecting D&D validators produce detailed operator-facing messages but -pair them with a fixed, general correction sentence. The framework correctly -constructs retry text from `CorrectionGuidance` alone, so the model does not see -the internal diagnostic—and therefore often does not learn which records were -wrong or how each record must change. Item-occurrence shape validation and the -combat-semantics validator already demonstrate the intended contextual pattern. - -Passing the operator message through unchanged is not an acceptable fix. -Operator messages may contain Go paths, array indexes that do not reliably map -back to canonicalized model output, reason codes, validator terminology, or -opaque application identities. Those details are useful for diagnosis but are -not appropriate model context. - -### Missing closed-value schema constraints - -Four private LLM response schemas describe categorical fields as unconstrained -strings even though their prompts, durable artifact schemas, and deterministic -validators define closed value sets: - -- `combat-turns`: `turn_kind`; -- `enemy-events`: `kind`; -- `npc-occurrences`: `kind`; and -- `scene-descriptions`: `kind`. - -This defers an inexpensive structural check until the outer semantic-validation -loop. Item and location occurrences already use the preferable private-schema -enum pattern. - -### First-defect validation - -Spell shape validation returns after the first invalid cast field, and enemy -engagement validation returns after the first duplicated engagement subject. -When multiple defects exist, each retry can therefore reveal only one of them. -That needlessly consumes the bounded producer-attempt budget and makes -correction less reliable for smaller models. - -### Duplicated source-range containment - -Nine extraction source-reference validators independently determine whether a -citation belongs to the current chunk. Five validate every source unit spanned -by a reference, while four check only the endpoints. Those behaviors are -equivalent for today's contiguous materialized chunks, but the duplicated -implementations and different apparent contracts create a drift risk. -Scene-description validation has a separate exact-one-scene range contract and -should remain specialized. - -## Target End State - -- Every producer-correctable D&D rejection supplies bounded contextual guidance - that identifies all affected records using transcript-grounded names, source - ranges, and response fields as appropriate, explains each semantic defect, - and requests one complete corrected replacement. -- Model-facing guidance never contains reason codes, validator keys, Go field - paths, opaque or hash-derived entity IDs, or raw internal error text. - Operator diagnostics remain detailed and separately available. -- Every detectable issue in a candidate is collected in one validator pass, - subject to the repository's established diagnostic bounds. A retry is not - spent merely to reveal the next defect. -- Closed categorical fields are constrained by compatible enums in the private - LLM schemas. Deterministic typed validators retain the same rules as - defense-in-depth and as protection for non-LLM producers and later stages. -- One D&D-owned helper implements full-span chunk containment for extraction - evidence. All applicable source-reference validators use it and no - module-local endpoint-only or coverage implementation remains. -- Validator results remain immutable, deterministic, bounded, and suitable for - the existing feedback-aware replacement-request protocol. Retry budgets, - warning policy, durable artifact schemas, and accepted output semantics do - not change. - -## Required Work - -### Contextual correction guidance - -Audit each rejecting D&D validator according to the stage that can actually -correct its result: - -- Extraction validators should build model-facing issue descriptions from the - candidate and its transcript context. Shape, registry-membership, identity, - catalog, engagement, and source-reference validators should name the - contextual artifact and cited range where useful, state the invalid value or - relationship in plain language, and state the required replacement shape. -- LLM-backed NPC, item, and location registry reconciliation should continue to - translate its typed proposal issues through the shared category-to-prose - renderer, including any domain-specific supplement. -- Normalization-only deterministic invariant validators should retain useful - operator diagnostics. They should not claim that a producer can directly - repair deterministically derived ordering, identity, or normalization state. - If such a rejection can reach a feedback-capable producer, guidance must be - expressed only in terms of the source candidate that producer controls. -- LLM-backed semantic validators may continue using their model explanation - when it is bounded and semantically meaningful. - -Introduce a small D&D-shared diagnostic utility only for demonstrated common -mechanics such as stable grouping, de-duplication, contextual source-range -rendering, and bounded correction aggregation. Domain validators must continue -to own the meaning and prose of their rules. The generic pipeline must not gain -D&D knowledge, and operator `Message` values must never be mechanically copied -into `CorrectionGuidance`. - -Update validator policy fingerprints wherever correction behavior changes so -checkpoints created under generic feedback are not reused as though the policy -were identical. - -### Private-schema enum guardrails - -Add the existing supported value sets to the four private LLM schemas: - -- `assets/dnd/combat-turns/schemas/dnd_combat_turns_llm.v1.json`; -- `assets/dnd/enemy-events/schemas/dnd_enemy_events_llm.v1.json`; -- `assets/dnd/npc-occurrences/schemas/dnd_npc_occurrences_llm.v1.json`; and -- `assets/dnd/scene-descriptions/schemas/dnd_scene_descriptions_llm.v1.json`. - -Use the exact values owned by the corresponding durable contract and typed -domain constants. Keep all object properties required, retain current nullable -types where present, and do not introduce `uniqueItems`, optional properties, -or conditional schema logic. Revise schema tests that currently accept unknown -values, and rely on computed asset fingerprints to invalidate incompatible -LLM-output checkpoints. Keep deterministic enum validation in place. - -### Complete per-attempt issue collection - -Refactor spell shape and enemy engagement validation to inspect the complete -candidate and collect every detectable violation before returning. Build the -operator diagnostic and contextual correction request from the same evaluated -issue set while preserving their different audiences. De-duplicate repeated -semantic instructions, retain enough contextual identification for every -affected record, and use the established bounded diagnostic behavior rather -than an unbounded error string. - -Review the surrounding D&D validators while applying the contextual-guidance -change. Remove any additional accidental early exits that prevent independent -candidate defects from being reported together, but retain immediate returns -for request-level prerequisites whose absence makes further inspection unsafe -or meaningless. - -### Shared full-span chunk containment - -Add one helper under `internal/modules/dnd/shared` that determines whether a -source reference's complete document-ordered span is contained by a chunk. The -helper must: - -- validate source identity; -- resolve endpoints through `source.DocumentIndex` rather than numeric-ID - assumptions; -- require every unit in the inclusive span to be present in the chunk; -- handle nil or unresolved inputs without panic; -- avoid mutating the source, chunk, or reference; and -- leave source-reference validity and error wording to the consuming validator. - -Use the helper from the spell, NPC-registry, NPC-occurrence, item-registry, -item-occurrence, location-registry, location-occurrence, combat-turn, and -enemy-event source-reference validators. Keep the scene-description exact-range -validator separate because it enforces a materially different contract. - -## Testing And Documentation - -Add lean offline behavioral coverage at the narrowest stable boundary: - -- correction tests should prove that multiple contextual defects produce one - actionable, bounded request and that internal identifiers and diagnostic - syntax are absent; they should not snapshot exact prose or message length; -- schema tests should prove rejection of representative unsupported categorical - values and acceptance of the supported sets without duplicating every - provider behavior; -- spell-shape and engagement tests should prove that independent defects are - reported together; -- the shared containment helper should own the full case matrix, including - partial spans, non-monotonic unit IDs, wrong sources, unresolved endpoints, - and nil inputs; consuming validators need only enough coverage to prove they - use the common policy; and -- existing validator tests should be simplified when the shared helper makes - module-local cases redundant. - -Update `docs/internal/dnd.md` to document the durable conventions for -contextual correction guidance, complete issue collection, private-schema enum -guardrails, and shared full-span extraction containment. Update -`docs/internal/pipeline.md` only if framework behavior changes; the intended -work applies its existing contract and should normally require only a link or -no change. No new ADR is required because ADR-0014 and the architecture policy -already decide the separation between semantic guidance and internal -diagnostics. Create an ADR only if implementation requires changing that -framework-level decision. - -## Non-Goals - -- Passing operator diagnostics or internal errors directly to an LLM. -- Exposing durable IDs, hashes, reason codes, validator keys, or raw provider - responses in correction text. -- Changing stage retry counts, PromptKit structural-repair budgets, validator - failure policy, or warning classification. -- Silently repairing domain-semantic defects or weakening deterministic - validators after adding schema guardrails. -- Adding provider-sensitive schema constructs beyond simple enums. -- Moving D&D-specific behavior into the generic pipeline framework. -- Generalizing the scene-description exact-range contract into the shared - extraction containment helper. - -## Acceptance Criteria - -- Every producer-correctable D&D rejection reviewed in this work gives the next - attempt all currently detectable, actionable corrections in contextual prose. -- No model-facing correction request contains an opaque entity ID, hash, - validator key, reason code, Go-style field path, or unfiltered operator error. -- Spell shape and enemy engagement validation aggregate independent defects in - one pass and keep outputs immutable. -- The four private LLM schemas reject unsupported categorical values, retain - provider-compatible required-only shapes, and leave durable v1 contracts - unchanged. -- All nine applicable extraction source-reference validators use one D&D-shared - full-span containment policy; scene descriptions retain their specialized - exact-range check. -- Validator and asset fingerprints change wherever their effective policy - changes, preventing reuse of stale checkpoints. -- Canonical internal documentation records the conventions future D&D - validators must follow. -- Focused tests, `go test ./...`, `go vet ./...`, and - `go build ./cmd/notarius` pass in a supported development environment. diff --git a/docs/roadmap/future.md b/docs/roadmap/future.md index d8220705..91f666eb 100644 --- a/docs/roadmap/future.md +++ b/docs/roadmap/future.md @@ -14,8 +14,9 @@ eligibility, and the separation of actionable process warnings from quality diagnostics. The remaining near-term work applies those completed foundations to domain review and empirical evaluation. -The active D&D reliability work is defined by -[Feedback-Aware Semantic-Reconciliation Retries](semantic-reconciliation-retry-feedback.md). +Near-term reliability work should now be selected from the concrete evaluation +and extension opportunities below. The retry, validation, and subprocess +foundations described above are implemented current behavior. ## Near-Term D&D Pipeline diff --git a/docs/roadmap/item-occurrence-holder-reliability.md b/docs/roadmap/item-occurrence-holder-reliability.md deleted file mode 100644 index ab888232..00000000 --- a/docs/roadmap/item-occurrence-holder-reliability.md +++ /dev/null @@ -1,95 +0,0 @@ -# D&D Item-Occurrence Holder Reliability - -## Purpose - -Improve item-occurrence extraction reliability for smaller models by making -holder rules easier to follow and semantic retry feedback specific enough to -correct a rejected response. - -## Problem - -The item-occurrence prompt correctly defines the permitted `from` and `to` -values for every occurrence kind, but presents the rules in dense prose. The -private schema requires both nullable fields but cannot express their -cross-field relationship. The deterministic shape validator correctly rejects -incompatible combinations, yet its model-facing correction guidance only asks -for generally valid holder combinations. Notarius therefore resends the -defective response without telling the model which contextual records failed or -what their corrected holder shape must be. - -## Target State - -- The module instructions present one compact, unambiguous holder matrix: - `discovered` uses two nulls; `acquired` uses a null `from` and non-null `to`; - `lost` and `consumed` use a non-null `from` and null `to`; and `transferred` - uses two distinct named party members. -- The private LLM schema continues requiring `quantity`, `from`, and `to` with - nullable types, and constrains `kind` to the five supported values. The - durable v1 artifact contract remains unchanged. -- A holder-combination rejection supplies bounded, semantically meaningful - correction guidance that identifies affected occurrences by contextual item - name and transcript evidence where useful, states the required JSON - null/non-null shape, and requests a complete corrected replacement. -- Correction guidance never exposes durable item IDs, hashes, validator keys, - reason codes, or other opaque implementation identifiers. It does not rely - solely on mapped array indexes, because canonical extraction ordering may - differ from the raw response order appended to the retry prompt. -- Operator-facing validation messages remain specific and include enough - information to distinguish a missing holder from an extra or misplaced - holder. -- The existing validator remains strict. Holder combinations are not silently - rewritten: unlike reversed evidence endpoints, changing a holder can conceal - a misclassified discovery, acquisition, loss, consumption, or transfer. - -## Required Work - -1. Rewrite the holder paragraph in - `assets/dnd/item-occurrences/prompts/instructions.md` as a compact matrix, - retaining the existing classification and currency rules without adding - contradictory repetition. -2. Add the five supported `kind` values as an enum in the private item-occurrence - response schema. Keep every listed field required and keep `quantity`, - `from`, and `to` explicitly nullable. -3. Refactor the item-occurrence shape validator to build detailed bounded - operator issues and separate actionable correction guidance from the same - evaluated candidate. Group repeated holder failures where practical while - retaining contextual record identification and every distinct correction - rule needed by the model. -4. Preserve the generic validation retry protocol: the complete original - prompt, exact defective assistant response, and one user correction message. - Do not add a module-local retry loop. -5. Ensure the changed prompt and schema produce new computed fingerprints, and - bump the validator policy fingerprint so stale checkpoints cannot be reused - across the behavior change. Update internal documentation where needed; the - external item-occurrence contract requires no semantic change. -6. Add lean offline regression coverage for the holder matrix, schema enum, - contextual correction guidance, bounded aggregation, absence of opaque IDs, - and a rejected-then-corrected extraction attempt. Avoid exact prose snapshots - and live-provider tests. - -## Non-Goals - -- Automatically deleting, moving, or inferring holder values. -- Changing the durable item-occurrence schema or event taxonomy. -- Encoding the complete semantic holder matrix through provider-sensitive - conditional JSON Schema constructs. -- Adding warnings for ordinary semantic rejection or successful correction. -- Tuning unrelated D&D extraction prompts or validators. - -## Acceptance Criteria - -- The maintained item-occurrence prompt shows the clear holder matrix, - and the private schema rejects unsupported kind strings while remaining - compatible with strict structured-output providers. -- A rejected `acquired` occurrence receives guidance that explicitly requires - `from: null` and a non-null `to`; equivalent exact guidance exists for the - other four kinds. -- Multiple invalid occurrences yield one bounded correction request containing - all distinct required fixes and contextual identifiers that can be matched to - the appended raw response. -- No correction message contains an item ID, hash, validator key, or reason - code, and the raw model candidate remains byte-faithful. -- Valid candidates and the durable output contract are unchanged; invalid - holder combinations still fail after configured retries are exhausted. -- Focused tests, `go test ./...`, `go vet ./...`, and `go build ./cmd/notarius` - pass. diff --git a/docs/roadmap/semantic-reconciliation-candidate-numbers.md b/docs/roadmap/semantic-reconciliation-candidate-numbers.md deleted file mode 100644 index 10339348..00000000 --- a/docs/roadmap/semantic-reconciliation-candidate-numbers.md +++ /dev/null @@ -1,270 +0,0 @@ -# Semantic Reconciliation Candidate Numbers - -## Purpose - -Make the private semantic-reconciliation protocol easier for smaller models to -follow by clearly separating request-local candidate selection from transcript -unit addressing. Replace the ambiguous `candidate_id` vocabulary with explicit -one-based candidate numbers, declare the valid number range for every request, -and ensure both initial and corrective instructions describe the exact response -the model can produce. - -## Motivation And Current State - -The domain-neutral semantic-reconciliation core supplies each eligible registry -candidate with a contiguous, one-based integer handle. The model returns groups -of those handles, deterministic code resolves them to retained typed records, -and unsafe groups are discarded without merging their candidates. This is the -architecture required by ADR-0013 and correctly prevents model-generated values -from becoming durable entity identity. - -The current private JSON vocabulary calls these handles `candidate_id`, -`candidate_ids`, and `canonical_candidate_id`. Candidate material and transcript -context also contain `start_unit_id`, `end_unit_id`, and transcript unit `id` -values. Although the shared protocol says to use only supplied candidate IDs, -all of these values are nearby integers and the term “ID” does not clearly -communicate which namespace is request-local. - -An observed small-model run exhausted semantic-proposal retries after returning -a duplicate group with an unknown candidate member and fewer than two remaining -distinct valid members. The safe fallback behaved correctly: the group was -omitted and all affected items were preserved separately. The issue pattern is -consistent with confusing a transcript or evidence unit ID for a candidate -handle, though an ordinary warning intentionally does not retain the exact -model response needed to prove that inference. - -The item-specific semantic instructions contain a related mismatch. They tell -the model to choose a supplied candidate name as canonical even though the -private response can return only the handle of the candidate whose label should -be retained. - -## Decisions - -### Use candidate-number vocabulary throughout the private protocol - -The model-visible candidate material will use `candidate_number`. Duplicate -groups will use `candidate_numbers` and `canonical_candidate_number`. Matching -Go names, internal comments, diagnostics, tests, and current-behavior internal -documentation will use “candidate number” where they refer to this private -request-local value. - -The rename applies coherently to the generic semantic-reconciliation core and -all current consumers: NPC, item, and location registry normalization. It does -not change the abstract architectural term “request-local candidate handle” in -ADR-0013; a candidate number remains the concrete representation of that -handle. - -The old JSON property names will not be accepted as aliases. No compatibility -adapter, dual-field schema, migration reader, or legacy prompt will remain. -This is a private model contract, not a durable artifact contract. - -### Declare the valid range in each request - -Prepared candidate material will contain an explicit request-level range: - -```json -{ - "candidate_number_range": { - "first": 1, - "last": 12 - }, - "candidates": [ - { - "candidate_number": 1, - "label": "Example", - "source_refs": [] - } - ] -} -``` - -`first` and `last` are required integers. For every model invocation, `first` -is 1 and `last` is the number of eligible candidates in the prepared request. -The candidate list remains contiguous and one-based. Preparation owns this -derived declaration and must keep it consistent with the emitted candidate -objects and retained mappings. - -The initial shared protocol will state that valid candidate numbers are exactly -the declared inclusive range and that the model must use the explicit -`candidate_number` attached to each candidate. Transcript unit `id` values and -evidence `start_unit_id` and `end_unit_id` values identify source positions; -they are never candidate numbers and must never appear in the response merely -because they are integers. - -### Make retry guidance request-aware - -When proposal assessment finds a non-positive or unknown member or canonical -number, the model-facing correction will include the request’s exact valid -inclusive candidate-number range. It will retain response-local duplicate-group -ordinals so the model can locate the defective group in the appended prior -response. It will explicitly distinguish candidate numbers from transcript and -evidence unit IDs. - -Correction guidance must remain bounded and semantically meaningful. It must -not expose internal issue categories, reason codes, Go paths, durable entity -IDs, source identities, or raw operator errors. The framework will continue to -append the exact latest defective response separately under the existing -replacement-request protocol; correction prose must not duplicate it. - -### Describe canonical selection in terms of the actual response - -The item-registry normalization instructions will say to set -`canonical_candidate_number` to the supplied candidate number whose label is -the clearest transcript-supported designation. Equivalent NPC and location -instructions must be reviewed for the same mismatch and corrected if present. -Domain instructions continue to decide semantic equivalence and preferred -canonical labels; the shared protocol continues to own selection mechanics. - -## Target End State - -- Every semantic-reconciliation prompt presents one unambiguous, request-local - candidate-number namespace. -- Candidate material declares the exact valid inclusive range and assigns every - eligible candidate one explicit number in that range. -- Transcript and evidence unit identifiers are clearly identified as a - separate namespace that is never valid in proposal selection fields. -- Structured responses contain only `duplicate_groups`, - `candidate_numbers`, and `canonical_candidate_number` at their existing - logical levels; the former `candidate_id` property family is rejected. -- Proposal assessment remains deterministic, validates exact membership in the - prepared candidate set, discards unsafe groups, applies independent safe - groups, and preserves affected candidates separately on fallback. -- A retry for an invalid selection tells the model which response-local groups - need correction and the exact valid candidate-number range without exposing - internal diagnostic vocabulary. -- NPC, item, and location registry reconciliation use the same generic private - protocol, while their typed consolidation and identity rules remain - domain-owned. -- Durable registry schemas, artifact IDs, evidence ranges, warning policy, - retry budgets, and PromptKit structural-repair budgets do not change. - -## Required Work - -### Generic protocol and model - -- Rename the model-facing fields in the generic candidate material, proposal - response types, private response schema, shared protocol prompt, and - presentation prompt. -- Rename internal semantic-reconciliation fields and accessors where they - represent request-local candidate numbers, so implementation terminology - does not immediately drift back toward ambiguous IDs. -- Add the required `candidate_number_range` object to prepared material and - derive it from the filtered eligible candidate set. -- Preserve the current contiguous one-based assignment, immutable retained - mapping, bounded material limits, stable plan ordering, and exact typed - resolution behavior. -- Keep exact candidate membership as deterministic semantic validation. The - shared static response schema may enforce positive integers and structural - shape, but it must not pretend to encode a request-specific maximum that it - cannot know. - -### Request-aware corrective feedback - -- Make the valid candidate-number range available to the generic correction - renderer through owned preparation or reconciliation result state. -- Add the exact range and the candidate-number/source-unit distinction to - corrections for unknown or non-positive proposal members and canonical - selections. Retain existing guidance for repeated members, undersized groups, - canonical membership, and overlapping groups. -- Preserve stable grouping, de-duplication, response-local group ordinals, byte - bounds, and separation between operator diagnostics and model guidance. -- Ensure NPC, item, and location normalizers use the generic renderer rather - than duplicating range or namespace language. Item-specific currency - guidance remains an additive domain rule. - -### D&D semantic instructions - -- Rewrite the item-registry canonical-selection sentence around - `canonical_candidate_number` and the selected candidate’s label. -- Review the NPC- and location-registry normalization instructions and apply - the same correction wherever they describe returning a name rather than - selecting its candidate number. -- Do not duplicate the generic number-range or namespace rules in D&D-specific - files; those rules belong in the mandatory shared protocol. - -### Fingerprints and checkpoints - -- Revise the existing private v1 prompt and schema assets in place; do not add - legacy schema variants or compatibility aliases. -- Ensure changed protocol, candidate-material, response-schema, correction, - and typed-policy behavior participates in the established prompt, asset, and - checkpoint fingerprint mechanisms. Explicitly bump a core or local policy - fingerprint wherever the effective behavior is not already covered by an - asset digest. -- A checkpoint produced under the former candidate-ID contract must not be - reused as compatible with the candidate-number contract. - -## Testing - -Add lean offline coverage at the narrowest stable boundaries: - -- preparation tests should prove contiguous candidate numbers, an exact - matching range declaration, filtering behavior, immutable mappings, and the - absence of former model-facing field names; -- private-schema tests should accept the candidate-number proposal shape and - reject the former candidate-ID fields, missing required properties, unknown - properties, non-integer numbers, non-positive numbers, and undersized arrays; -- proposal-assessment tests should retain the existing safety matrix using the - renamed vocabulary and prove that exact prepared-set membership still owns - acceptance; -- correction tests should prove that an unknown candidate number produces the - exact valid range and namespace distinction without internal issue labels, - reason codes, or operator paths; -- prompt tests should prove selection of the shared protocol and the corrected - domain instruction, without snapshotting complete prose or exact message - lengths; and -- representative NPC, item, and location normalizer tests should prove safe - application, corrective retry, and exhaustion fallback through the renamed - private contract. - -Do not add live-provider tests to the default suite. After implementation, a -maintainer may repeat the previously problematic small-model workload as -observational validation, preferably with an explicitly requested debug bundle -if the warning persists. - -## Documentation - -Update `docs/internal/modules.md` as the canonical description of the generic -semantic-reconciliation mechanism and `docs/internal/dnd.md` for its three D&D -registry consumers. Describe candidate numbers as request-local handles, -including the declared range and namespace separation, without repeating the -private JSON schema in both documents. - -No new ADR is required. This work reinforces ADR-0012’s prohibition on opaque -model-reproduced identity and ADR-0013’s accepted request-local-handle design; -it changes only the concrete private vocabulary and reliability presentation. -The architecture policy already owns the durable invariant and need not change -unless implementation reveals a genuinely new architectural decision. - -## Non-Goals - -- Changing durable NPC, item, or location artifact schemas or identifiers. -- Asking the model to return candidate labels, evidence ranges, transcript - text, source identities, or replacement records. -- Accepting both candidate-ID and candidate-number response shapes. -- Guessing that an unknown number denotes a transcript unit, list position, or - different candidate and silently repairing the proposal. -- Weakening deterministic proposal validation or typed domain guards. -- Increasing stage retries or PromptKit structural-repair attempts to mask an - ambiguous protocol. -- Introducing a request-specific dynamic response schema in this work set. - -## Acceptance Criteria - -- No model-facing semantic-reconciliation asset or private JSON shape uses the - former `candidate_id`, `candidate_ids`, or `canonical_candidate_id` fields. -- Candidate material and retry guidance expose the exact valid inclusive - candidate-number range for the current request. -- Shared instructions explicitly prohibit using transcript unit IDs or evidence - range endpoints as candidate numbers. -- Item, NPC, and location canonical-selection instructions refer to the actual - candidate-number response contract. -- Invalid or ambiguous proposals remain safely rejected without false merges; - retry exhaustion preserves affected candidates separately and emits the - existing process warning. -- Durable outputs and public schemas are unchanged, while incompatible private - checkpoints are invalidated. -- Canonical internal documentation records the implemented terminology and - ownership boundaries. -- Focused tests, `go test ./...`, `go vet ./...`, and - `go build ./cmd/notarius` pass in a supported development environment. diff --git a/docs/roadmap/semantic-reconciliation-retry-feedback.md b/docs/roadmap/semantic-reconciliation-retry-feedback.md deleted file mode 100644 index d8185d13..00000000 --- a/docs/roadmap/semantic-reconciliation-retry-feedback.md +++ /dev/null @@ -1,117 +0,0 @@ -# Feedback-Aware Semantic-Reconciliation Retries - -## Purpose - -Improve semantic registry reconciliation for smaller models by turning the -existing module-requested retry into a bounded corrective exchange. Preserve -the current deterministic safety checks, shared retry budget, and safe fallback -when correction is unsuccessful. - -## Problem - -NPC, item, and location registry normalizers can reject unsafe duplicate-group -proposals and retain a safe partial result. They currently ask the pipeline for -another attempt, but the next attempt receives the original prompt without the -defective response or an explanation of the defect. The normalizers also build -their retry messages from internal issue-category strings, which are suitable -for diagnostics but not useful model instructions. If all attempts are -exhausted, the warning reports the omitted-group count without explaining the -final defect or confirming that affected candidates were preserved separately. - -## Target State - -- A normalizer retry directive may carry model-facing correction guidance - separately from its stable reason code, operator message, and fallback - diagnostics. -- When the directive includes correction guidance and a valid - `single_response_v1` model candidate, the shared producer-attempt state - machine reconstructs the complete original prompt and appends the exact - defective response as an assistant message followed by one user correction - message. Earlier failed attempts do not accumulate. -- This correction consumes the existing normalize-stage retry budget. It does - not add a module-local loop or alter PromptKit's independent structured-output - repair budget. -- A retry caused by structurally invalid output remains a fresh stage attempt - without semantic feedback when no valid model candidate is available. -- Internal issue categories remain available for operator and debug - diagnostics, but are never sent to a model. A shared renderer translates - every known category into concise prose that explains the defect and the - required valid shape. -- Item reconciliation appends its domain rule when a proposal attempts to - combine different currency denominations or currency with a non-currency - item. NPC and location reconciliation use only the shared protocol guidance. -- On final exhaustion, safe proposal groups remain applied, unsafe groups - remain separate, and one process warning gives a bounded summary of the final - defect and explicitly states that the affected candidates were preserved - separately. - -## Required Work - -1. Extend `contracts.NormalizeRetry` and its clone, validation, and debug paths - with an optional, bounded model-facing correction-guidance field. Keep - reason codes and operator messages out of model input. Require guidance to - be paired with a valid `single_response_v1` model candidate; preserve the - existing feedback-free directive for structural retry cases. -2. Extend the generic producer-attempt state machine so a feedback-capable - module retry constructs `SemanticCorrection` from the latest exact response - and the supplied guidance. Preserve attempt provenance, cancellation, - terminal policy, diagnostic handling, byte-faithful response provenance, - and the rule that only the latest response is included. -3. Add a model-facing renderer to `internal/framework/semanticreconcile`. - Implement it with a typed lookup table keyed by `IssueCategory` and an - authoritative list of all categories, rather than a positional array. - Render response-local duplicate-group numbers in human-friendly form, - de-duplicate repeated guidance, enforce a size bound, and return an error for - any unmapped category instead of exposing its raw value. Retain - `IssueDetails` for operator/debug diagnostics. -4. Update the NPC, item, and location registry normalizers to populate the new - correction guidance for semantically invalid proposals. Add the item-only - currency guidance after the shared prose when typed application rejects a - group. Do not add guidance to the invalid-structured-output path when no - exact valid candidate exists. -5. Improve exhausted-fallback diagnostics across all three registries. Describe - the final proposal failure accurately, state that affected candidates were - preserved separately, and keep warning samples bounded and free of raw model - responses. -6. Update the shared semantic-reconciliation and affected normalizer policy - fingerprints so checkpoints produced under blind retry behavior are not - reused. Record the durable module-retry correction decision in a new ADR, - and update the architecture, pipeline, LLM-runtime, and D&D internal docs in - their canonical scopes when the behavior is implemented. -7. Add lean offline behavioral coverage for corrective module retries, exact - latest-response use, feedback-free structural retries, contract validation, - complete category-to-prose coverage, domain-specific item guidance, bounded - messages, safe final fallback, and warning content. Avoid exact prose - snapshots, prompt-length assertions, live providers, and redundant tests of - PromptKit internals. - -## Non-Goals - -- Changing normalize retry counts, PromptKit repair settings, or pipeline - terminal policies. -- Changing the reconciliation prompt protocol, response schema, registry - artifact schemas, or request-local candidate IDs. -- Reintroducing provider-sensitive schema constraints such as `uniqueItems`. -- Automatically applying an unsafe duplicate proposal or treating a fallback - warning as an extraction-quality warning. -- Persisting raw model responses or correction text in ordinary warnings, - receipts, manifests, caches, or checkpoints. - -## Acceptance Criteria - -- A semantic proposal rejected by an NPC, item, or location registry normalizer - receives the exact defective response and actionable prose on the next - normalize attempt, using only the configured stage retry budget. -- Every `IssueCategory` has non-empty model-facing guidance, no raw category or - reason-code string reaches the model, and an unmapped future category fails - closed. -- An invalid item consolidation additionally tells the model to keep currency - denominations distinct and not mix currency with non-currency items. -- Structural-output failures without a valid model candidate retain the current - fresh-retry behavior. -- After retry exhaustion, safe consolidations remain applied, affected - candidates remain separate, and the single fallback warning explains both - the final cause and preservation behavior without including raw model output. -- Checkpoint identities change where policy changed, canonical documentation - and a new ADR describe the implemented behavior, and focused tests plus - `go test ./...`, `go vet ./...`, and `go build ./cmd/notarius` pass. diff --git a/docs/roadmap/source-reference-canonicalization.md b/docs/roadmap/source-reference-canonicalization.md deleted file mode 100644 index adb2be71..00000000 --- a/docs/roadmap/source-reference-canonicalization.md +++ /dev/null @@ -1,96 +0,0 @@ -# D&D Source-Reference Endpoint Canonicalization - -## Purpose - -Make model-supplied D&D evidence ranges resilient to reversed endpoints without -spending producer retries on a losslessly repairable representation error. - -## Problem - -The shared extraction-evidence prompt requires `start_unit_id` and -`end_unit_id`, but does not state that start means the earlier unit in -transcript order. The private LLM schemas cannot express that cross-field -relationship. Current extraction adapters sort and de-duplicate ranges but -preserve reversed endpoints, so deterministic source-reference validators -reject otherwise useful candidates and may exhaust the producer retry budget. - -## Target State - -- The shared D&D extraction-evidence prompt states that `start_unit_id` is the - earlier endpoint and `end_unit_id` is the same or a later endpoint according - to transcript order. -- The existing LLM-facing and durable `start_unit_id`/`end_unit_id` shapes stay - unchanged. No unordered-pair schema or compatibility migration is introduced. -- One helper owned by `internal/modules/dnd/shared` orders the endpoints of a - source reference using `source.DocumentIndex` positions. It never assumes - that numerically smaller unit IDs occur earlier. -- The helper swaps endpoints only when the reference belongs to the current - source and both endpoint IDs resolve in that source. Forward and single-unit - ranges remain unchanged. Wrong-source, missing, non-positive, or otherwise - unresolved endpoints remain unchanged for deterministic validators to reject. -- Every D&D extraction adapter that consumes model-supplied evidence ranges - applies this endpoint operation before its existing range sorting, - de-duplication, earliest-evidence calculation, and validation. The covered - artifact families are spells, NPC registry, NPC occurrences, item registry, - item occurrences, location registry, location occurrences, combat turns, and - enemy events. -- Successful endpoint swapping is ordinary deterministic canonicalization. It - does not emit a warning, quality diagnostic, or retry request. -- The exact raw model response remains available through existing model-candidate - and debug provenance. Only the typed candidate presented to validators and - later stages receives the canonical endpoint order. -- Source-reference validators remain strict and continue rejecting reversed - ranges that reach them from another producer or indicate an application bug. - Normalizers do not become a fallback repair boundary for invalid extraction - candidates. - -## Required Work - -1. Clarify endpoint ordering in - `assets/dnd/shared/prompts/common-dnd-extraction-evidence.md`, preserving one - byte-identical shared instruction for every consuming prompt. -2. Extend the shared source-reference ordering utility with a mutation-safe - endpoint-ordering operation that uses document position and preserves - unresolvable references. -3. Integrate the shared operation into all nine model-response-to-candidate - mappings listed above. Do not duplicate endpoint comparison logic in the - individual artifact packages. -4. Update `docs/internal/dnd.md` to distinguish lossless endpoint ordering from - malformed-reference validation. External artifact documentation and schemas - require no change because their contract already requires ordered ranges. -5. Add lean regression coverage for forward, reversed, single-unit, - non-monotonic-ID, wrong-source, and unresolved endpoint cases. Add enough - adapter-level coverage to prove the shared operation is wired into every - affected extraction family without duplicating the helper's full case - matrix in each package. -6. Update prompt-asset and fingerprint expectations affected by the shared - prompt revision, while avoiding exact token-, byte-length-, or prose-snapshot - change-detector tests. - -## Non-Goals - -- Changing durable source-reference schemas or replacing named endpoints with - an unordered LLM response shape. -- Weakening source-reference validation or silently repairing missing, - out-of-source, or out-of-chunk evidence. -- Applying this policy to scene chunk-plan ranges, which are not extraction - evidence references and have a separate ownership and validation boundary. -- Adding warnings or durable normalization observations for routine endpoint - ordering. - -## Acceptance Criteria - -- A D&D extraction candidate whose two resolvable evidence endpoints are - reversed reaches validation as the equivalent forward range without a - producer retry. -- Ordering follows document position even when unit IDs are not numerically - increasing. -- Every affected D&D extractor uses the shared implementation; no module-local - endpoint-swap implementation remains. -- References that cannot be safely ordered remain unchanged and are rejected by - the existing deterministic validators where applicable. -- Raw model-candidate provenance remains byte-faithful to the provider response. -- LLM-facing and durable schemas, validator strictness, module keys, default - chains, and output contracts remain unchanged. -- Focused package tests, repository-wide tests, `go vet ./...`, and - `go build ./cmd/notarius` pass.