Improve semantic reconciliation retries

docs/roadmap/future.md

@@ -15,7 +15,7 @@ diagnostics. The remaining near-term work applies those completed foundations
 to domain review and empirical evaluation.
 
 The active D&D reliability work is defined by
-[D&D Item-Occurrence Holder Reliability](item-occurrence-holder-reliability.md).
+[Feedback-Aware Semantic-Reconciliation Retries](semantic-reconciliation-retry-feedback.md).
 
 ## Near-Term D&D Pipeline
 

docs/roadmap/semantic-reconciliation-retry-feedback.md

@@ -0,0 +1,117 @@
+# Feedback-Aware Semantic-Reconciliation Retries
+
+## Purpose
+
+Improve semantic registry reconciliation for smaller models by turning the
+existing module-requested retry into a bounded corrective exchange. Preserve
+the current deterministic safety checks, shared retry budget, and safe fallback
+when correction is unsuccessful.
+
+## Problem
+
+NPC, item, and location registry normalizers can reject unsafe duplicate-group
+proposals and retain a safe partial result. They currently ask the pipeline for
+another attempt, but the next attempt receives the original prompt without the
+defective response or an explanation of the defect. The normalizers also build
+their retry messages from internal issue-category strings, which are suitable
+for diagnostics but not useful model instructions. If all attempts are
+exhausted, the warning reports the omitted-group count without explaining the
+final defect or confirming that affected candidates were preserved separately.
+
+## Target State
+
+- A normalizer retry directive may carry model-facing correction guidance
+  separately from its stable reason code, operator message, and fallback
+  diagnostics.
+- When the directive includes correction guidance and a valid
+  `single_response_v1` model candidate, the shared producer-attempt state
+  machine reconstructs the complete original prompt and appends the exact
+  defective response as an assistant message followed by one user correction
+  message. Earlier failed attempts do not accumulate.
+- This correction consumes the existing normalize-stage retry budget. It does
+  not add a module-local loop or alter PromptKit's independent structured-output
+  repair budget.
+- A retry caused by structurally invalid output remains a fresh stage attempt
+  without semantic feedback when no valid model candidate is available.
+- Internal issue categories remain available for operator and debug
+  diagnostics, but are never sent to a model. A shared renderer translates
+  every known category into concise prose that explains the defect and the
+  required valid shape.
+- Item reconciliation appends its domain rule when a proposal attempts to
+  combine different currency denominations or currency with a non-currency
+  item. NPC and location reconciliation use only the shared protocol guidance.
+- On final exhaustion, safe proposal groups remain applied, unsafe groups
+  remain separate, and one process warning gives a bounded summary of the final
+  defect and explicitly states that the affected candidates were preserved
+  separately.
+
+## Required Work
+
+1. Extend `contracts.NormalizeRetry` and its clone, validation, and debug paths
+   with an optional, bounded model-facing correction-guidance field. Keep
+   reason codes and operator messages out of model input. Require guidance to
+   be paired with a valid `single_response_v1` model candidate; preserve the
+   existing feedback-free directive for structural retry cases.
+2. Extend the generic producer-attempt state machine so a feedback-capable
+   module retry constructs `SemanticCorrection` from the latest exact response
+   and the supplied guidance. Preserve attempt provenance, cancellation,
+   terminal policy, diagnostic handling, byte-faithful response provenance,
+   and the rule that only the latest response is included.
+3. Add a model-facing renderer to `internal/framework/semanticreconcile`.
+   Implement it with a typed lookup table keyed by `IssueCategory` and an
+   authoritative list of all categories, rather than a positional array.
+   Render response-local duplicate-group numbers in human-friendly form,
+   de-duplicate repeated guidance, enforce a size bound, and return an error for
+   any unmapped category instead of exposing its raw value. Retain
+   `IssueDetails` for operator/debug diagnostics.
+4. Update the NPC, item, and location registry normalizers to populate the new
+   correction guidance for semantically invalid proposals. Add the item-only
+   currency guidance after the shared prose when typed application rejects a
+   group. Do not add guidance to the invalid-structured-output path when no
+   exact valid candidate exists.
+5. Improve exhausted-fallback diagnostics across all three registries. Describe
+   the final proposal failure accurately, state that affected candidates were
+   preserved separately, and keep warning samples bounded and free of raw model
+   responses.
+6. Update the shared semantic-reconciliation and affected normalizer policy
+   fingerprints so checkpoints produced under blind retry behavior are not
+   reused. Record the durable module-retry correction decision in a new ADR,
+   and update the architecture, pipeline, LLM-runtime, and D&D internal docs in
+   their canonical scopes when the behavior is implemented.
+7. Add lean offline behavioral coverage for corrective module retries, exact
+   latest-response use, feedback-free structural retries, contract validation,
+   complete category-to-prose coverage, domain-specific item guidance, bounded
+   messages, safe final fallback, and warning content. Avoid exact prose
+   snapshots, prompt-length assertions, live providers, and redundant tests of
+   PromptKit internals.
+
+## Non-Goals
+
+- Changing normalize retry counts, PromptKit repair settings, or pipeline
+  terminal policies.
+- Changing the reconciliation prompt protocol, response schema, registry
+  artifact schemas, or request-local candidate IDs.
+- Reintroducing provider-sensitive schema constraints such as `uniqueItems`.
+- Automatically applying an unsafe duplicate proposal or treating a fallback
+  warning as an extraction-quality warning.
+- Persisting raw model responses or correction text in ordinary warnings,
+  receipts, manifests, caches, or checkpoints.
+
+## Acceptance Criteria
+
+- A semantic proposal rejected by an NPC, item, or location registry normalizer
+  receives the exact defective response and actionable prose on the next
+  normalize attempt, using only the configured stage retry budget.
+- Every `IssueCategory` has non-empty model-facing guidance, no raw category or
+  reason-code string reaches the model, and an unmapped future category fails
+  closed.
+- An invalid item consolidation additionally tells the model to keep currency
+  denominations distinct and not mix currency with non-currency items.
+- Structural-output failures without a valid model candidate retain the current
+  fresh-retry behavior.
+- After retry exhaustion, safe consolidations remain applied, affected
+  candidates remain separate, and the single fallback warning explains both
+  the final cause and preservation behavior without including raw model output.
+- Checkpoint identities change where policy changed, canonical documentation
+  and a new ADR describe the implemented behavior, and focused tests plus
+  `go test ./...`, `go vet ./...`, and `go build ./cmd/notarius` pass.