Clarify semantic reconciliation candidate numbers

docs/internal/dnd.md

@@ -266,13 +266,16 @@ proposal to reconcile identity groups.
 
 The three registry normalizers instantiate the domain-neutral
 `internal/framework/semanticreconcile` engine with default bounds. Each
-eligible candidate receives a contiguous, one-based `candidate_id` for that
-request. The model sees that handle, the candidate label and source-free
-evidence ranges, plus bounded transcript windows; it returns only duplicate
+eligible candidate receives a contiguous, one-based `candidate_number` for that
+request, and candidate material declares the exact inclusive range. The model
+sees that handle, the candidate label and source-free evidence ranges, plus
+bounded transcript windows; it returns only duplicate
 groups of supplied handles and one supplied canonical handle per group. It
 never returns names, evidence, durable IDs, or replacement records. Identical
 labels and evidence remain independently selectable because their handles are
-distinct.
+distinct. Transcript unit `id` values and evidence `start_unit_id` and
+`end_unit_id` values are source positions in a separate namespace and are
+never valid candidate numbers.
 
 The generic core owns the mandatory handle protocol, candidate and transcript
 presentation, the private response schema, source-reference validation,

docs/internal/modules.md

@@ -95,19 +95,24 @@ source-backed artifact-family normalizer projects its deterministic records
 into contextual candidates and owned typed record envelopes, supplies its
 chosen prompt identity and resolved LLM profile, and constructs an engine with
 explicit limits. The core filters invalid evidence, assigns contiguous
-request-local integer handles, renders bounded candidate and transcript
-materials, invokes the structured-completion boundary, and assesses the
-returned duplicate groups into a stable non-overlapping plan.
+one-based request-local candidate numbers, renders bounded candidate and
+transcript materials, invokes the structured-completion boundary, and assesses
+the returned duplicate groups into a stable non-overlapping plan. Candidate
+material declares the exact inclusive number range for the request. Transcript
+unit IDs and evidence range endpoints remain source positions in a separate
+namespace and are never valid candidate numbers.
 
 The normalizer then applies that plan through a typed `ApplicationPolicy`. The
 core preserves ungrouped records, contribution order, and provenance while the
 artifact family owns group guards, field and evidence consolidation, durable
-ID derivation, retry and fallback presentation, classified diagnostics, and postconditions.
-Request-local handles do not enter the typed value or durable artifact. Fewer
-than two eligible candidates skips model invocation; exceeding a candidate or
-combined-material bound preserves the deterministic result under the family's
-fallback policy. Provider, transport, cancellation, and context-construction
-failures remain execution errors.
+ID derivation, retry and fallback presentation, classified diagnostics, and
+postconditions.
+These candidate numbers are the concrete private representation of ADR-0013's
+request-local handles; they do not enter the typed value or durable artifact.
+Fewer than two eligible candidates skips model invocation; exceeding a
+candidate or combined-material bound preserves the deterministic result under
+the family's fallback policy. Provider, transport, cancellation, and
+context-construction failures remain execution errors.
 
 When the engine actually makes a proposal call, its typed result carries the
 owned exact proposal response under the same correction contract as other

docs/roadmap/semantic-reconciliation-candidate-numbers.md

@@ -0,0 +1,270 @@
+# Semantic Reconciliation Candidate Numbers
+
+## Purpose
+
+Make the private semantic-reconciliation protocol easier for smaller models to
+follow by clearly separating request-local candidate selection from transcript
+unit addressing. Replace the ambiguous `candidate_id` vocabulary with explicit
+one-based candidate numbers, declare the valid number range for every request,
+and ensure both initial and corrective instructions describe the exact response
+the model can produce.
+
+## Motivation And Current State
+
+The domain-neutral semantic-reconciliation core supplies each eligible registry
+candidate with a contiguous, one-based integer handle. The model returns groups
+of those handles, deterministic code resolves them to retained typed records,
+and unsafe groups are discarded without merging their candidates. This is the
+architecture required by ADR-0013 and correctly prevents model-generated values
+from becoming durable entity identity.
+
+The current private JSON vocabulary calls these handles `candidate_id`,
+`candidate_ids`, and `canonical_candidate_id`. Candidate material and transcript
+context also contain `start_unit_id`, `end_unit_id`, and transcript unit `id`
+values. Although the shared protocol says to use only supplied candidate IDs,
+all of these values are nearby integers and the term “ID” does not clearly
+communicate which namespace is request-local.
+
+An observed small-model run exhausted semantic-proposal retries after returning
+a duplicate group with an unknown candidate member and fewer than two remaining
+distinct valid members. The safe fallback behaved correctly: the group was
+omitted and all affected items were preserved separately. The issue pattern is
+consistent with confusing a transcript or evidence unit ID for a candidate
+handle, though an ordinary warning intentionally does not retain the exact
+model response needed to prove that inference.
+
+The item-specific semantic instructions contain a related mismatch. They tell
+the model to choose a supplied candidate name as canonical even though the
+private response can return only the handle of the candidate whose label should
+be retained.
+
+## Decisions
+
+### Use candidate-number vocabulary throughout the private protocol
+
+The model-visible candidate material will use `candidate_number`. Duplicate
+groups will use `candidate_numbers` and `canonical_candidate_number`. Matching
+Go names, internal comments, diagnostics, tests, and current-behavior internal
+documentation will use “candidate number” where they refer to this private
+request-local value.
+
+The rename applies coherently to the generic semantic-reconciliation core and
+all current consumers: NPC, item, and location registry normalization. It does
+not change the abstract architectural term “request-local candidate handle” in
+ADR-0013; a candidate number remains the concrete representation of that
+handle.
+
+The old JSON property names will not be accepted as aliases. No compatibility
+adapter, dual-field schema, migration reader, or legacy prompt will remain.
+This is a private model contract, not a durable artifact contract.
+
+### Declare the valid range in each request
+
+Prepared candidate material will contain an explicit request-level range:
+
+```json
+{
+  "candidate_number_range": {
+    "first": 1,
+    "last": 12
+  },
+  "candidates": [
+    {
+      "candidate_number": 1,
+      "label": "Example",
+      "source_refs": []
+    }
+  ]
+}
+```
+
+`first` and `last` are required integers. For every model invocation, `first`
+is 1 and `last` is the number of eligible candidates in the prepared request.
+The candidate list remains contiguous and one-based. Preparation owns this
+derived declaration and must keep it consistent with the emitted candidate
+objects and retained mappings.
+
+The initial shared protocol will state that valid candidate numbers are exactly
+the declared inclusive range and that the model must use the explicit
+`candidate_number` attached to each candidate. Transcript unit `id` values and
+evidence `start_unit_id` and `end_unit_id` values identify source positions;
+they are never candidate numbers and must never appear in the response merely
+because they are integers.
+
+### Make retry guidance request-aware
+
+When proposal assessment finds a non-positive or unknown member or canonical
+number, the model-facing correction will include the request’s exact valid
+inclusive candidate-number range. It will retain response-local duplicate-group
+ordinals so the model can locate the defective group in the appended prior
+response. It will explicitly distinguish candidate numbers from transcript and
+evidence unit IDs.
+
+Correction guidance must remain bounded and semantically meaningful. It must
+not expose internal issue categories, reason codes, Go paths, durable entity
+IDs, source identities, or raw operator errors. The framework will continue to
+append the exact latest defective response separately under the existing
+replacement-request protocol; correction prose must not duplicate it.
+
+### Describe canonical selection in terms of the actual response
+
+The item-registry normalization instructions will say to set
+`canonical_candidate_number` to the supplied candidate number whose label is
+the clearest transcript-supported designation. Equivalent NPC and location
+instructions must be reviewed for the same mismatch and corrected if present.
+Domain instructions continue to decide semantic equivalence and preferred
+canonical labels; the shared protocol continues to own selection mechanics.
+
+## Target End State
+
+- Every semantic-reconciliation prompt presents one unambiguous, request-local
+  candidate-number namespace.
+- Candidate material declares the exact valid inclusive range and assigns every
+  eligible candidate one explicit number in that range.
+- Transcript and evidence unit identifiers are clearly identified as a
+  separate namespace that is never valid in proposal selection fields.
+- Structured responses contain only `duplicate_groups`,
+  `candidate_numbers`, and `canonical_candidate_number` at their existing
+  logical levels; the former `candidate_id` property family is rejected.
+- Proposal assessment remains deterministic, validates exact membership in the
+  prepared candidate set, discards unsafe groups, applies independent safe
+  groups, and preserves affected candidates separately on fallback.
+- A retry for an invalid selection tells the model which response-local groups
+  need correction and the exact valid candidate-number range without exposing
+  internal diagnostic vocabulary.
+- NPC, item, and location registry reconciliation use the same generic private
+  protocol, while their typed consolidation and identity rules remain
+  domain-owned.
+- Durable registry schemas, artifact IDs, evidence ranges, warning policy,
+  retry budgets, and PromptKit structural-repair budgets do not change.
+
+## Required Work
+
+### Generic protocol and model
+
+- Rename the model-facing fields in the generic candidate material, proposal
+  response types, private response schema, shared protocol prompt, and
+  presentation prompt.
+- Rename internal semantic-reconciliation fields and accessors where they
+  represent request-local candidate numbers, so implementation terminology
+  does not immediately drift back toward ambiguous IDs.
+- Add the required `candidate_number_range` object to prepared material and
+  derive it from the filtered eligible candidate set.
+- Preserve the current contiguous one-based assignment, immutable retained
+  mapping, bounded material limits, stable plan ordering, and exact typed
+  resolution behavior.
+- Keep exact candidate membership as deterministic semantic validation. The
+  shared static response schema may enforce positive integers and structural
+  shape, but it must not pretend to encode a request-specific maximum that it
+  cannot know.
+
+### Request-aware corrective feedback
+
+- Make the valid candidate-number range available to the generic correction
+  renderer through owned preparation or reconciliation result state.
+- Add the exact range and the candidate-number/source-unit distinction to
+  corrections for unknown or non-positive proposal members and canonical
+  selections. Retain existing guidance for repeated members, undersized groups,
+  canonical membership, and overlapping groups.
+- Preserve stable grouping, de-duplication, response-local group ordinals, byte
+  bounds, and separation between operator diagnostics and model guidance.
+- Ensure NPC, item, and location normalizers use the generic renderer rather
+  than duplicating range or namespace language. Item-specific currency
+  guidance remains an additive domain rule.
+
+### D&D semantic instructions
+
+- Rewrite the item-registry canonical-selection sentence around
+  `canonical_candidate_number` and the selected candidate’s label.
+- Review the NPC- and location-registry normalization instructions and apply
+  the same correction wherever they describe returning a name rather than
+  selecting its candidate number.
+- Do not duplicate the generic number-range or namespace rules in D&D-specific
+  files; those rules belong in the mandatory shared protocol.
+
+### Fingerprints and checkpoints
+
+- Revise the existing private v1 prompt and schema assets in place; do not add
+  legacy schema variants or compatibility aliases.
+- Ensure changed protocol, candidate-material, response-schema, correction,
+  and typed-policy behavior participates in the established prompt, asset, and
+  checkpoint fingerprint mechanisms. Explicitly bump a core or local policy
+  fingerprint wherever the effective behavior is not already covered by an
+  asset digest.
+- A checkpoint produced under the former candidate-ID contract must not be
+  reused as compatible with the candidate-number contract.
+
+## Testing
+
+Add lean offline coverage at the narrowest stable boundaries:
+
+- preparation tests should prove contiguous candidate numbers, an exact
+  matching range declaration, filtering behavior, immutable mappings, and the
+  absence of former model-facing field names;
+- private-schema tests should accept the candidate-number proposal shape and
+  reject the former candidate-ID fields, missing required properties, unknown
+  properties, non-integer numbers, non-positive numbers, and undersized arrays;
+- proposal-assessment tests should retain the existing safety matrix using the
+  renamed vocabulary and prove that exact prepared-set membership still owns
+  acceptance;
+- correction tests should prove that an unknown candidate number produces the
+  exact valid range and namespace distinction without internal issue labels,
+  reason codes, or operator paths;
+- prompt tests should prove selection of the shared protocol and the corrected
+  domain instruction, without snapshotting complete prose or exact message
+  lengths; and
+- representative NPC, item, and location normalizer tests should prove safe
+  application, corrective retry, and exhaustion fallback through the renamed
+  private contract.
+
+Do not add live-provider tests to the default suite. After implementation, a
+maintainer may repeat the previously problematic small-model workload as
+observational validation, preferably with an explicitly requested debug bundle
+if the warning persists.
+
+## Documentation
+
+Update `docs/internal/modules.md` as the canonical description of the generic
+semantic-reconciliation mechanism and `docs/internal/dnd.md` for its three D&D
+registry consumers. Describe candidate numbers as request-local handles,
+including the declared range and namespace separation, without repeating the
+private JSON schema in both documents.
+
+No new ADR is required. This work reinforces ADR-0012’s prohibition on opaque
+model-reproduced identity and ADR-0013’s accepted request-local-handle design;
+it changes only the concrete private vocabulary and reliability presentation.
+The architecture policy already owns the durable invariant and need not change
+unless implementation reveals a genuinely new architectural decision.
+
+## Non-Goals
+
+- Changing durable NPC, item, or location artifact schemas or identifiers.
+- Asking the model to return candidate labels, evidence ranges, transcript
+  text, source identities, or replacement records.
+- Accepting both candidate-ID and candidate-number response shapes.
+- Guessing that an unknown number denotes a transcript unit, list position, or
+  different candidate and silently repairing the proposal.
+- Weakening deterministic proposal validation or typed domain guards.
+- Increasing stage retries or PromptKit structural-repair attempts to mask an
+  ambiguous protocol.
+- Introducing a request-specific dynamic response schema in this work set.
+
+## Acceptance Criteria
+
+- No model-facing semantic-reconciliation asset or private JSON shape uses the
+  former `candidate_id`, `candidate_ids`, or `canonical_candidate_id` fields.
+- Candidate material and retry guidance expose the exact valid inclusive
+  candidate-number range for the current request.
+- Shared instructions explicitly prohibit using transcript unit IDs or evidence
+  range endpoints as candidate numbers.
+- Item, NPC, and location canonical-selection instructions refer to the actual
+  candidate-number response contract.
+- Invalid or ambiguous proposals remain safely rejected without false merges;
+  retry exhaustion preserves affected candidates separately and emits the
+  existing process warning.
+- Durable outputs and public schemas are unchanged, while incompatible private
+  checkpoints are invalidated.
+- Canonical internal documentation records the implemented terminology and
+  ownership boundaries.
+- Focused tests, `go test ./...`, `go vet ./...`, and
+  `go build ./cmd/notarius` pass in a supported development environment.