Plan semantic reconciliation improvements
This commit is contained in:
@@ -24,28 +24,50 @@ not as committed release dates.
|
||||
|
||||
## Shared Normalization And Quality Work
|
||||
|
||||
### Generic LLM-Assisted Deduplication
|
||||
The active [Semantic Reconciliation Roadmap](semantic-reconciliation.md) owns
|
||||
the shared source-backed reconciliation core, request-local candidate protocol,
|
||||
generic prompt policy, typed adapter boundary, and initial D&D registry
|
||||
migrations.
|
||||
|
||||
- Add a reusable normalizer that asks an LLM to identify duplicate sets in a
|
||||
list and propose one replacement element for each set.
|
||||
- Define the minimum domain-neutral input contract, initially an ordered list
|
||||
whose elements retain stable unique IDs as internal deterministic state.
|
||||
Model proposals use contextual descriptors, or a specifically justified
|
||||
request-local short label, rather than durable IDs. Artifact-kind
|
||||
registrations or adapters may expose that structure without moving domain
|
||||
rules into the generic package.
|
||||
- Keep mutation deterministic: parse and validate the model's duplicate groups,
|
||||
resolve every supplied descriptor or local label exactly, reject overlapping
|
||||
or malformed groups, prevent unrelated insertion or deletion, and apply only
|
||||
approved replacement operations in code.
|
||||
- Preserve provenance needed for audit and downstream validation, and emit
|
||||
warnings describing every collapsed group.
|
||||
- Evaluate batching and context-window limits before applying the normalizer to
|
||||
large artifact collections.
|
||||
### Large-Collection Semantic Reconciliation
|
||||
|
||||
The model may use its own domain knowledge to judge semantic duplication; the
|
||||
generic implementation is responsible only for the common proposal contract,
|
||||
safety checks, and deterministic application of accepted changes.
|
||||
- Evaluate deterministic candidate blocking only after representative registry
|
||||
inputs exceed the active roadmap's bounded single-request limits. Blocking
|
||||
should use cheap, explainable signals to form plausible comparison sets while
|
||||
preserving the possibility that a duplicate appears outside a lexical name
|
||||
match.
|
||||
- Define correctness for candidates that appear in more than one block,
|
||||
conflicting canonical selections, transitive identity across blocks, retry
|
||||
isolation, and deterministic final ordering before implementation.
|
||||
- Prefer a reconciliation graph or union plan with explicit conflict checks
|
||||
over arbitrary fixed-size slices. Never silently treat a batch boundary as
|
||||
evidence that two candidates are distinct.
|
||||
- Record per-request bounds, block provenance, model calls, discarded
|
||||
proposals, and final group derivation well enough to audit a collapse.
|
||||
|
||||
### Operator-Selected Semantic Policies
|
||||
|
||||
- Consider allowing an operator to select an approved semantic-policy prompt
|
||||
for a typed reconciliation module without replacing the shared protocol,
|
||||
response schema, or deterministic safety rules.
|
||||
- Define the trusted asset source, configuration syntax, compatibility checks,
|
||||
startup validation, provenance, prompt fingerprinting, checkpoint effects,
|
||||
and support boundary before exposing the option.
|
||||
- Prefer selection among registered, typed-policy-compatible prompt assets over
|
||||
arbitrary filesystem prompt paths. Do not add this flexibility until an
|
||||
operator workflow requires it; artifact-family-owned policy remains simpler
|
||||
and safer for the initial implementation.
|
||||
|
||||
### Broader Reconciliation Inputs And Module Selection
|
||||
|
||||
- Revisit alternate context providers when a concrete non-source-backed entity
|
||||
collection needs semantic reconciliation. Any extension must preserve the
|
||||
same request-local identity, deterministic proposal validation, provenance,
|
||||
and typed application guarantees.
|
||||
- Consider a selectable generic normalizer only if Notarius gains a real
|
||||
domain-neutral typed artifact contract that can safely support it. Do not
|
||||
weaken exact artifact registration or introduce reflection-based arbitrary
|
||||
JSON mutation merely to expose a universal module key.
|
||||
|
||||
### Validation And Review
|
||||
|
||||
@@ -102,6 +124,18 @@ checkpoint reuse, when an older artifact may be decoded or adapted, and when a
|
||||
producer or all dependents must be recomputed. Do not add a general migration
|
||||
framework until an actual contract change requires one.
|
||||
|
||||
### Artifact-family-oriented physical packaging
|
||||
|
||||
[ADR-0004](../adr/0004-package-modules-by-domain.md) currently groups production
|
||||
extensions by domain and then by pipeline stage. After artifact-family
|
||||
ownership terminology is established and more families span extraction,
|
||||
normalization, validation, codecs, references, and assets, reassess whether a
|
||||
feature-first physical layout would improve navigation and reduce scattered
|
||||
changes enough to justify a repository-wide package migration. Any change must
|
||||
address Go dependency cycles, registrar ownership, stable public module keys,
|
||||
and supersession of the affected ADR-0004 decision. Conceptual artifact-family
|
||||
ownership does not by itself require this move.
|
||||
|
||||
## Blue-Sky Platform And Operations
|
||||
|
||||
These ideas are intentionally less specified. Promote one into an earlier
|
||||
|
||||
809
docs/roadmap/implementation.md
Normal file
809
docs/roadmap/implementation.md
Normal file
@@ -0,0 +1,809 @@
|
||||
# Semantic Reconciliation Implementation Plan
|
||||
|
||||
## Purpose
|
||||
|
||||
This document is the ordered implementation plan for the target state defined
|
||||
in the [Semantic Reconciliation Roadmap](semantic-reconciliation.md). Each
|
||||
numbered stage is sized for one gpt-5.6-terra implementation prompt and must be
|
||||
completed in order.
|
||||
|
||||
The feature roadmap owns product intent and durable policy. This document owns
|
||||
implementation sequence, package placement, private contracts, migration
|
||||
mechanics, and verification. When a detail below conflicts with a current
|
||||
implementation detail, preserve the feature-roadmap policy and deliberately
|
||||
replace the superseded implementation.
|
||||
|
||||
## Execution Rules
|
||||
|
||||
For every stage:
|
||||
|
||||
1. Read `docs/development.md`, all files under `docs/policy/`, the feature
|
||||
roadmap, the files named by the stage, and focused tests before editing.
|
||||
2. Implement only the assigned stage and prerequisites left incomplete by an
|
||||
earlier stage. Do not begin a later migration opportunistically.
|
||||
3. Preserve unrelated user changes and public module keys. Do not weaken typed
|
||||
artifact registration or introduce untyped JSON mutation.
|
||||
4. Keep all default tests deterministic, offline, and credential-free. Use a
|
||||
small fake structured-LLM client only at the external completion boundary.
|
||||
5. Test package-level behavior and consequential invariants. Do not add prompt
|
||||
length, exact asset hash, private constant, or collaborator-choreography
|
||||
change detectors.
|
||||
6. Run `gofmt` on changed Go files, the focused commands listed for the stage,
|
||||
and `git diff --check`. Resolve failures before handing off the stage.
|
||||
7. Do not update current-behavior documentation before Stage 11. ADR-0013 is
|
||||
the documentation-policy exception and may record the accepted decision
|
||||
before the complete implementation exists.
|
||||
|
||||
## Fixed Implementation Decisions
|
||||
|
||||
The following decisions are not left to individual stages:
|
||||
|
||||
- The shared package is `internal/framework/semanticreconcile`. It is
|
||||
domain-neutral framework support, not a configured pipeline module and not a
|
||||
D&D package.
|
||||
- The initial core supports source-backed entity candidates only. It accepts
|
||||
generic source references and source documents but no D&D types.
|
||||
- The model-facing candidate object uses `candidate_id`, `label`, and
|
||||
source-free `source_refs`. Eligible visible candidates receive contiguous
|
||||
integer IDs beginning with `1`.
|
||||
- The private response uses `duplicate_groups`, `candidate_ids`, and
|
||||
`canonical_candidate_id`. It contains no names, labels, evidence ranges,
|
||||
durable IDs, or replacement records.
|
||||
- The generic response contract is version `v1`, with schema key
|
||||
`semantic_reconciliation_llm`, schema ID
|
||||
`notarius.generic.semantic_reconciliation.llm`, schema name
|
||||
`notarius_semantic_reconciliation_llm_v1`, and registered filename
|
||||
`semantic_reconciliation_llm.v1.json`.
|
||||
- The complete generic default prompt ID is
|
||||
`generic.semantic_reconciliation`, version `v1`. It has no default LLM
|
||||
profile; callers must pass the resolved normalization profile.
|
||||
- Core prompt protocol and schema are mandatory. A domain prompt may replace
|
||||
only the generic semantic-policy message and must continue to select the
|
||||
core-owned protocol and input presentation assets.
|
||||
- Default source-context limits are radius `2`, at most `128` eligible
|
||||
candidates, and at most `262144` total serialized bytes across candidate and
|
||||
transcript input materials. Define these through one core-owned default
|
||||
limits value. Test custom limits relationally rather than duplicating the
|
||||
production literals throughout tests.
|
||||
- Fewer than two eligible candidates skips the LLM without warning. Exceeding
|
||||
a deterministic limit skips the LLM, preserves the deterministic
|
||||
preprocessed artifact, and emits a bounded domain fallback warning without a
|
||||
futile retry.
|
||||
- Invalid structured output requests the normalizer's existing retry behavior.
|
||||
A structurally valid proposal may retain independent safe groups while
|
||||
discarding invalid groups; any discarded group requests a retry, and retry
|
||||
exhaustion uses the existing deterministic fallback-warning contract.
|
||||
- Transport, provider, cancellation, and context-material construction errors
|
||||
remain execution errors. They are not converted into empty semantic results.
|
||||
- The shared application helper never emits domain warnings or derives durable
|
||||
IDs. It owns partition application, stable ordering, preservation of
|
||||
ungrouped records, and provenance bookkeeping; typed domain policy owns value
|
||||
consolidation, group guards, IDs, and warning presentation.
|
||||
- Production registration of the generic prompt and schema belongs to the
|
||||
generic registrar, which already runs before the D&D registrar. D&D prompt
|
||||
definitions may mount core-owned shared assets but remain owned and hashed by
|
||||
their D&D normalizer packages.
|
||||
- Existing D&D prompt IDs and prompt versions remain unchanged. The new generic
|
||||
schema has its own `v1` identity, so no private selector-schema compatibility
|
||||
layer is required.
|
||||
- Bump normalization policies to `dnd.npc_registry.normalize.v5`,
|
||||
`dnd.item_registry.normalize.v3`, and
|
||||
`dnd.location_registry.normalize.v3` during their respective migrations.
|
||||
- No public configuration, durable D&D schema, integration contract, module
|
||||
key, or validator-chain change is part of this work.
|
||||
|
||||
## Stage 1: Record The Semantic Reconciliation Decision
|
||||
|
||||
### Goal
|
||||
|
||||
Record the accepted architectural decision before introducing the reusable
|
||||
mechanism, without describing unimplemented behavior in current architecture
|
||||
or internal documentation.
|
||||
|
||||
### Work
|
||||
|
||||
- Create
|
||||
`docs/adr/0013-use-request-local-candidate-handles-for-semantic-reconciliation.md`
|
||||
using the repository's Nygard ADR format and `Status: Accepted`.
|
||||
- Cite ADR-0003, ADR-0004, ADR-0009, and ADR-0012 where their typed-artifact,
|
||||
package-boundary, evidence, and opaque-ID decisions apply.
|
||||
- Record these decisions:
|
||||
- semantic reconciliation is a framework mechanism used by typed normalize
|
||||
stage modules;
|
||||
- a model receives contextual candidate data but returns request-local
|
||||
one-based integer handles only;
|
||||
- the LLM proposes duplicate groups and a supplied canonical member;
|
||||
- deterministic code validates and applies the proposal;
|
||||
- a mandatory shared protocol is combined with an explicit generic or
|
||||
domain-owned semantic policy; and
|
||||
- durable IDs, copied contextual selectors, synthesized replacements,
|
||||
reflection-based arbitrary JSON, and hidden cross-stage behavior are
|
||||
rejected alternatives.
|
||||
- State explicitly that acceptance does not imply implementation completion
|
||||
and link to the feature and implementation roadmaps for status.
|
||||
- Do not modify the accepted decision text of ADR-0012.
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
- The ADR contains context, decision, alternatives, and consequences and does
|
||||
not claim that the new core already exists.
|
||||
- No current-behavior document outside `docs/adr/` or `docs/roadmap/` changes.
|
||||
- All ADR and roadmap links resolve.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
git diff --check
|
||||
```
|
||||
|
||||
## Stage 2: Build Bounded Candidate And Source-Context Preparation
|
||||
|
||||
### Goal
|
||||
|
||||
Create the domain-neutral candidate preparation boundary with contiguous
|
||||
request-local IDs and bounded prompt materials, without changing any D&D
|
||||
normalizer yet.
|
||||
|
||||
### Work
|
||||
|
||||
- Create `internal/framework/semanticreconcile` with package documentation and
|
||||
source-context preparation code adapted from
|
||||
`internal/modules/dnd/shared/entityreconcile/context.go`. Leave the old D&D
|
||||
package in place until all migrations finish.
|
||||
- Define a domain-neutral `Candidate` containing a contextual `Label` and owned
|
||||
generic `[]source.SourceRef`. Do not add a durable ID field.
|
||||
- Define `Limits` and one `DefaultLimits()` value with the fixed radius,
|
||||
candidate, and serialized-material bounds above. Validation rejects negative
|
||||
radius and non-positive maximums before doing work.
|
||||
- Prepare candidates in caller order:
|
||||
- validate every source reference against `source.DocumentIndex`;
|
||||
- exclude a candidate with no references or any invalid reference;
|
||||
- canonicalize its source-free ranges in source order and remove exact
|
||||
duplicate ranges;
|
||||
- assign IDs only to eligible candidates and make them contiguous from `1`;
|
||||
and
|
||||
- retain an internal ID-to-original-candidate-position mapping.
|
||||
- Do not exclude two candidates merely because their labels and ranges are
|
||||
identical. Their local integer IDs now disambiguate them.
|
||||
- Serialize candidate input as an object containing `candidates`, where each
|
||||
entry has `candidate_id`, `label`, and `source_refs` with only
|
||||
`start_unit_id` and `end_unit_id`.
|
||||
- Adapt the existing coalesced transcript-window algorithm. Preserve source
|
||||
order, cloned unit metadata, and the `cited` marker; do not mutate the source
|
||||
document or candidate references.
|
||||
- Return an explicit preparation disposition for:
|
||||
- ready materials;
|
||||
- fewer than two eligible candidates; and
|
||||
- candidate or combined-material limit exceeded.
|
||||
- Enforce the candidate limit before rendering large materials and the byte
|
||||
limit over the sum of serialized candidate and transcript material. Do not
|
||||
split the request.
|
||||
- Construct `contracts.LLMInputMaterial` values with stable content type,
|
||||
content digest, and the existing `candidates` and `transcript` input names.
|
||||
- Expose owned accessors for visible candidate mappings and materials. Do not
|
||||
expose mutable retained slices or maps.
|
||||
|
||||
### Tests
|
||||
|
||||
- Port and revise the valuable behavior tests from `entityreconcile`, replacing
|
||||
selector-collision expectations with contiguous-ID behavior.
|
||||
- Cover invalid and missing references, non-sequential source unit IDs,
|
||||
coalesced windows, candidate filtering, identical descriptors, nil source,
|
||||
input ownership, and deterministic serialization.
|
||||
- Use custom small limits to prove exactly-at-limit acceptance and one-over
|
||||
skipping for candidate count and material bytes. Do not assert the production
|
||||
default literals merely as change detectors.
|
||||
- Prove model material contains no source document ID or application entity ID.
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
- The package imports no D&D or production-module package.
|
||||
- Same-label and same-range candidates remain independently selectable.
|
||||
- No LLM client or response schema is introduced in this stage.
|
||||
- Existing D&D behavior remains unchanged and all prior tests still pass.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
go test ./internal/framework/semanticreconcile
|
||||
go test ./internal/modules/dnd/shared/entityreconcile
|
||||
git diff --check
|
||||
```
|
||||
|
||||
## Stage 3: Implement The Integer Proposal Contract And Assessor
|
||||
|
||||
### Goal
|
||||
|
||||
Add the private integer response schema and a pure deterministic assessor that
|
||||
turns model proposals into immutable, safely ordered reconciliation plans.
|
||||
|
||||
### Work
|
||||
|
||||
- Add core response types corresponding exactly to:
|
||||
- `duplicate_groups`;
|
||||
- each group's `candidate_ids`; and
|
||||
- `canonical_candidate_id`.
|
||||
- Add
|
||||
`assets/generic/normalize/deduplication/schemas/semantic_reconciliation_llm.v1.json`
|
||||
with Draft 2020-12 metadata and the fixed schema identity above.
|
||||
- Require all fields, reject unknown fields, require at least two group members,
|
||||
require positive integers, and use `uniqueItems: true` for member IDs. Keep
|
||||
application semantic validation authoritative.
|
||||
- Add schema loading metadata and helpers to the core package, following the
|
||||
existing `internal/framework/llm` response-schema pattern.
|
||||
- Implement assessment against the prepared request mapping:
|
||||
- reject zero, negative, or unknown member and canonical IDs;
|
||||
- reject repeated members even if schema validation was bypassed in a unit
|
||||
test;
|
||||
- reject groups with fewer than two distinct valid members;
|
||||
- reject a canonical ID that is not a valid member;
|
||||
- mark every otherwise-local-valid group sharing a member with another such
|
||||
group as conflicting and discard all conflicting groups;
|
||||
- retain independent safe groups even when another group is discarded; and
|
||||
- report stable issue categories and original response group indexes for
|
||||
retry diagnostics.
|
||||
- Produce an immutable plan expressed in original candidate positions rather
|
||||
than model-visible IDs. Sort members by original position and safe groups by
|
||||
earliest member so response ordering cannot alter deterministic output.
|
||||
- Provide owned accessors for plan groups, issues, discarded-group count, and
|
||||
whether retry is required.
|
||||
|
||||
### Tests
|
||||
|
||||
- Cover empty proposals, valid multi-member groups, unknown IDs, repeated IDs,
|
||||
canonical-not-member, too-small groups, overlapping groups, independent safe
|
||||
plus invalid groups, and response-order independence.
|
||||
- Compile and validate representative accepted and rejected JSON against the
|
||||
actual schema offline.
|
||||
- Prove returned plans and issues cannot mutate retained assessment state.
|
||||
- Add a focused fuzz test only if it remains small and protects the partition
|
||||
and no-panic invariants more efficiently than table cases.
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
- The model response types contain no contextual selectors or source ranges.
|
||||
- Invalid groups cannot enter the plan, and overlapping groups cannot be
|
||||
partially accepted.
|
||||
- Plan ordering depends only on deterministic candidate order.
|
||||
- The old D&D response schema remains temporarily available for unmigrated
|
||||
normalizers.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
go test ./internal/framework/semanticreconcile
|
||||
git diff --check
|
||||
```
|
||||
|
||||
## Stage 4: Add Generic Prompt Assets And Production Asset Registration
|
||||
|
||||
### Goal
|
||||
|
||||
Provide one complete generic default prompt, reusable mandatory protocol/input
|
||||
assets for domain prompts, and one production registration owner for the new
|
||||
prompt and schema.
|
||||
|
||||
### Work
|
||||
|
||||
- Replace the two draft files currently under
|
||||
`assets/generic/normalize/deduplication/` with a coherent asset set:
|
||||
- `prompts/prompt.yaml` for `generic.semantic_reconciliation` `v1`;
|
||||
- `prompts/system.md` with provider-neutral structured reconciliation role;
|
||||
- `prompts/protocol.md` with the mandatory integer selection and safety
|
||||
contract;
|
||||
- `prompts/instructions.md` with the conservative generic semantic default;
|
||||
- `prompts/candidates.md`; and
|
||||
- `prompts/transcript-windows.md`.
|
||||
- The default prompt declares required `candidates` and `transcript`
|
||||
`application/json` inputs, omits `default_profile`, selects the new schema,
|
||||
uses zero PromptKit repair attempts, and orders messages from stable to
|
||||
variable:
|
||||
1. generic system;
|
||||
2. mandatory protocol;
|
||||
3. generic semantic instructions, carrying the ephemeral cache marker for
|
||||
the stable prefix;
|
||||
4. candidate material; and
|
||||
5. transcript windows.
|
||||
- Keep protocol, semantic policy, and presentation as separate assets even
|
||||
though the rendered prompt is compact. Domain prompts must be able to reuse
|
||||
protocol and presentation while substituting only semantic instructions.
|
||||
- In `internal/framework/semanticreconcile`, add scoped asset loading,
|
||||
deterministic prompt/schema hashes, `RegisterAssets`, and a narrow allowlist
|
||||
helper that returns core-owned shared prompt files for a consuming manifest.
|
||||
The root `assets` package remains unchanged and contains no logic.
|
||||
- Extend `internal/modules/dnd/shared.PromptAssetManifest` to accept explicitly
|
||||
supplied external shared files in addition to named D&D shared files. Include
|
||||
those external files in both the mounted prompt filesystem and prompt hash.
|
||||
Reuse `promptfs.SharedPromptFile`; do not teach D&D shared code paths about a
|
||||
hard-coded generic directory.
|
||||
- Update `internal/modules/generic/register.Register` to require a non-nil asset
|
||||
registry and register the semantic-reconciliation assets before its existing
|
||||
modules and validators. Preserve contextual duplicate-registration errors.
|
||||
- Do not yet change the three D&D normalization prompts or remove their old
|
||||
schema registration.
|
||||
|
||||
### Tests
|
||||
|
||||
- Test the complete generic prompt and schema can be registered and prepared
|
||||
offline with an explicit test profile and representative integer candidate
|
||||
material.
|
||||
- Verify the prepared output contract selects the generic schema and that the
|
||||
rendered prompt asks for integer IDs rather than names or source ranges in
|
||||
the response. Do not snapshot all prose or its exact length.
|
||||
- Extend D&D manifest tests for external shared-file mounting, hashing, missing
|
||||
files, and duplicate destinations at the stable manifest boundary.
|
||||
- Update generic registrar tests for nil assets, successful asset registration,
|
||||
and duplicate registration without duplicating PromptKit internals.
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
- Generic production registration makes the complete default prompt and schema
|
||||
available before D&D registration.
|
||||
- Domain prompt manifests can mount the exact core protocol bytes without
|
||||
copying them.
|
||||
- The generic prompt has a stable prefix followed only by variable candidates
|
||||
and transcript material.
|
||||
- Existing D&D prompt behavior remains operational during the migration.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
go test ./internal/framework/semanticreconcile ./internal/framework/promptfs
|
||||
go test ./internal/modules/generic/register ./internal/modules/dnd/shared
|
||||
go test ./internal/cli -run 'Production|Catalog|Prompt'
|
||||
git diff --check
|
||||
```
|
||||
|
||||
## Stage 5: Add The Shared Structured-LLM Reconciliation Engine
|
||||
|
||||
### Goal
|
||||
|
||||
Centralize prompt execution and outcome classification behind one
|
||||
provider-neutral engine while leaving typed artifact mutation to later stages.
|
||||
|
||||
### Work
|
||||
|
||||
- Add an `Engine` in `internal/framework/semanticreconcile` constructed with a
|
||||
non-nil `contracts.StructuredLLMClient`, validated limits, and a prompt spec.
|
||||
A prompt spec contains a non-empty prompt ID, version, and SHA-256 digest; it
|
||||
may identify the generic default or a domain-owned prompt that obeys the same
|
||||
schema. Validate the digest at construction so manifest metadata and
|
||||
checkpoint fingerprints never silently omit the selected prompt bytes.
|
||||
- Define a request containing stage name, source document, candidates, resolved
|
||||
profile ID, and session ID. Preserve profile and session values exactly in
|
||||
`contracts.StructuredCompletionRequest`.
|
||||
- The engine must:
|
||||
- reject nil engine, client, context, and invalid construction state with
|
||||
contextual errors;
|
||||
- honor context cancellation before material preparation and completion;
|
||||
- use Stage 2 preparation and skip without calling the LLM for insufficient
|
||||
or over-limit inputs;
|
||||
- call `CompleteStructured` exactly once for a ready request, using only the
|
||||
`candidates` and `transcript` materials;
|
||||
- classify `contracts.ErrInvalidStructuredOutput` as a retryable semantic
|
||||
outcome rather than a transport error;
|
||||
- return other completion failures as errors with stage context;
|
||||
- assess a decoded response through Stage 3; and
|
||||
- return a safe plan, stable issues, discarded count, and a disposition that
|
||||
distinguishes complete, retryable invalid structured output, retryable
|
||||
discarded proposal groups, insufficient candidates, and deterministic
|
||||
limit skip.
|
||||
- Keep normalize-stage reason codes, warning text, and `NormalizeRetry`
|
||||
construction out of the engine. The engine exposes neutral classifications
|
||||
that typed normalizers translate through their existing policies.
|
||||
- Expose core manifest metadata and checkpoint fingerprints for prompt ID,
|
||||
version and hash, schema identity and hash, core reconciliation policy, and
|
||||
complete limits. Domain normalizers append their identity and normalization
|
||||
policy fingerprints rather than rebuilding core metadata independently.
|
||||
- Fingerprint the limit policy as one canonical core-owned value so any limit
|
||||
change invalidates relevant normalization checkpoints without test suites
|
||||
duplicating every literal.
|
||||
|
||||
### Tests
|
||||
|
||||
- Use a compact recording fake client to cover request propagation, successful
|
||||
assessment, empty groups, invalid structured output, transport failure,
|
||||
cancellation, insufficient candidates, candidate limit, and material limit.
|
||||
- Prove no LLM call occurs for deterministic skip outcomes.
|
||||
- Prove repeated calls do not retain plans, issues, candidate mappings, or
|
||||
warnings from earlier calls.
|
||||
- Test metadata and fingerprints relationally: required categories must be
|
||||
present and change when supplied prompt/schema/limit policy changes; do not
|
||||
freeze current digest values.
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
- Consuming modules no longer need to construct the common structured
|
||||
completion request once migrated.
|
||||
- The engine has no dependency on pipeline retry orchestration or a D&D type.
|
||||
- Provider and transport behavior remain behind `StructuredLLMClient`.
|
||||
- The old normalizers remain unchanged and compiling.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
go test ./internal/framework/semanticreconcile
|
||||
go test ./internal/framework/llm ./internal/framework/contracts
|
||||
git diff --check
|
||||
```
|
||||
|
||||
## Stage 6: Add Generic Typed Plan Application
|
||||
|
||||
### Goal
|
||||
|
||||
Centralize safe partition application, ordering, and provenance bookkeeping for
|
||||
typed records without moving domain consolidation rules into the core.
|
||||
|
||||
### Work
|
||||
|
||||
- Add a generic record envelope that owns a typed value, sorted unique original
|
||||
input indexes, and earliest deterministic input position. Constructors and
|
||||
accessors must preserve caller ownership.
|
||||
- Add a deliberately small typed application policy with these responsibilities:
|
||||
- clone one typed value;
|
||||
- optionally reject an otherwise safe semantic group through a stable neutral
|
||||
category; and
|
||||
- consolidate owned member values using one supplied canonical member.
|
||||
- Implement plan application that:
|
||||
- validates plan positions against the provided record slice;
|
||||
- never mutates records or policy inputs;
|
||||
- preserves every ungrouped record as an owned clone;
|
||||
- emits exactly one record for an accepted group;
|
||||
- preserves all members unchanged when the typed guard rejects a group;
|
||||
- unions and sorts original input indexes in core bookkeeping;
|
||||
- sets the resulting earliest position to the earliest member;
|
||||
- places output by earliest contributing position; and
|
||||
- returns neutral applied-group and rejected-group events containing the
|
||||
provenance needed for domain warning and retry construction.
|
||||
- Treat a policy consolidation error as an execution error unless it is the
|
||||
explicit safe-group rejection result. Do not silently preserve records after
|
||||
an unexpected application failure.
|
||||
- Do not make the helper aware of source references, names, durable IDs,
|
||||
currencies, people, or locations. Domain consolidators continue to union and
|
||||
canonicalize evidence inside their typed value and derive the final ID.
|
||||
|
||||
### Tests
|
||||
|
||||
- Use a minimal synthetic typed record to cover no groups, one and multiple
|
||||
groups, ungrouped preservation, canonical member selection, output ordering,
|
||||
provenance union, guard rejection, malformed plan positions, consolidation
|
||||
failure, nil versus empty ownership, and non-mutation.
|
||||
- Prefer table-driven invariant tests over tests for private loops or callback
|
||||
invocation order.
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
- A future domain adapter can apply a plan without reimplementing partition
|
||||
traversal or provenance ordering.
|
||||
- Rejected groups preserve all members exactly once.
|
||||
- The helper cannot derive a domain ID or emit a domain warning.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
go test ./internal/framework/semanticreconcile
|
||||
git diff --check
|
||||
```
|
||||
|
||||
## Stage 7: Migrate NPC Registry Normalization
|
||||
|
||||
### Goal
|
||||
|
||||
Use the shared engine and typed application path for NPC reconciliation while
|
||||
preserving the existing NPC artifact and failure semantics.
|
||||
|
||||
### Work
|
||||
|
||||
- Update the NPC normalization prompt manifest and assets:
|
||||
- keep prompt ID `dnd.npc_registry.normalize` and version `v1`;
|
||||
- retain the D&D system prompt and NPC-owned semantic instructions;
|
||||
- mount the generic protocol, candidate presentation, and transcript-window
|
||||
assets from `semanticreconcile`;
|
||||
- order messages as D&D system, generic protocol, NPC semantic instructions
|
||||
with the stable-prefix cache marker, candidates, then transcript windows;
|
||||
and
|
||||
- select `semantic_reconciliation_llm.v1.json`.
|
||||
- Remove requirements for the model to copy names or ranges. Adjust NPC policy
|
||||
prose to describe integer candidate selection while retaining all current
|
||||
individual-person and canonical-name distinctions.
|
||||
- Construct a shared engine in `npcregistry.New`, using the NPC prompt spec and
|
||||
default limits. Replace direct `BuildContext`, `CompleteStructured`, and
|
||||
`Assessment` orchestration with one engine call.
|
||||
- Convert deterministic preprocessed NPC records into core typed envelopes and
|
||||
candidates. Keep exact deterministic name/evidence normalization before the
|
||||
semantic pass.
|
||||
- Implement the narrow NPC typed policy:
|
||||
- no additional group guard;
|
||||
- choose the canonical member's normalized NPC name;
|
||||
- union and source-order all member references;
|
||||
- derive the final NPC ID through `npcs/identity`; and
|
||||
- return an owned value.
|
||||
- Translate engine and application outcomes to existing NPC warnings and retry
|
||||
behavior. Safe independent groups may appear in the retry candidate result;
|
||||
invalid structured output retries from the deterministic value; limit skip
|
||||
returns the deterministic value with one bounded reconciliation-exhausted
|
||||
warning and no retry.
|
||||
- Preserve existing NPC reason-code strings and warning cap. Bump only the
|
||||
normalization policy to `v5`.
|
||||
- Replace locally assembled common prompt/schema/context metadata and
|
||||
fingerprints with the core-provided values plus NPC identity and
|
||||
normalization policy. Retain user-useful manifest field names where practical
|
||||
but ensure the new generic schema identity and complete limits are recorded.
|
||||
- Delete NPC-only key translation, safe-group traversal, and duplicated generic
|
||||
application helpers made obsolete by the core. Retain NPC-specific
|
||||
preprocessing, consolidation policy, and diagnostics.
|
||||
|
||||
### Tests
|
||||
|
||||
- Rewrite fake LLM responses to return integer IDs and inspect candidate input
|
||||
only where needed to prove contiguous IDs and no durable identity leakage.
|
||||
- Preserve behavior coverage for exact deterministic consolidation, semantic
|
||||
aliases, canonical name choice, evidence union, ID recomputation, warnings,
|
||||
invalid proposal retry, exhaustion fallback, cancellation, ownership, and
|
||||
no-call cases.
|
||||
- Add focused coverage for identical contextual NPC descriptors remaining
|
||||
independently addressable and for a limit skip producing no LLM call.
|
||||
- Update prompt preparation tests for the generic schema and shared protocol
|
||||
without snapshotting full prompt prose or exact message length.
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
- NPC normalization imports `semanticreconcile` and no longer imports the old
|
||||
D&D `entityreconcile` package.
|
||||
- Its model response requires only integer candidate handles.
|
||||
- Durable NPC output and module key remain unchanged.
|
||||
- Focused tests demonstrate behavior parity plus the safer selection protocol.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
go test ./internal/modules/dnd/normalize/npcregistry
|
||||
go test ./internal/framework/semanticreconcile
|
||||
go test ./internal/modules/dnd/register -run 'NPC|Prompt|Schema'
|
||||
git diff --check
|
||||
```
|
||||
|
||||
## Stage 8: Migrate Item Registry Normalization
|
||||
|
||||
### Goal
|
||||
|
||||
Move item registry reconciliation onto the shared path while preserving its
|
||||
item-type and currency safety rules.
|
||||
|
||||
### Work
|
||||
|
||||
- Update the item normalization prompt exactly as in Stage 7, retaining prompt
|
||||
ID `dnd.item_registry.normalize`, version `v1`, and item-owned semantic policy
|
||||
while selecting the generic protocol, presentation assets, and schema.
|
||||
- Construct and use the shared engine with default limits. Convert deterministic
|
||||
item records into core candidates and typed envelopes.
|
||||
- Implement the item typed policy:
|
||||
- retain the existing currency-denomination group guard;
|
||||
- reject a group mixing currency with non-currency or different
|
||||
denominations through the core's explicit typed-group rejection path;
|
||||
- choose the canonical member's normalized item name;
|
||||
- union and source-order evidence;
|
||||
- derive the item ID through `items/identity`; and
|
||||
- preserve all current item-type and unique-designation semantics.
|
||||
- Count a typed currency guard rejection with discarded model groups for retry
|
||||
and exhaustion diagnostics. Preserve rejected group members unchanged and
|
||||
exactly once.
|
||||
- Translate insufficient, limit, invalid structured, invalid proposal, valid,
|
||||
and transport outcomes consistently with NPC behavior while retaining item
|
||||
reason codes and warning cap.
|
||||
- Bump the item normalization policy to `v3`, adopt core metadata and
|
||||
fingerprints, and remove item copies of generic key translation, group
|
||||
traversal, and application logic.
|
||||
|
||||
### Tests
|
||||
|
||||
- Convert semantic response fixtures to integer IDs.
|
||||
- Retain coverage for item aliases, canonical names, evidence, IDs, warnings,
|
||||
retries, fallback, ownership, and no-call cases.
|
||||
- Specifically protect same-denomination alias acceptance, cross-denomination
|
||||
rejection, currency/non-currency rejection, preservation of every rejected
|
||||
member, and combination of an accepted independent group with a rejected
|
||||
group.
|
||||
- Update prompt/schema preparation tests without exact prompt snapshots.
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
- Item normalization no longer imports old `entityreconcile` code.
|
||||
- Its domain guard is expressed only through the typed policy and cannot be
|
||||
bypassed by a schema-valid proposal.
|
||||
- Durable item output and public module behavior remain unchanged.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
go test ./internal/modules/dnd/normalize/itemregistry
|
||||
go test ./internal/framework/semanticreconcile
|
||||
go test ./internal/modules/dnd/register -run 'Item|Prompt|Schema'
|
||||
git diff --check
|
||||
```
|
||||
|
||||
## Stage 9: Migrate Location Registry Normalization
|
||||
|
||||
### Goal
|
||||
|
||||
Complete adoption by moving location registry reconciliation onto the shared
|
||||
engine and typed application path.
|
||||
|
||||
### Work
|
||||
|
||||
- Update the location normalization prompt as in Stages 7 and 8, retaining
|
||||
prompt ID `dnd.location_registry.normalize`, version `v1`, and the
|
||||
location-owned physical-place semantic policy.
|
||||
- Construct and use the shared engine with default limits. Convert deterministic
|
||||
location records into core candidates and typed envelopes.
|
||||
- Implement the location typed policy:
|
||||
- no additional group guard beyond existing location semantic policy;
|
||||
- choose the supplied canonical member's normalized location name;
|
||||
- union and source-order all member references;
|
||||
- derive the location ID from the final name and final references through
|
||||
`locations/identity`; and
|
||||
- preserve same-name, parent/child, and physical-place distinctions.
|
||||
- Translate outcomes through existing location retry, fallback, reason-code,
|
||||
and warning-cap behavior. Bump the normalization policy to `v3`.
|
||||
- Adopt core metadata and fingerprints and delete location copies of generic
|
||||
reconciliation and application mechanics.
|
||||
|
||||
### Tests
|
||||
|
||||
- Convert semantic response fixtures to integer IDs while retaining physical
|
||||
place, same-name, canonical-name, evidence-dependent ID, warning, retry,
|
||||
fallback, ownership, and no-call coverage.
|
||||
- Prove same-name records with distinct evidence can both be addressed by IDs
|
||||
without copied selectors.
|
||||
- Update prompt/schema preparation tests without freezing prompt prose.
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
- All three semantic D&D registry normalizers use the shared core.
|
||||
- Location durable IDs are still derived only after final evidence union.
|
||||
- No location code imports old `entityreconcile`.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
go test ./internal/modules/dnd/normalize/locationregistry
|
||||
go test ./internal/framework/semanticreconcile
|
||||
go test ./internal/modules/dnd/register -run 'Location|Prompt|Schema'
|
||||
git diff --check
|
||||
```
|
||||
|
||||
## Stage 10: Remove The Legacy Reconciliation Path And Consolidate Registration
|
||||
|
||||
### Goal
|
||||
|
||||
Delete superseded selector-copying code and assets, leave one schema and core,
|
||||
and verify production composition as a whole.
|
||||
|
||||
### Work
|
||||
|
||||
- Delete `internal/modules/dnd/shared/entityreconcile` after confirming no
|
||||
imports remain.
|
||||
- Delete `assets/dnd/entity-reconciliation/` and
|
||||
`assets/dnd/shared/prompts/common-dnd-entity-reconciliation.md`.
|
||||
- Remove the obsolete shared prompt-name entry and the D&D registrar's old
|
||||
entity-reconciliation schema registration.
|
||||
- Remove any obsolete local candidate or transcript template files from the
|
||||
three normalization asset trees after their manifests use the shared generic
|
||||
presentation assets. Keep only their `prompt.yaml` and domain semantic
|
||||
`instructions.md` unless another file remains genuinely domain-specific.
|
||||
- Ensure the generic registrar is the sole production owner of generic
|
||||
reconciliation prompt/schema registration and that the existing production
|
||||
registrar order remains generic, Seriatim, D&D.
|
||||
- Update registrar and CLI composition tests so they prove assembled production
|
||||
assets are complete. A D&D registrar unit test may pre-register required
|
||||
generic assets or limit itself to D&D-owned assets; do not duplicate generic
|
||||
asset ownership inside D&D merely to preserve an old isolated-test setup.
|
||||
- Search all Go, asset, and roadmap-adjacent current documentation for old
|
||||
schema keys, selector response types, `common-dnd-entity-reconciliation.md`,
|
||||
and model-output source ranges. Remove only obsolete uses; source ranges
|
||||
remain valid model input.
|
||||
- Review the three normalizers side by side. Consolidate any remaining
|
||||
demonstrated neutral retry/adaptation helper into the core if it can be done
|
||||
without domain warning semantics; otherwise keep the difference explicit.
|
||||
|
||||
### Tests
|
||||
|
||||
- Run the complete framework, generic module, D&D module, D&D registrar, and
|
||||
production composition suites.
|
||||
- Add or retain one assembled prompt/schema test per meaningful owner rather
|
||||
than reproducing every core schema case in all three domains.
|
||||
- Confirm duplicate production registration still fails contextually and no
|
||||
prompt or schema path collides.
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
- Exactly one semantic reconciliation response schema and assessor exist.
|
||||
- No model-facing reconciliation response requires a name or evidence range.
|
||||
- Production composition registers every selected D&D normalization prompt and
|
||||
the generic schema exactly once.
|
||||
- No compatibility shim or dead legacy asset remains.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
go test ./internal/framework/...
|
||||
go test ./internal/modules/generic/...
|
||||
go test ./internal/modules/dnd/...
|
||||
go test ./internal/cli -run 'Production|Catalog|Prompt|Schema'
|
||||
rg -n 'dnd_entity_reconcile_llm|common-dnd-entity-reconciliation|entityreconcile' internal assets docs/internal docs/policy docs/config.md
|
||||
git diff --check
|
||||
```
|
||||
|
||||
The `rg` command should return no live legacy references. Separately review
|
||||
roadmap and ADR historical references rather than deleting accurate decision
|
||||
history.
|
||||
|
||||
## Stage 11: Finalize Current Documentation And Repository Verification
|
||||
|
||||
### Goal
|
||||
|
||||
Make current documentation accurately describe the implemented architecture,
|
||||
then perform the complete offline verification pass.
|
||||
|
||||
### Work
|
||||
|
||||
- Update `docs/policy/architecture.md` to:
|
||||
- distinguish an artifact family from one configured stage module without
|
||||
changing the fixed pipeline;
|
||||
- identify semantic reconciliation as a domain-neutral framework mechanism;
|
||||
- state that request-local candidate handles are an approved application of
|
||||
ADR-0012 and link ADR-0013; and
|
||||
- retain typed domain mutation, evidence, and dependency-direction
|
||||
invariants.
|
||||
- Update `docs/internal/overview.md` with the implemented
|
||||
`internal/framework/semanticreconcile` responsibility, linking to the focused
|
||||
internal owners rather than duplicating mechanics.
|
||||
- Update `docs/internal/modules.md` to document artifact-family ownership,
|
||||
stage-module registration, and how a typed normalizer instantiates the shared
|
||||
strategy.
|
||||
- Update `docs/internal/dnd.md` to replace selector-copying descriptions with
|
||||
the integer candidate protocol, generic protocol/domain policy composition,
|
||||
limit behavior, typed application boundary, and the three registry-specific
|
||||
rules. Keep exact public keys and validator chains in `docs/config.md` rather
|
||||
than duplicating them.
|
||||
- Update `docs/internal/llm.md` only as needed to identify ownership and
|
||||
registration of the generic prompt/schema assets and checkpoint metadata.
|
||||
- Do not change integration contracts unless repository inspection finds an
|
||||
incorrect statement: the durable artifact shapes are intentionally
|
||||
unchanged. Do not add user configuration documentation because this feature
|
||||
adds no configuration.
|
||||
- Confirm ADR-0013 accurately matches the final implementation. Do not rewrite
|
||||
its accepted decision to accommodate accidental implementation drift; fix
|
||||
the implementation or create a superseding decision if a genuine conflict
|
||||
was discovered.
|
||||
- Review `docs/roadmap/future.md` only for links and scope boundaries. Keep
|
||||
batching, operator-selected policy, broader inputs, a universal module key,
|
||||
and physical package reorganization deferred.
|
||||
- Run formatting, tests, vet, build, whitespace, link, and stale-reference
|
||||
checks. No command may contact a live LLM provider.
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
- Current docs describe only implemented behavior and assign each fact to its
|
||||
canonical owner.
|
||||
- The architecture clearly separates artifact families, stage modules, shared
|
||||
model judgment, and typed deterministic application.
|
||||
- All durable D&D schemas, module keys, and examples remain valid.
|
||||
- No stale selector-copying documentation or asset path remains.
|
||||
- The repository-wide test, vet, and build checks pass offline.
|
||||
|
||||
### Validation
|
||||
|
||||
```sh
|
||||
go test ./...
|
||||
go vet ./...
|
||||
go build ./cmd/notarius
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Run `gofmt` on every changed Go file before these commands; do not pass package
|
||||
directories to `gofmt`.
|
||||
|
||||
Also verify every changed Markdown link target manually or with the repository's
|
||||
available link checker. Review `git status --short` and the complete diff to
|
||||
confirm that no unrelated file or generated secret-bearing material was added.
|
||||
426
docs/roadmap/semantic-reconciliation.md
Normal file
426
docs/roadmap/semantic-reconciliation.md
Normal file
@@ -0,0 +1,426 @@
|
||||
# Semantic Reconciliation Roadmap
|
||||
|
||||
## Purpose
|
||||
|
||||
This roadmap defines a reusable, LLM-assisted semantic-reconciliation facility
|
||||
for source-backed entity registries. The facility will centralize the common
|
||||
candidate preparation, prompt execution, proposal validation, safety, retry,
|
||||
and consolidation mechanics currently implemented by the D&D NPC, item, and
|
||||
location registry normalizers while preserving typed, domain-owned output.
|
||||
|
||||
The model will remain a constrained proposal source. Notarius will retain
|
||||
authority over candidate identity, proposal validation, deterministic mutation,
|
||||
provenance, durable identifiers, warnings, and final artifact construction.
|
||||
|
||||
## Motivation
|
||||
|
||||
NPC, item, and location registry normalization now demonstrate the same useful
|
||||
pattern: deterministic preprocessing produces candidate records, an LLM judges
|
||||
whether some candidates describe the same underlying entity, and deterministic
|
||||
code applies only safe proposed groups. The current D&D-shared implementation
|
||||
proves the approach, but it still duplicates orchestration and application
|
||||
logic across normalizers and requires the model to reproduce complete
|
||||
contextual selectors containing names and evidence ranges.
|
||||
|
||||
The target design should provide one efficient and thoroughly tested
|
||||
reconciliation core that can support additional artifact families without
|
||||
moving domain semantics into generic code. It should also simplify the model's
|
||||
task by replacing selector reproduction with small request-local integer
|
||||
handles.
|
||||
|
||||
## Goals
|
||||
|
||||
- Establish one domain-neutral semantic-reconciliation core for source-backed
|
||||
entity candidates.
|
||||
- Move all demonstrated common mechanics into that core, including structured
|
||||
LLM execution and deterministic proposal assessment.
|
||||
- Preserve exact typed artifact ownership from merged input through normalized
|
||||
output.
|
||||
- Give the model semantic evidence while asking it to return only small,
|
||||
request-local candidate identifiers.
|
||||
- Provide a conservative generic semantic prompt policy that a typed artifact
|
||||
family may use by default.
|
||||
- Allow an artifact family to supply narrower domain semantic policy without
|
||||
replacing the mandatory shared protocol and safety instructions.
|
||||
- Have the NPC, item, and location registry normalizers use the shared core
|
||||
without changing their durable artifact contracts, configured module keys,
|
||||
domain identity rules, or fallback guarantees.
|
||||
- Make addition of another eligible registry normalizer primarily an adapter
|
||||
and policy exercise rather than a copy of reconciliation machinery.
|
||||
|
||||
## Terminology And Ownership
|
||||
|
||||
A **stage module** remains one configured implementation of one pipeline stage.
|
||||
An extractor and a normalizer are separate stage modules even when they
|
||||
collaborate on the same artifact kind.
|
||||
|
||||
An **artifact family** is the cohesive domain feature that owns an artifact's
|
||||
types, codec, extractor, merge choice, normalizer, validators, prompt policy,
|
||||
schemas, identity helpers, and reference projections. For example, D&D spells
|
||||
and the D&D NPC registry are artifact families whose implementations span
|
||||
multiple explicit pipeline stages.
|
||||
|
||||
This terminology clarifies existing ownership without changing Notarius's
|
||||
fixed pipeline or combining stages. It is consistent with the accepted
|
||||
domain-first organization and typed generic-strategy boundary in
|
||||
[ADR-0004](../adr/0004-package-modules-by-domain.md). The D&D registrar remains
|
||||
responsible for composing D&D-owned stage modules and their policies; the
|
||||
generic core remains unaware of D&D types or semantics.
|
||||
|
||||
## Target Architecture
|
||||
|
||||
### Shared Core
|
||||
|
||||
A domain-neutral framework package should own semantic reconciliation. It may
|
||||
depend on generic source and structured-completion contracts and may consume
|
||||
its scoped assets under `assets/generic/`, but it must not import a production
|
||||
domain or encode D&D identity rules.
|
||||
|
||||
The shared core owns:
|
||||
|
||||
- validation and defensive copying of its inputs;
|
||||
- candidate eligibility and stable input ordering;
|
||||
- assignment of model-visible request-local candidate IDs;
|
||||
- construction of bounded candidate and source-context materials;
|
||||
- invocation of the configured structured LLM prompt;
|
||||
- the private duplicate-group response schema;
|
||||
- exact resolution of response IDs to the candidates visible in that request;
|
||||
- rejection of malformed, unknown, repeated, ambiguous, or overlapping
|
||||
proposals;
|
||||
- canonical ordering of accepted groups and group members;
|
||||
- construction of an immutable reconciliation plan;
|
||||
- shared retry, fallback, cancellation, and invalid-output classification;
|
||||
- generic consolidation and provenance mechanics that are demonstrably common
|
||||
across typed consumers; and
|
||||
- metadata and checkpoint fingerprints for shared prompts, schemas, and policy
|
||||
versions.
|
||||
|
||||
The core should expose stable behavior rather than a collection of unrelated
|
||||
helpers. Its API should make the safe path direct: a caller supplies typed
|
||||
candidates and a narrow domain policy, and receives either a deterministic
|
||||
typed result or a validated reconciliation plan that can only be applied
|
||||
through the typed policy boundary.
|
||||
|
||||
### Typed Artifact-Family Adapter
|
||||
|
||||
Each consuming artifact family owns a typed adapter or policy that supplies the
|
||||
irreducibly domain-specific behavior:
|
||||
|
||||
- projection of merged typed records into eligible reconciliation candidates;
|
||||
- the semantic identity scope being reconciled;
|
||||
- an optional domain semantic-policy prompt asset;
|
||||
- canonical-field selection beyond choosing the supplied canonical member;
|
||||
- domain field and evidence consolidation rules;
|
||||
- durable ID derivation;
|
||||
- domain warning scopes, reason codes, and messages; and
|
||||
- domain postconditions and fallback behavior not covered by the shared core.
|
||||
|
||||
The adapter must not parse untyped durable JSON or weaken the artifact codec's
|
||||
exact Go type. The core must not use reflection to infer domain fields. Shared
|
||||
typed consolidation support may use Go generics and a deliberately small policy
|
||||
interface where that removes demonstrated duplication without hiding domain
|
||||
rules.
|
||||
|
||||
### Pipeline Boundary
|
||||
|
||||
Semantic reconciliation remains an implementation of the normalize stage. It
|
||||
does not add a pipeline stage, hide normalization inside extraction, or create
|
||||
an arbitrary workflow edge. Existing typed normalizer registrations remain the
|
||||
public configuration boundary.
|
||||
|
||||
The initial implementation does not register one universal
|
||||
`generic/deduplication` module key. A module key cannot safely accept arbitrary
|
||||
artifact kinds under the current exact typed-registration contract. Domain
|
||||
registrars instead instantiate the shared strategy for their own artifact
|
||||
types.
|
||||
|
||||
## Request-Local Candidate Protocol
|
||||
|
||||
### Candidate Presentation
|
||||
|
||||
After deterministic preprocessing and eligibility filtering, the core assigns
|
||||
the candidates visible to one completion request contiguous integer
|
||||
`candidate_id` values beginning with `1`. The mapping is owned by that request
|
||||
and retains the original typed candidate and input position internally.
|
||||
|
||||
Each model-facing candidate includes its `candidate_id`, contextual display
|
||||
name or label, and the evidence and context needed for semantic judgment. In
|
||||
the initial source-backed implementation, that context consists of validated
|
||||
source-reference ranges and bounded, source-ordered transcript windows.
|
||||
Application-owned durable entity IDs are never included.
|
||||
|
||||
Candidate IDs:
|
||||
|
||||
- are identifiers for prompt selections, not entity identities;
|
||||
- have meaning only within one structured completion request;
|
||||
- restart for each request or future batch;
|
||||
- never enter a durable artifact or public schema;
|
||||
- are not used to derive durable IDs; and
|
||||
- must be resolved through the core's retained request-local mapping.
|
||||
|
||||
Only model-visible eligible candidates receive IDs. Filtering must not produce
|
||||
gaps that increase model burden or reveal unrelated internal ordering.
|
||||
|
||||
### Proposal Response
|
||||
|
||||
The private structured response contains an ordered `duplicate_groups` array.
|
||||
Each group contains:
|
||||
|
||||
- `candidate_ids`: at least two distinct supplied integer IDs; and
|
||||
- `canonical_candidate_id`: one supplied ID that is also a member of that
|
||||
group.
|
||||
|
||||
The model does not return candidate names, source ranges, replacement records,
|
||||
durable IDs, or synthesized canonical values. Choosing a canonical candidate
|
||||
means selecting one supplied member; typed deterministic code constructs the
|
||||
resulting record.
|
||||
|
||||
The response schema should reject unknown fields and require every defined
|
||||
field. Application validation remains authoritative and must additionally
|
||||
reject out-of-range IDs, repeated members, a canonical ID outside its group,
|
||||
and any candidate appearing in more than one group. Schema validation is not a
|
||||
substitute for these semantic checks.
|
||||
|
||||
### Deterministic Plan
|
||||
|
||||
The core resolves accepted IDs to internal candidates, orders members by their
|
||||
original deterministic positions, and orders groups by their earliest member.
|
||||
Candidates omitted from the response remain distinct. Invalid groups are never
|
||||
partially applied.
|
||||
|
||||
The reconciliation plan retains enough internal provenance for typed
|
||||
application, warning generation, debugging, and validation without exposing
|
||||
request-local IDs as durable identity.
|
||||
|
||||
## Prompt Policy
|
||||
|
||||
### Mandatory Shared Protocol
|
||||
|
||||
The core owns a shared prompt fragment that defines the response protocol and
|
||||
non-negotiable safety behavior. It instructs the model to:
|
||||
|
||||
- identify only well-supported groups that denote the same underlying entity;
|
||||
- preserve candidates that are merely similar or uncertain;
|
||||
- return only supplied candidate IDs;
|
||||
- select one supplied group member as canonical;
|
||||
- omit uncertain groups; and
|
||||
- invent no candidates, evidence, attributes, identities, or replacements.
|
||||
|
||||
This protocol fragment and the private schema are not replaceable by a domain
|
||||
adapter. Keeping them shared ensures identical mechanics across consumers and
|
||||
provides one prompt prefix for review, testing, and provider caching.
|
||||
|
||||
### Generic Semantic Default
|
||||
|
||||
The core provides a conservative generic semantic-policy fragment suitable for
|
||||
an artifact family whose notion of entity identity is adequately conveyed by
|
||||
its candidate labels and evidence. It asks whether candidates refer to the same
|
||||
underlying entity and treats uncertainty as a reason not to collapse them.
|
||||
|
||||
Use of the generic default is explicit in the typed adapter. It is not an
|
||||
implicit fallback for an adapter that failed to declare its policy.
|
||||
|
||||
### Domain Semantic Policy
|
||||
|
||||
An artifact family may select a domain-owned semantic-policy fragment in place
|
||||
of the generic semantic fragment. The replacement defines only domain judgment
|
||||
and canonical-member preferences; it does not replace the shared protocol,
|
||||
response schema, or deterministic safety rules.
|
||||
|
||||
The existing NPC, item, and location policies remain domain-owned because they
|
||||
encode meaningful distinctions among people, item types or unique
|
||||
designations, currency denominations, parent and child places, and same-name
|
||||
physical locations. Their prompt manifests should select the shared protocol
|
||||
and their local semantic fragment while using the same generic response
|
||||
schema.
|
||||
|
||||
Prompt selection is initially an implementation-time artifact-family choice.
|
||||
This work does not add arbitrary operator-supplied prompt paths or configuration
|
||||
that can replace reconciliation safety policy.
|
||||
|
||||
## Context, Bounds, And Model Invocation
|
||||
|
||||
The first shared core is intentionally scoped to candidates grounded in a
|
||||
Notarius source document. It validates candidate source references, creates
|
||||
bounded windows around their evidence, preserves source order, and supplies
|
||||
the model with the relationship between each candidate ID and its evidence.
|
||||
Source ranges are model input but are never model output.
|
||||
|
||||
The core skips the LLM call when fewer than two candidates remain eligible. It
|
||||
must also impose explicit candidate-count and rendered-context bounds. When an
|
||||
input cannot be reconciled safely within those bounds, normalization preserves
|
||||
the deterministic preprocessed result and produces bounded diagnostics under
|
||||
the consuming module's established fallback policy. It must not silently
|
||||
process arbitrary fixed-size slices that could separate duplicates.
|
||||
|
||||
Every completion uses the injected scheduled structured-LLM client, propagated
|
||||
profile and session ID, cancellation, and the normalizer retry contract.
|
||||
Invalid structured output or an unusable proposal follows the existing
|
||||
retry-then-deterministic-fallback model. Transport and provider failures remain
|
||||
execution errors rather than being silently converted into semantic absence.
|
||||
|
||||
## Deterministic Typed Application
|
||||
|
||||
The model never mutates the artifact. The shared core and typed adapter apply
|
||||
only fully validated groups.
|
||||
|
||||
Application must preserve these invariants:
|
||||
|
||||
- no ungrouped candidate is inserted, removed, or changed by semantic
|
||||
reconciliation;
|
||||
- every accepted group produces exactly one typed output record;
|
||||
- canonical display fields come from a supplied group member unless a domain
|
||||
policy explicitly performs a deterministic transformation;
|
||||
- all required provenance from group members is retained and canonicalized;
|
||||
- output order follows the earliest contributing deterministic input position;
|
||||
- durable IDs are recomputed by the domain identity policy after
|
||||
consolidation;
|
||||
- caller-owned input and request material are never mutated;
|
||||
- warnings identify every collapsed group using domain-owned scopes and reason
|
||||
codes; and
|
||||
- warning volume remains bounded.
|
||||
|
||||
Any shared typed application helper must make these invariants structural while
|
||||
leaving domain field merging and ID derivation explicit.
|
||||
|
||||
## Target D&D Consumers
|
||||
|
||||
### NPC Registry
|
||||
|
||||
NPC normalization uses the shared core while preserving its individual-person
|
||||
identity semantics, proper-name canonicalization policy, evidence union,
|
||||
deterministic NPC ID derivation, warning behavior, retry, and fallback result.
|
||||
Its model response uses integer candidate selection rather than copied
|
||||
contextual selectors.
|
||||
|
||||
### Item Registry
|
||||
|
||||
Item normalization uses the shared core while preserving the distinction
|
||||
between item types and unique designations, distinct currency denominations,
|
||||
non-inference of item properties or uniqueness, item identity derivation,
|
||||
warnings, and fallback behavior.
|
||||
|
||||
### Location Registry
|
||||
|
||||
Location normalization uses the shared core while preserving its
|
||||
physical-place identity semantics, treatment of parent and child places and
|
||||
same-name places, source-reference-dependent durable IDs, warnings, and
|
||||
fallback behavior.
|
||||
|
||||
The migrations must retain the existing durable artifact schemas and public
|
||||
module keys. Prompt, schema, policy, and implementation fingerprint changes
|
||||
must invalidate only the affected normalization checkpoints through the normal
|
||||
checkpoint identity mechanism.
|
||||
|
||||
The target tree contains no superseded D&D-specific selector-copy response
|
||||
schema, shared reconciliation implementation, or obsolete prompt assets. It
|
||||
does not retain compatibility shims for the private pre-release LLM response
|
||||
contract.
|
||||
|
||||
## Verification Strategy
|
||||
|
||||
Tests should protect the reconciliation contract and realistic failure modes,
|
||||
not private helper structure or exact prompt length.
|
||||
|
||||
The shared core warrants focused behavioral coverage for:
|
||||
|
||||
- contiguous ID assignment after eligibility filtering;
|
||||
- absence of durable IDs from model inputs and absence of evidence ranges from
|
||||
model outputs;
|
||||
- exact ID resolution and rejection of zero, negative, unknown, repeated, and
|
||||
overlapping IDs;
|
||||
- canonical membership and minimum group size;
|
||||
- deterministic group and output ordering regardless of response order;
|
||||
- preservation of ungrouped candidates and complete provenance;
|
||||
- ownership and defensive-copy guarantees;
|
||||
- cancellation, invalid structured output, retry, and fallback behavior;
|
||||
- candidate and context bounds; and
|
||||
- offline prompt/schema registration and structured response decoding.
|
||||
|
||||
Use table-driven or property-oriented tests where they efficiently protect
|
||||
group-partition and preservation invariants. Fuzzing is appropriate for the
|
||||
pure proposal assessor if it remains fast and deterministic. Do not add tests
|
||||
that merely freeze prompt text, message counts, asset hashes, private constant
|
||||
values, or implementation call choreography.
|
||||
|
||||
Each migrated D&D normalizer retains focused tests for its domain identity,
|
||||
consolidation, warnings, durable IDs, and fallback behavior. A small number of
|
||||
integration tests should prove that production registration supplies the
|
||||
shared schema and selected prompt policy. Live-model evaluation remains a
|
||||
human review tool and is not part of the default offline test suite.
|
||||
|
||||
## Architectural Decisions And Documentation
|
||||
|
||||
The target documentation set includes an accepted ADR applying
|
||||
[ADR-0012](../adr/0012-resolve-opaque-entity-identifiers-deterministically.md)
|
||||
to semantic reconciliation. That decision records:
|
||||
|
||||
- request-local ordinal candidate handles as the standard reconciliation
|
||||
selection mechanism;
|
||||
- why requiring the model to reproduce contextual selectors is unnecessary
|
||||
and error-prone;
|
||||
- the mandatory shared protocol plus generic or domain semantic-policy
|
||||
composition;
|
||||
- LLM proposal versus deterministic application ownership;
|
||||
- the typed adapter boundary; and
|
||||
- rejected alternatives, including durable IDs, name-only selection,
|
||||
model-synthesized replacement records, and arbitrary untyped normalization.
|
||||
|
||||
ADR-0012's accepted decision text remains unchanged. The new ADR cites it and
|
||||
provides the concrete justification it requires for request-local labels.
|
||||
|
||||
In the target state, architecture and internal documentation:
|
||||
|
||||
- distinguish artifact families from configured stage modules;
|
||||
- identify the shared reconciliation core and its dependency direction;
|
||||
- document the generic prompt protocol and domain semantic-policy ownership;
|
||||
- document the integer candidate protocol and deterministic safety boundary;
|
||||
- describe the three migrated registry normalizers accurately; and
|
||||
- remove descriptions of contextual-selector response copying.
|
||||
|
||||
Until then, this roadmap remains the canonical description of the proposed
|
||||
behavior; current-behavior documents must not describe it as implemented.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
This work does not include:
|
||||
|
||||
- a new pipeline stage or a compound module that combines extraction and
|
||||
normalization;
|
||||
- a universal configured normalizer for arbitrary artifact kinds or untyped
|
||||
JSON;
|
||||
- a physical reorganization of every D&D package around artifact-family
|
||||
directories;
|
||||
- operator-configurable arbitrary prompt assets or replacement of core safety
|
||||
instructions;
|
||||
- naive batching, cross-batch clustering, or unbounded reconciliation inputs;
|
||||
- semantic deduplication of event artifacts whose identity dimensions are
|
||||
already handled deterministically;
|
||||
- alternate non-source context providers;
|
||||
- changes to durable D&D artifact schemas or public module keys; or
|
||||
- live provider calls in the default test suite.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
The feature is complete when:
|
||||
|
||||
- one domain-neutral core owns candidate IDs, source context, prompt execution,
|
||||
response assessment, retry classification, and common safe application
|
||||
mechanics;
|
||||
- the private response schema uses only integer candidate handles for member
|
||||
and canonical selection;
|
||||
- the generic safety protocol and conservative semantic default exist as
|
||||
shared assets;
|
||||
- domain adapters can explicitly select the generic semantic default or a
|
||||
domain-owned semantic fragment without replacing core safety behavior;
|
||||
- NPC, item, and location registry normalizers use the shared core and retain
|
||||
their typed domain behavior and durable contracts;
|
||||
- the model is not required to reproduce names, evidence ranges, durable IDs,
|
||||
or replacement records in reconciliation output;
|
||||
- invalid proposals cannot partially mutate or partially collapse an artifact;
|
||||
- oversized inputs preserve deterministic results rather than being naively
|
||||
divided;
|
||||
- the superseded D&D-specific response and reconciliation path is removed;
|
||||
- the new ADR and current-behavior documentation accurately reflect the
|
||||
implemented boundary; and
|
||||
- focused package tests and the repository-wide Go test and vet suites pass.
|
||||
Reference in New Issue
Block a user