2263 lines
162 KiB
Markdown
2263 lines
162 KiB
Markdown
# Codebase Audit
|
||
|
||
## Audit Metadata
|
||
|
||
- **Production target:** `92e89076a268089e703978fb9d7176200e93344c`
|
||
- **Branch at target:** `main`
|
||
- **Audit date:** 2026-08-08
|
||
- **Go version:** `go1.26.5 linux/amd64`
|
||
- **PromptKit version:** `gitea.maximumdirect.net/eric/promptkit v0.5.0`
|
||
- **Knowledge-graph project:** `notarius-audit-92e8907`
|
||
- **Knowledge-graph target:** branch `main`, head
|
||
`92e89076a268089e703978fb9d7176200e93344c`
|
||
- **Initial worktree:** Clean. There were no pre-existing production or roadmap
|
||
changes to record.
|
||
|
||
The commit above is the production snapshot under audit. Later commits that
|
||
change only roadmap audit documents do not change that production target.
|
||
|
||
## Executive Summary
|
||
|
||
Pending final synthesis. The initial baseline is healthy. The architecture,
|
||
configuration/CLI, pipeline composition, reference/handoff, runtime, state,
|
||
LLM, generic/Seriatim, shared D&D, registry-family, occurrence-family, and
|
||
spell/scene-family reviews have found three High findings, six Medium findings,
|
||
and twenty-three Low findings, with no production dependency inversion,
|
||
unbounded framework worker pool, completion-order-dependent result assembly,
|
||
debug-to-cache coupling, model-visible credential material in the embedded LLM
|
||
assets, or domain leakage across the Seriatim and generic module boundaries.
|
||
|
||
## Finding Index
|
||
|
||
Final cross-area ordering is pending synthesis.
|
||
|
||
| ID | Severity | Category | Title |
|
||
| --- | --- | --- | --- |
|
||
| ARCH-001 | Low | Documentation/Comments | Repair broken ADR cross-references |
|
||
| CFGCLI-001 | Medium | Correctness | Reject additional YAML documents |
|
||
| CFGCLI-002 | Low | Correctness | Reject a blank command-level LLM profile |
|
||
| PIPE-001 | Low | Correctness | Reject normalized module-reference collisions in the resolver |
|
||
| PIPE-002 | Low | Efficiency | Clone construction inputs once per builder boundary |
|
||
| REF-001 | Medium | Efficiency | Bound reference reads before allocating the file |
|
||
| REF-002 | Low | Efficiency | Index accepted outputs once per ordered handoff |
|
||
| REF-003 | Low | Correctness | Include canonical size in generated-reference fingerprints |
|
||
| RUN-001 | High | Correctness | Check cancellation at unguarded dispatch boundaries |
|
||
| RUN-002 | High | Correctness | Isolate typed validator values from stage output |
|
||
| RUN-003 | Medium | Correctness | Preserve warnings from the terminal rejected attempt |
|
||
| RUN-004 | Low | Documentation/Comments | Document the lane collector's liveness invariant |
|
||
| STATE-001 | Medium | Correctness | Preserve distinct identities in state paths |
|
||
| STATE-002 | Low | Efficiency | Remove redundant post-decode clones from canonical codecs |
|
||
| STATE-003 | Low | Duplication | Publish output through the confined file writer |
|
||
| LLM-001 | High | Correctness | Keep raw provider errors inside the adapter |
|
||
| LLM-002 | Low | Correctness | Recheck cancellation after scheduler admission |
|
||
| LLM-003 | Low | Correctness | Reject duplicate virtual prompt names |
|
||
| LLM-004 | Low | Duplication | Share the read-only in-memory filesystem mechanics |
|
||
| MOD-001 | Low | Simplicity | Narrow generic integer option decoding |
|
||
| MOD-002 | Low | Efficiency | Reuse compiled response schemas within a prepared validator |
|
||
| MOD-003 | Low | Simplicity | Remove unreachable JSON metadata clone helpers |
|
||
| DND-CORE-001 | Low | Simplicity | Remove the unused lossy unit-reference constructor |
|
||
| DND-REG-001 | Low | Correctness | Reject reversed evidence ranges at durable D&D codec boundaries |
|
||
| DND-REG-002 | Low | Correctness | Keep NPC and spell extraction evidence inside the current chunk |
|
||
| DND-REG-003 | Low | Efficiency | Index item and location duplicate groups |
|
||
| DND-REG-004 | Low | Efficiency | Select location identity anchors without sorting |
|
||
| DND-OCC-001 | Medium | Correctness | Align item occurrence evidence fields with the shared prompt |
|
||
| DND-OCC-002 | Low | Simplicity | Remove unused campaign references from NPC occurrence normalization |
|
||
| DND-OCC-003 | Low | Efficiency | Reuse canonical item occurrence evidence |
|
||
| DND-SCENE-001 | Medium | Correctness | Expose catalog aliases to spell extraction |
|
||
| DND-SCENE-002 | Low | Correctness | Make scene and spell normalization diagnostics complete and bounded |
|
||
|
||
## Findings
|
||
|
||
### Architecture And Dependency Boundaries
|
||
|
||
### ARCH-001 — Repair broken ADR cross-references
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Documentation/Comments
|
||
- **Evidence:**
|
||
`docs/adr/0012-resolve-opaque-entity-identifiers-deterministically.md:15`
|
||
links ADR-0003 as `0003-strongly-typed-stage-interfaces.md`, and line 17
|
||
links ADR-0009 as
|
||
`0009-prefer-minimal-evidence-grounded-extraction-artifacts.md`. Neither file
|
||
exists. The maintained files are
|
||
`0003-typed-interfaces-with-two-zone-data-model.md` and
|
||
`0009-minimal-evidence-grounded-extraction-artifacts.md`.
|
||
- **Impact:** Readers and documentation tooling cannot follow ADR-0012 to the
|
||
two architectural decisions it explicitly relies on. Runtime behavior is
|
||
unaffected.
|
||
- **Recommendation:** Correct only the two relative link targets in ADR-0012.
|
||
- **Preserve:** Keep the accepted decision text and its intended references to
|
||
ADR-0003 and ADR-0009 unchanged.
|
||
- **Validation:** Run a relative Markdown-link check across `docs/adr/` and
|
||
confirm both targets resolve; verify the change contains no decision-text
|
||
edits.
|
||
- **Grouping:** Independent.
|
||
|
||
### Configuration And CLI Composition
|
||
|
||
### CFGCLI-001 — Reject additional YAML documents
|
||
|
||
- **Severity:** Medium
|
||
- **Category:** Correctness
|
||
- **Evidence:** `internal/core/config/file_config.go:327`–`353`
|
||
constructs a `yaml.Decoder`, enables `KnownFields`, and calls `Decode` only
|
||
once. `yaml.Decoder.Decode` reads the next YAML document, so input such as a
|
||
valid version 4 configuration followed by `---` and another configuration is
|
||
accepted with the second document ignored. The strict file tests in
|
||
`internal/core/config/file_config_contract_test.go` cover malformed values,
|
||
unknown fields, and duplicate normalized identifiers, but not an additional
|
||
document.
|
||
- **Impact:** An operator can append a syntactically valid configuration
|
||
document, receive a successful validation result, and then run with only the
|
||
first document. Settings in the ignored document—including operational or
|
||
pipeline settings—have no effect without a diagnostic, contradicting the
|
||
documented strict single-file model.
|
||
- **Recommendation:** After decoding `FileConfig`, decode once more and require
|
||
`io.EOF`; reject any second document, including an empty or malformed one,
|
||
with a contextual configuration error.
|
||
- **Preserve:** Keep version gating before the full strict decode, unknown-field
|
||
and duplicate-key rejection, and the existing defaults → file → environment
|
||
precedence unchanged.
|
||
- **Validation:** Add focused parser cases for a second valid document, a
|
||
second malformed document, and ordinary trailing whitespace/comments; run
|
||
`go test ./internal/core/config ./internal/cli`.
|
||
- **Grouping:** Independent.
|
||
|
||
### CFGCLI-002 — Reject a blank command-level LLM profile
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Correctness
|
||
- **Evidence:** `internal/cli/run.go:149`–`166` registers `--llm-profile` as a
|
||
plain string flag, while the command's presence-aware empty-value checks
|
||
cover session ID, reasoning effort, output/debug directories, and recompute
|
||
step but not this flag. `runPipelineCommand` passes the resulting string to
|
||
`Config.Resolve`; `internal/framework/pipeline/profile.go:1255`–`1277` trims
|
||
an empty override and treats it as absent. The run contract tests cover a
|
||
valid override and an unknown non-empty profile, but not an explicitly
|
||
supplied blank value.
|
||
- **Impact:** A shell expansion such as `--llm-profile "$PROFILE"` with an
|
||
unset or blank value succeeds by silently using binding, pipeline, or
|
||
PromptKit defaults. The run can therefore use a different model/profile than
|
||
the operator explicitly intended to select.
|
||
- **Recommendation:** Make the CLI flag presence-aware and reject an explicitly
|
||
supplied empty or whitespace-only profile ID as command syntax before config
|
||
loading or physical-state allocation.
|
||
- **Preserve:** Keep a genuinely omitted override optional, trim non-empty IDs,
|
||
retain command → binding → pipeline → PromptKit precedence, and continue to
|
||
apply overrides only to selected LLM-backed bindings and validators.
|
||
- **Validation:** Add command-contract cases for blank and whitespace-only
|
||
values that assert exit status 2 and no state allocation, plus retain the
|
||
valid and unknown-profile run cases; run `go test ./internal/cli`.
|
||
- **Grouping:** Independent.
|
||
|
||
### Pipeline Resolution, Preparation, And Typed Registries
|
||
|
||
### PIPE-001 — Reject normalized module-reference collisions in the resolver
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Correctness
|
||
- **Evidence:** `internal/framework/pipeline/profile.go:1239`–`1252` sends every
|
||
module binding's reference map through `normalizeReferenceMap`. That helper,
|
||
at lines 1361–1377, trims each key but overwrites
|
||
`rawByNormalized[trimmedKey]` without checking whether another raw key already
|
||
produced the same identity. A programmatic binding containing both `slot`
|
||
and ` slot ` therefore retains whichever raw key is visited last by Go's map
|
||
iteration, then silently emits only one resolved binding. The later strict
|
||
reference resolver sees only the collapsed map and cannot diagnose the
|
||
collision. `internal/core/config/validation.go:256`–`266` correctly rejects
|
||
this shape for validated file/config flows, but direct `ResolvePipeline`
|
||
callers do not pass through that owner and there is no focused framework
|
||
regression case.
|
||
- **Impact:** A programmatically assembled profile can resolve successfully to
|
||
different external paths or generated selectors across processes from the
|
||
same ambiguous input. The normal CLI configuration path is protected by
|
||
upstream validation, which limits current production exposure, but the
|
||
resolver's own contract is nondeterministic.
|
||
- **Recommendation:** Make binding reference normalization return an error for
|
||
empty or duplicate trimmed keys before constructing the normalized map, and
|
||
propagate stage/lane context through `resolveBinding` callers. Avoid relying
|
||
on the config layer to make the framework resolver deterministic.
|
||
- **Preserve:** Keep whitespace normalization, exact external-versus-generated
|
||
source validation, sorted resolved bindings, local-over-pipeline precedence,
|
||
and the config layer's earlier contextual diagnostics.
|
||
- **Validation:** Add focused `ResolvePipeline` cases for whitespace-equivalent
|
||
chunk, extract, merge, and normalize reference keys, including different
|
||
source forms, and assert deterministic contextual rejection; run
|
||
`go test ./internal/framework/pipeline ./internal/core/config`.
|
||
- **Grouping:** Independent.
|
||
|
||
### PIPE-002 — Clone construction inputs once per builder boundary
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Efficiency
|
||
- **Evidence:** `Prepare` and `prepareLane` clone binding option maps while
|
||
forming requests (`internal/framework/pipeline/prepare.go:102`–`104` and
|
||
202–206), and `prepareValidatorChain` does the same at lines 258–263.
|
||
Registry/build boundaries then clone the complete request again. Typed
|
||
extractors, mergers, normalizers, and validators add another clone inside
|
||
their registered erased-builder adapters
|
||
(`extractor_registry.go:56`, `merger_registry.go:68`–`74`,
|
||
`normalizer_registry.go:63`–`69`, and `validator_registry.go:101`–`108`),
|
||
after `buildErasedModule` or `buildPreparedValidator` already called
|
||
`cloneBuildRequest` (`prepare.go:272`–`299` and 325–327). Each request clone
|
||
deep-copies materialized reference content as well as options, so typed
|
||
builders receive two reference copies and as many as three option copies;
|
||
untyped stage and validator builders use fewer copies. Spell extractor,
|
||
normalizer, and catalog-validator construction then call
|
||
`spells/catalog.ResolveEffectiveCatalog`; that helper clones every slot and
|
||
every item byte slice in the already owned `ReferenceSet`
|
||
(`internal/modules/dnd/spells/catalog/effective.go:55`–`62` and 411–426), even
|
||
though it reads only the `spell_catalog` slot and decodes its content into a
|
||
new overlay value. On extractor construction this needlessly recopies the
|
||
optional NPC registry and campaign-context references as well.
|
||
- **Impact:** Every preparation repeats allocation and byte copying for bounded
|
||
external references and nested options, with the highest cost and a
|
||
different ownership path specifically for typed lanes and validators. The
|
||
work is run-construction-time rather than a concurrent operation hot path,
|
||
so the issue is low severity.
|
||
- **Recommendation:** Designate one private construction invocation as the
|
||
ownership boundary and clone the complete `BuildRequest` exactly there.
|
||
Store raw builders or remove the caller-side clone consistently so all stage
|
||
and validator registry variants follow the same single-copy rule. Let spell
|
||
catalog resolution read the owned catalog slot directly instead of cloning
|
||
unrelated slots.
|
||
- **Preserve:** Builders must continue to receive independently owned options,
|
||
reference maps, slot slices, metadata, and content bytes; preparation must
|
||
retain its own immutable resolved/reference state; nil, key/name, execution
|
||
class, and exact artifact-type checks must remain contextual errors.
|
||
- **Validation:** Extend construction hooks to mutate nested options and
|
||
reference bytes for typed and untyped modules/validators, assert no aliasing
|
||
with resolved or sibling requests, and use allocation/byte-copy observations
|
||
or a focused benchmark to confirm a single defensive copy. Add a spell
|
||
catalog construction case with large unrelated slots and verify that
|
||
resolution neither copies nor mutates them; run
|
||
`go test ./internal/framework/contracts ./internal/framework/pipeline
|
||
./internal/modules/dnd/spells/catalog`.
|
||
- **Grouping:** Independent.
|
||
|
||
### References And Ordered Handoffs
|
||
|
||
### REF-001 — Bound reference reads before allocating the file
|
||
|
||
- **Severity:** Medium
|
||
- **Category:** Efficiency
|
||
- **Evidence:** `internal/framework/pipeline/references.go:88`–`164` implements
|
||
the external-reference materialization boundary. At lines 128–146,
|
||
`materializeReferenceTarget` calls `os.ReadFile(path)` before comparing the
|
||
resulting allocation with `ReferenceSlot.MaxBytes`. The focused
|
||
`TestMaterializeReferencesEnforcesMaxBytes` case uses a nine-byte file and
|
||
verifies the post-read diagnostic, but does not prove that reads are bounded
|
||
by the declared three-byte limit.
|
||
- **Impact:** A mistakenly selected very large file, growing file, device, or
|
||
named pipe can consume memory far beyond the slot's advertised bound before
|
||
the framework rejects it. CLI reference overrides expose the same path, so a
|
||
local operator error can terminate the process instead of producing the
|
||
intended bounded validation failure.
|
||
- **Recommendation:** Open the path and read through a limit of
|
||
`MaxBytes + 1` when a positive maximum is declared, rejecting an extra byte
|
||
before retaining or cloning content. A regular-file size precheck may improve
|
||
diagnostics, but the bounded reader must remain authoritative for changing or
|
||
non-regular inputs. Preserve the existing unbounded behavior only for slots
|
||
that explicitly declare no maximum.
|
||
- **Preserve:** Keep config-relative versus working-directory-relative path
|
||
resolution, UTF-8 and media-type validation, empty-file warnings, canonical
|
||
digest/size/origin metadata, contextual errors without content, and owned
|
||
reference bytes.
|
||
- **Validation:** Add a reader or file fixture that proves no more than
|
||
`MaxBytes + 1` bytes are consumed, including a non-regular or growing-input
|
||
case, while retaining the current UTF-8, media, empty, and ordinary oversize
|
||
diagnostics; run `go test ./internal/framework/pipeline ./internal/cli`.
|
||
- **Grouping:** Independent.
|
||
|
||
### REF-002 — Index accepted outputs once per ordered handoff
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Efficiency
|
||
- **Evidence:** `buildStepReferenceSets` walks every generated binding in the
|
||
receiving step (`internal/framework/pipeline/handoff.go:37`–`89`). For each
|
||
previously unseen producer, `generatedReferenceItem` allocates a `matches`
|
||
slice and scans the complete cumulative `outputs` slice to find that
|
||
step/lane (`handoff.go:104`–`133`). Its cache avoids rescanning when several
|
||
targets fan out from the same producer, but a step consuming `P` distinct
|
||
producers from `O` earlier outputs still performs `P * O` comparisons and up
|
||
to `P` temporary allocations.
|
||
- **Impact:** Ordered pipelines with many distinct generated dependencies pay
|
||
quadratic handoff preparation work before the consumer step can start. The
|
||
current production profiles are small and the scan is outside the lane
|
||
worker hot path, which limits present impact.
|
||
- **Recommendation:** Build one map from normalized `(step ID, lane ID)` to an
|
||
explicit zero/one/many accepted-output state at the start of
|
||
`buildStepReferenceSets`, then let `generatedReferenceItem` perform a direct
|
||
lookup. Keep ambiguity as data in the index so duplicate producer outputs are
|
||
still rejected rather than overwritten.
|
||
- **Preserve:** Retain exact accepted-output cardinality, codec
|
||
decode/re-encode canonicalization, producer lookup, complete schema/media
|
||
validation, per-target byte ownership, deterministic contextual errors, and
|
||
the rule that no consumer lane starts after a failed handoff.
|
||
- **Validation:** Add a many-producer/fanout case that retains missing and
|
||
duplicate rejection, then use a focused benchmark or comparison counter to
|
||
demonstrate one output-index pass plus direct producer lookups; run
|
||
`go test ./internal/framework/pipeline`.
|
||
- **Grouping:** Independent; do not combine with PIPE-002, which concerns
|
||
construction-request copying rather than runtime producer lookup.
|
||
|
||
### REF-003 — Include canonical size in generated-reference fingerprints
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Correctness
|
||
- **Evidence:** The generated `ReferenceItem` records canonical content length
|
||
in `SizeBytes` (`internal/framework/pipeline/handoff.go:154`–`177`), and the
|
||
manifest provenance also retains that value at lines 210–232. However,
|
||
`generatedReferenceFingerprintIdentity` and
|
||
`generatedReferenceDependencies` (`handoff.go:234`–`287`) hash producer,
|
||
kind, complete schema identity, media type, and content digest without the
|
||
canonical size. This differs from the documented resume contract in
|
||
`docs/internal/state.md:49`–`54`. The focused fingerprint test changes only
|
||
canonical content and does not assert size participation.
|
||
- **Impact:** Consumer checkpoint identity does not cover one field that the
|
||
handoff and manifest declare part of canonical reference identity. Current
|
||
construction derives size directly from canonical bytes, so a practical
|
||
stale reuse also requires malformed internal metadata, a digest collision,
|
||
or a future producer-path change; the immediate risk is therefore low.
|
||
- **Recommendation:** Add `SizeBytes` to the private fingerprint identity and
|
||
populate it from the canonical generated item before JSON hashing. Keep the
|
||
existing normalized fingerprint name and all current identity fields.
|
||
- **Preserve:** Do not weaken the content digest, producer provenance,
|
||
artifact kind, complete schema digest/fields, media type, or canonical codec
|
||
checks. Continue to keep content bytes out of checkpoints, manifests, debug
|
||
summaries, and errors.
|
||
- **Validation:** Add a direct dependency-fingerprint case that holds the
|
||
other identity fields constant while changing size metadata, retain the
|
||
canonical-content sensitivity case, and run
|
||
`go test ./internal/framework/pipeline ./internal/modules/integration/...`.
|
||
- **Grouping:** Independent.
|
||
|
||
### Execution, Validation, Retry, And Concurrency
|
||
|
||
### RUN-001 — Check cancellation at unguarded dispatch boundaries
|
||
|
||
- **Severity:** High
|
||
- **Category:** Correctness
|
||
- **Evidence:** `Runner.Run` invokes the input adapter at
|
||
`internal/framework/pipeline/runner.go:159`–`177` without checking
|
||
`ctx.Err()` first; the first framework-owned cancellation check on that path
|
||
is inside `runWithRetry` at lines 389–418, after source work and its
|
||
checkpoint/debug side effects. At the other end of the run, ordered lane work
|
||
completes at lines 278–282, then evidence/debug assembly and
|
||
`encoder.Encode` run at lines 297–346 without another cancellation check.
|
||
The worker and retry paths do check cancellation, but
|
||
`TestRunnerReturnsParentCancellationAndStopsQueuedExtracts` covers only
|
||
cancellation while two extract calls are active
|
||
(`runner_concurrency_test.go:559`–`585`). A context-ignoring input or output
|
||
module can therefore be called with an already-cancelled context; in the
|
||
output case it can return files and make the cancelled run report success.
|
||
- **Impact:** Work can start after cancellation, and a cancellation arriving
|
||
after lane completion or during output debug assembly can still publish a
|
||
successful logical output. This violates the documented guarantee that
|
||
parent cancellation prevents queued work and output encoding, and it leaves
|
||
correctness dependent on every module independently honoring an already
|
||
cancelled context.
|
||
- **Recommendation:** Add framework-owned cancellation gates at run entry and
|
||
immediately before every module dispatch not already protected by
|
||
`runWithRetry` or a lane worker, especially input parsing and output encoding.
|
||
Recheck after intervening collaborator/debug/evidence work so a cancellation
|
||
cannot slip between the gate and the operation, and after an unguarded module
|
||
returns so a module that did not observe mid-call cancellation cannot publish
|
||
success. Return the parent context error through the existing failed-output
|
||
path.
|
||
- **Preserve:** Continue to pass the caller context into active operations,
|
||
wait for all started lane work, suppress output files on framework failure,
|
||
select an actual framework error deterministically when the parent remains
|
||
live, and prefer the parent cancellation when it is set.
|
||
- **Validation:** Add a pre-cancelled run whose input adapter ignores context
|
||
and must not be called, plus a run whose debug recorder cancels immediately
|
||
before output dispatch while a context-ignoring encoder records calls, and an
|
||
encoder that cancels mid-call but returns files. Assert `context.Canceled`, no
|
||
invocation after pre-dispatch cancellation, and no output files in either
|
||
output case; retain the active-extract cancellation case and run the package
|
||
under `-race`.
|
||
- **Grouping:** Independent.
|
||
|
||
### RUN-002 — Isolate typed validator values from stage output
|
||
|
||
- **Severity:** High
|
||
- **Category:** Correctness
|
||
- **Evidence:** Extract, merge, and normalize serialize a canonical candidate
|
||
before validation, but pass the operation's original typed value alongside
|
||
it; representative extract code is
|
||
`internal/framework/pipeline/runner_concurrent.go:424`–`439`, and merge and
|
||
normalize do the same in `runner_typed.go:264`–`277` and 359–392.
|
||
`validateTypedArtifact` clones source input, references, metadata, and chunks,
|
||
but `requestTarget := target` leaves `target.value` shallow-copied
|
||
(`runner_typed.go:458`–`499`). The registered typed adapter then exact-casts
|
||
that value and passes it directly to the validator
|
||
(`validator_registry.go:109`–`119`). Struct values containing slices, maps,
|
||
or pointers therefore retain aliases to the stage output. A validator can
|
||
mutate what later validators and final checkpoint serialization observe,
|
||
while serialized validators and attempt debug continue to describe the
|
||
pre-mutation canonical candidate. Candidate-encoding tests verify encode
|
||
counts and rejection-before-checkpoint behavior, but no focused test mutates
|
||
a typed validator request.
|
||
- **Impact:** A buggy validator can corrupt an accepted output, cause validators
|
||
in one chain to inspect different values, or make persisted output differ
|
||
from the candidate that was serialized for validation and debug. This breaks
|
||
the documented immutable whole-output validation contract at all three typed
|
||
stages.
|
||
- **Recommendation:** Treat the already serialized canonical candidate as the
|
||
clone boundary. Decode a fresh exact typed value through the active codec for
|
||
each typed validator, or provide an equivalent codec-backed deep clone, and
|
||
never expose the stage operation's retained value. Keep serialized validators
|
||
on independent schema/content copies of the same candidate.
|
||
- **Preserve:** Retain exact Go-type checks, one canonical candidate encode per
|
||
attempt, validator declaration order, isolated request metadata/references,
|
||
serialized-validator byte ownership, contextual errors, and final encoding
|
||
only after the entire chain approves.
|
||
- **Validation:** Use an artifact with both slice and map fields. Have the first
|
||
typed validator mutate both, then assert that the next typed validator, a
|
||
serialized validator, attempt debug, and accepted extract/merge/normalize
|
||
output all observe the original canonical value. Run the focused package
|
||
tests under `-race` as well as normally.
|
||
- **Grouping:** Independent.
|
||
|
||
### RUN-003 — Preserve warnings from the terminal rejected attempt
|
||
|
||
- **Severity:** Medium
|
||
- **Category:** Correctness
|
||
- **Evidence:** The runtime contract says warnings from the final accepted or
|
||
rejected attempt are preserved (`docs/internal/pipeline.md:122`–`126`). Each
|
||
retry closure assembles per-attempt operation and validator warnings, but
|
||
stores them only on acceptance. Generated chunk-plan rejection returns at
|
||
`internal/framework/pipeline/runner_chunk_plan.go:142`–`150` before assigning
|
||
`result.warnings` at line 154, and `Runner.Run` promotes chunk warnings only
|
||
for acceptance or a cache hit (`runner.go:228`–`233`). Extract follows the
|
||
same pattern at `runner_concurrent.go:425`–`457`; `finalizeLaneExtract` skips
|
||
directly from a rejected result to the next chunk at lines 469–476. Merge
|
||
and normalize collect `attemptWarnings` but return terminal rejections before
|
||
appending them to `RunOutput` (`runner_typed.go:264`–`301` and 359–415).
|
||
`runWithRetry` returns only the last rejection, not its warnings
|
||
(`runner.go:389`–`428`). Existing retry-warning tests prove that discarded
|
||
attempts are not promoted, and final-rejection debug tests prove rejection
|
||
recording, but neither asserts terminal rejection warnings.
|
||
- **Impact:** Module warnings and warnings from validators that approved before
|
||
the rejecting validator disappear exactly when the final candidate is
|
||
rejected. Operators receive the rejection but lose diagnostics produced by
|
||
that terminal attempt, contrary to the documented output contract.
|
||
- **Recommendation:** Carry the warnings associated with the last rejection
|
||
through the retry result, then promote only that terminal attempt's warnings
|
||
at chunk, extract, merge, and normalize rejection handling. Keep warnings
|
||
from earlier failed or rejected attempts confined to attempt debug. Coordinate
|
||
any persistence-format addition with the later checkpoint/state audit rather
|
||
than silently changing state ownership here.
|
||
- **Preserve:** Do not promote warnings from attempts superseded by a later
|
||
retry, do not turn rejection into a framework error, keep deterministic
|
||
lane/chunk warning order, and retain accepted and reused checkpoint warning
|
||
behavior.
|
||
- **Validation:** Add table-driven chunk, extract, merge, and normalize cases
|
||
with distinct first-attempt and final-rejection warning scopes. Assert that
|
||
only the final scopes reach `RunOutput`, rejection attempt counts remain
|
||
correct, intermediate warnings remain in attempt debug, and shuffled/race
|
||
runs preserve order.
|
||
- **Grouping:** Independent.
|
||
|
||
### RUN-004 — Document the lane collector's liveness invariant
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Documentation/Comments
|
||
- **Evidence:** `laneEngine.collect` coordinates a dynamically enabled
|
||
continuation send, extract-result closure, cancellation, and the
|
||
`pending`/`launched`/`completed` counters in
|
||
`internal/framework/pipeline/runner_concurrent.go:256`–`281`, with state
|
||
changes split across `handleExtractResult` and `handleCompletion` at lines
|
||
283–313. The loop has no invariant-level comment. Its non-obvious purpose is
|
||
to discard only unlaunched continuations after cancellation while continuing
|
||
to drain extract results and exactly one completion from every launched
|
||
continuation before the continuation channel is closed and workers are
|
||
awaited.
|
||
- **Impact:** Current tests and the race run support the implementation, but a
|
||
future change to channel buffering, cancellation cleanup, or a counter can
|
||
introduce a deadlock or early close without the ownership/liveness rule being
|
||
visible at the maintenance point.
|
||
- **Recommendation:** Add one short comment immediately above
|
||
`laneEngine.collect` stating that it is the sole owner of pending/launch/
|
||
completion accounting and must drain closed extract results plus every
|
||
launched continuation completion even after cancellation. Do not narrate the
|
||
`select` cases.
|
||
- **Preserve:** Keep chunk-first/lane-second extract dispatch, bounded extract
|
||
and continuation pools, overlap between completed lanes and remaining
|
||
extracts, cancellation of undispatched work, and the final worker wait.
|
||
- **Validation:** Review the comment against the collector termination
|
||
predicate and retain the existing cancellation, reverse-completion,
|
||
continuation-overlap, worker-bound, race, and shuffled tests.
|
||
- **Grouping:** Independent.
|
||
|
||
### State, Checkpoints, Debugging, And File Safety
|
||
|
||
### STATE-001 — Preserve distinct identities in state paths
|
||
|
||
- **Severity:** Medium
|
||
- **Category:** Correctness
|
||
- **Evidence:** The resolver accepts any non-empty trimmed ordered-step and
|
||
lane IDs (`internal/framework/pipeline/profile.go:341`–`351` and
|
||
1382–1392). Checkpoint path construction then sends both IDs through
|
||
`checkpointPathComponent` (`internal/framework/checkpoint/recorder.go:476`–
|
||
`508`), which maps `.`, `..`, and every value containing `..` to the single
|
||
component `_`. The trace side duplicates that encoder in
|
||
`debugPathComponent` (`internal/framework/pipeline/debug.go:37`–`62`), and
|
||
`cleanDebugPath` at lines 364–374 additionally applies `path.Clean` before
|
||
encoding. Thus valid distinct identities such as `.` and `_`, or `a..b` and
|
||
`_`, select the same checkpoint component and can also collapse to the same
|
||
debug path. Existing filesystem tests cover traversal and symlink rejection,
|
||
but no focused test asserts that accepted identities map injectively.
|
||
- **Impact:** Two valid lanes or steps in one resolved pipeline can overwrite
|
||
each other's checkpoint manifests and payloads. Exact manifest validation
|
||
prevents a mismatched artifact from being silently reused, but ordinary
|
||
resume repeatedly invalidates and re-executes the colliding work, and a
|
||
required selective-recompute predecessor can become unavailable after its
|
||
sibling overwrites it. Requested debug records can likewise overwrite or be
|
||
attributed to the wrong identity. Atomic rename does not protect against a
|
||
logical-name collision.
|
||
- **Recommendation:** Introduce one shared, injective path-component encoding
|
||
for accepted application identities. Preserve already-safe components, but
|
||
encode reserved dot components and every unsafe rune rather than replacing a
|
||
whole value with `_`; do not run identity-bearing debug paths through a
|
||
lossy clean operation first. Keep path joining and confinement separate from
|
||
identity encoding.
|
||
- **Preserve:** Retain trimmed non-empty identity validation, human-readable
|
||
ordinary IDs, exact step/lane checks inside checkpoint manifests, narrow
|
||
relative state paths, symlink rejection, and recoverable cold execution for
|
||
incompatible reconstructible cache entries.
|
||
- **Validation:** Add table and uniqueness cases covering `.`, `..`, `_`,
|
||
embedded `..`, separators, tildes, and Unicode, then run a two-identity
|
||
filesystem checkpoint/resume and debug trace case that proves distinct files
|
||
and successful accepted-normalize hydration. Include selective recomputation
|
||
so a required predecessor remains reusable.
|
||
- **Grouping:** Independent.
|
||
|
||
### STATE-002 — Remove redundant post-decode clones from canonical codecs
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Efficiency
|
||
- **Evidence:** `chunkmap.Codec.Decode` schema-validates and decodes JSON into a
|
||
fresh value, canonicalizes that value, then deep-clones it again before
|
||
returning (`internal/framework/chunkmap/codec.go:149`–`171`).
|
||
`evidencecontext.Codec.Decode` follows the same sequence
|
||
(`internal/framework/evidencecontext/codec.go:79`–`101`), while its
|
||
`canonicalize` already deep-clones the decoded document at lines 169–203;
|
||
the final `clone` at lines 299–320 therefore makes a second complete copy of
|
||
context/reference slices, source-unit structs, and nested metadata. Unlike an
|
||
encode caller, the JSON decoder retains no caller-owned object graph that
|
||
must be protected from canonicalization.
|
||
- **Impact:** Hydrating or consuming canonical artifacts performs avoidable
|
||
slice and nested-metadata allocations. The evidence-context path can clone a
|
||
large source-derived object graph twice after JSON decoding, increasing peak
|
||
allocation and latency without adding an ownership boundary.
|
||
- **Recommendation:** Separate canonicalization of an already-owned decoded
|
||
value from the defensive-copy entry point used by encoders and external
|
||
callers. Return the canonical decoded value directly once validation and
|
||
normalization succeed; retain exactly one clone wherever caller-owned input
|
||
could otherwise be mutated.
|
||
- **Preserve:** Keep schema validation, unknown-field and trailing-value
|
||
rejection, digest reconstruction, annotation/context normalization,
|
||
encode-time caller isolation, and a fully independently owned decode result.
|
||
- **Validation:** Retain and extend ownership tests that mutate both the input
|
||
value after encode and the returned value after decode, then use focused
|
||
allocation assertions or benchmarks for a nested chunk map and evidence
|
||
context to confirm that decode no longer performs the redundant deep copy.
|
||
- **Grouping:** Independent.
|
||
|
||
### STATE-003 — Publish output through the confined file writer
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Duplication
|
||
- **Evidence:** The CLI's private `writeFileAtomic`
|
||
(`internal/cli/run.go:826`–`856`) independently implements the same
|
||
create-temp, write, chmod, close, rename, and cleanup sequence as
|
||
`internal/core/fileio.writeAtomic` (`internal/core/fileio/fileio.go:105`–
|
||
`133`). `writeOutputFiles` calls the CLI copy after its own path checks and
|
||
directory creation (`internal/cli/run.go:754`–`787`), while
|
||
`fileio.WriteBytes` additionally centralizes relative-path confinement and
|
||
symlink-component rejection. Debug summaries, traces, and checkpoints
|
||
already use that core primitive; durable output is the divergent state
|
||
writer.
|
||
- **Impact:** Atomic-publication fixes must be made and tested in two places,
|
||
and the copies have already drifted in their confinement checks. The fresh,
|
||
exclusively created run directory limits current output exposure, so this is
|
||
a maintenance and defense-in-depth issue rather than an observed escape.
|
||
- **Recommendation:** After prevalidating all logical output names and
|
||
exclusively creating the run directory, publish each file through
|
||
`fileio.WriteBytes` (or a narrowly exported equivalent) using the run
|
||
directory as root. Remove the CLI atomic-write copy rather than introducing
|
||
another generic filesystem abstraction.
|
||
- **Preserve:** Validate every logical name before allocating state, refuse an
|
||
existing run directory unchanged, retain a newly created partial bundle on a
|
||
later write failure, keep output directory/file modes at 0755/0644, and
|
||
preserve logical-name context in errors.
|
||
- **Validation:** Retain the existing output ordering, collision, permission,
|
||
and partial-bundle cases; add a symlink-component case at the shared writer
|
||
boundary and verify no temporary file remains after write, chmod, close, or
|
||
rename failure.
|
||
- **Grouping:** Independent.
|
||
|
||
### LLM Runtime, Prompt Filesystems, And Assets
|
||
|
||
### LLM-001 — Keep raw provider errors inside the adapter
|
||
|
||
- **Severity:** High
|
||
- **Category:** Correctness
|
||
- **Evidence:** Preparation and ordinary execution failures wrap
|
||
`redactPromptKitError` with `%w` at
|
||
`internal/framework/llm/promptkit_client.go:140`–`145` and 149–172. The
|
||
returned `redactedProviderError` replaces bearer-token text only in its
|
||
`Error` method, then exposes the original upstream error unchanged through
|
||
`Unwrap` at lines 424–442. The outer error therefore prints a redacted
|
||
diagnostic, but `errors.Unwrap`, `errors.Is`, or `errors.As` can recover the
|
||
raw PromptKit/provider error, including its original text and concrete type.
|
||
This contradicts the adapter contract in `docs/internal/llm.md:177`–`190`
|
||
and 231–236, which requires provider-specific errors and credentials to stay
|
||
behind the provider-neutral boundary. Capacity exhaustion avoids this
|
||
particular chain by mapping the PromptKit sentinel explicitly and formatting
|
||
the sanitized diagnostic with `%v`.
|
||
- **Impact:** Any framework collaborator, logger, test helper, or future error
|
||
serializer that walks the standard error chain can disclose a bearer
|
||
credential that normal string formatting appeared to redact. It can also
|
||
couple application code to PromptKit/provider error types despite the
|
||
documented neutral transport boundary.
|
||
- **Recommendation:** Make sanitized upstream diagnostics opaque: do not
|
||
implement `Unwrap` and do not wrap a provider error directly after crossing
|
||
the adapter boundary. Classify context cancellation, capacity, invalid
|
||
structured output, and any other intentionally supported neutral categories
|
||
before sanitization, then expose only the corresponding contracts sentinel
|
||
plus credential-redacted text.
|
||
- **Preserve:** Keep caller-context precedence, prompt context, the
|
||
provider-neutral capacity and invalid-output sentinels, full internal
|
||
PromptKit error inspection before adaptation, and useful redacted operational
|
||
diagnostics.
|
||
- **Validation:** Return a typed upstream error containing a bearer secret from
|
||
both preparation and generation. Assert that every reachable error-chain
|
||
level and formatted form omits the secret, `errors.As` cannot recover the
|
||
upstream type, and `errors.Is` still recognizes only the documented context,
|
||
capacity, and invalid-output categories.
|
||
- **Grouping:** Independent.
|
||
|
||
### LLM-002 — Recheck cancellation after scheduler admission
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Correctness
|
||
- **Evidence:** A queued scheduler waiter selects between its closed `ready`
|
||
channel and `ctx.Done()` at `internal/framework/llm/scheduler.go:50`–`73`.
|
||
`grantQueuedLocked` marks the waiter granted and closes `ready` at lines
|
||
103–111. If cancellation and that grant become observable together, Go may
|
||
choose the ready case, so `Acquire` returns a permit even though the context
|
||
is already canceled. `Scheduler.Run` then invokes `fn(ctx)` immediately at
|
||
lines 77–86 without another cancellation check. The focused queued-
|
||
cancellation test cancels before releasing the existing permit and therefore
|
||
does not exercise the simultaneous grant/cancel branch.
|
||
- **Impact:** A scheduler user that does not independently reject an already-
|
||
canceled context can begin backend work after the caller canceled while it
|
||
was queued. The production PromptKit path receives the canceled context and
|
||
ordinarily stops downstream work, and the deferred release prevents a permit
|
||
leak, which limits current severity; the scheduler's own cancellation
|
||
contract is nevertheless incomplete.
|
||
- **Recommendation:** After admission and before dispatch, recheck `ctx.Err()`
|
||
while retaining the deferred permit release. Keep `Acquire`'s locked
|
||
grant-versus-remove accounting as the owner of queue state; the extra gate
|
||
should only prevent the admitted callback from starting.
|
||
- **Preserve:** Retain one process-wide FIFO queue, the fixed concurrency
|
||
bound, idempotent release functions, cancellation removal for still-queued
|
||
waiters, and permit transfer/release after all errors.
|
||
- **Validation:** Add a focused grant/cancel race case with a callback that
|
||
records invocation and intentionally ignores its context. Prove that an
|
||
already-canceled admitted request returns `context.Canceled`, never invokes
|
||
the callback, releases its permit, and allows the next FIFO waiter to run;
|
||
retain the concurrency, cancellation, and idempotent-release tests under
|
||
`-race`.
|
||
- **Grouping:** Independent; this is the provider scheduler boundary rather
|
||
than RUN-001's pipeline module-dispatch boundary.
|
||
|
||
### LLM-003 — Reject duplicate virtual prompt names
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Correctness
|
||
- **Evidence:** `promptfs.ModulePromptFS` validates and reads every declared
|
||
module file, then assigns its bytes directly into a map at
|
||
`internal/framework/promptfs/prompt_fs.go:40`–`58`; shared files use the same
|
||
direct assignment at lines 60–80. Two names that are equal after trimming,
|
||
leading-slash removal, and cleaning therefore select the same virtual path,
|
||
and the later declaration silently replaces the earlier bytes. The asset
|
||
registry rejects duplicate flattened roots, but the module manifest boundary
|
||
has no equivalent check or focused duplicate test. All fourteen current D&D
|
||
manifests declare unique normalized names, so no maintained asset is
|
||
presently shadowed.
|
||
- **Impact:** A future or programmatically supplied prompt manifest can hash,
|
||
register, and execute successfully while using different prompt or shared
|
||
fragment bytes than one of its declarations implies. List order silently
|
||
decides the winner instead of producing the contextual asset-construction
|
||
error expected at startup.
|
||
- **Recommendation:** Track each cleaned virtual destination while assembling
|
||
`ModulePromptFS` and reject a second declaration before overwriting it. Report
|
||
whether the collision is module-owned or shared and include only the safe
|
||
virtual name, not file content.
|
||
- **Preserve:** Keep module and `sharedassets` namespaces distinct, reject
|
||
nested virtual names, read only explicitly declared files, return owned
|
||
bytes, and retain manifest order for fingerprinting and diagnostics where it
|
||
is semantically relevant.
|
||
- **Validation:** Add exact and normalization-equivalent duplicate cases for
|
||
module files and for shared files backed by different filesystems. Assert
|
||
deterministic contextual rejection while distinct module/shared basenames
|
||
remain valid; run the package under `-race`.
|
||
- **Grouping:** Independent.
|
||
|
||
### LLM-004 — Share the read-only in-memory filesystem mechanics
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Duplication
|
||
- **Evidence:** `internal/framework/llm/asset_registry.go:279`–`430` implements
|
||
a complete map-backed read-only filesystem—path cleaning, `Open`, `ReadFile`,
|
||
`ReadDir`, directory discovery, sorted entries, cursor reads, file metadata,
|
||
and modes. `internal/framework/promptfs/prompt_fs.go:84`–`240` independently
|
||
repeats the same mechanics with renamed types. The copies have already
|
||
drifted: the asset filesystem represents an empty root as an existing empty
|
||
directory while the prompt filesystem reports it missing, and directory
|
||
`Read` returns a custom error in one implementation versus `io.EOF` in the
|
||
other.
|
||
- **Impact:** Filesystem-contract fixes and edge-case tests must be duplicated,
|
||
while callers receive subtly different behavior from two byte maps assembled
|
||
for the same PromptKit engine. The maintained registries are non-empty, so
|
||
the observed drift is a maintenance risk rather than a current prompt load
|
||
failure.
|
||
- **Recommendation:** Extract one narrow framework-owned read-only byte-map
|
||
filesystem with standard `io/fs` behavior and independently owned file
|
||
handles. Keep asset flattening, duplicate/root validation, schema selection,
|
||
and module/shared manifest construction in their existing domain owners.
|
||
- **Preserve:** Retain valid-path enforcement, deterministic directory order,
|
||
0444/0555 metadata, immutable source bytes, independent reader offsets,
|
||
contextual domain errors before filesystem construction, and the asset
|
||
registry's direct owned `ReadFile` result.
|
||
- **Validation:** Run `fstest.TestFS` over empty, single-file, and nested trees;
|
||
test root and directory `Open`/`ReadDir`/`Read` behavior, file-handle
|
||
independence, sorted entries, missing/invalid paths, modes, and mutation
|
||
isolation. Retain both asset-registry and `ModulePromptFS` integration tests.
|
||
- **Grouping:** Independent; implement after or alongside LLM-003 without
|
||
moving manifest collision policy into the generic filesystem.
|
||
|
||
### Generic And Seriatim Modules
|
||
|
||
### MOD-001 — Narrow generic integer option decoding
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Simplicity
|
||
- **Evidence:** `internal/modules/generic/chunk/units/chunker.go:138`–`220`
|
||
routes only `max_units` and `overlap_units` through a 61-line numeric
|
||
conversion family that accepts every signed and unsigned Go integer type,
|
||
integral `float64` values, and `json.Number`. Its only production callers are
|
||
the two local positive/non-negative wrappers. In contrast, the other integer
|
||
option in this family, JSON output's `window_units`, accepts the
|
||
configuration-native `int` directly
|
||
(`internal/modules/generic/output/json/encoder.go:178`–`188`). The focused
|
||
chunker tests reject a fractional float and malformed `json.Number`, but do
|
||
not establish a caller or contract for any accepted non-`int`
|
||
representation.
|
||
- **Impact:** The generic chunker's private option surface is substantially
|
||
larger than its two-option configuration contract and silently treats a
|
||
floating-point value such as `2.0` as an integer while the adjacent generic
|
||
output decoder rejects that representation. Additional numeric branches and
|
||
architecture-dependent range logic can drift without providing behavior
|
||
used by the maintained configuration path.
|
||
- **Recommendation:** Define the supported module-option scalar type at the
|
||
configuration boundary and reduce these two options to that representation,
|
||
currently `int`, with their existing positive/non-negative and overlap
|
||
checks. If a second real option source requires `json.Number` or another
|
||
representation, normalize it once at that source rather than retaining a
|
||
speculative all-Go-numeric converter in this leaf module.
|
||
- **Preserve:** Keep defaults, strict unknown-option rejection, precise option
|
||
names in diagnostics, `overlap_units < max_units`, platform-safe values, and
|
||
validation/build decoding parity.
|
||
- **Validation:** Add focused rejection cases for integral floats and other
|
||
unsupported numeric representations, retain boundary/overlap tests, and run
|
||
`go test ./internal/modules/generic/chunk/units ./internal/core/config`.
|
||
- **Grouping:** Independent.
|
||
|
||
### MOD-002 — Reuse compiled response schemas within a prepared validator
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Efficiency
|
||
- **Evidence:** Every call to
|
||
`internal/modules/generic/validate/valid_json_schema.Validator.Validate`
|
||
delegates to `validate`, which parses the candidate, reparses the complete
|
||
response schema, creates a new `jsonschema.Compiler`, adds the schema
|
||
resource, and compiles it before validation
|
||
(`validator.go:29`–`56`). The framework constructs one validator instance per
|
||
prepared chain position (`internal/framework/pipeline/prepare.go:258`–`299`)
|
||
and invokes that same prepared instance for each chunk/artifact candidate
|
||
and retry (`runner.go:430`–`489` and `runner_typed.go:458`–`542`). Within a
|
||
chain position the active artifact schema is stable, while candidate content
|
||
changes.
|
||
- **Impact:** A run with many chunks or retries repeatedly pays schema parse,
|
||
compiler construction, resource loading, and compilation for identical
|
||
schema bytes. The cost is deterministic local CPU/allocation work on every
|
||
validation attempt; current LLM-backed workloads limit its overall severity.
|
||
- **Recommendation:** Cache the compiled schema on the prepared validator,
|
||
keyed by the complete schema identity including its JSON bytes or digest,
|
||
and make concurrent first use safe because extract validators can run in
|
||
parallel. Continue parsing each candidate independently and preserve support
|
||
for a validator instance receiving a different schema rather than assuming
|
||
one globally.
|
||
- **Preserve:** Retain operational errors for missing/malformed schemas,
|
||
ordinary `invalid_json` and `json_schema_invalid` rejections, support for
|
||
both chunk and artifact targets, exact candidate ownership, and standalone
|
||
use without a preceding `valid_json` validator.
|
||
- **Validation:** Add repeated-schema and changed-schema cases, exercise one
|
||
validator concurrently, verify malformed-schema errors are stable, and use a
|
||
focused benchmark or compiler hook to demonstrate one compilation per
|
||
distinct schema; run
|
||
`go test -race ./internal/modules/generic/validate/valid_json_schema ./internal/framework/pipeline`.
|
||
- **Grouping:** Independent.
|
||
|
||
### MOD-003 — Remove unreachable JSON metadata clone helpers
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Simplicity
|
||
- **Evidence:** `cloneMetadata` and `cloneJSONMetadataValue` at
|
||
`internal/modules/generic/output/json/encoder.go:497`–`527` form a mutually
|
||
recursive deep-clone implementation. Knowledge-graph inbound-call inspection
|
||
and a scoped source search find only the calls between those two functions;
|
||
no encoder path, registration path, or test reaches either helper. Actual
|
||
output ownership is handled by `cloneNormalizeOutputs`, serialized-artifact
|
||
cloning, and immediate JSON serialization.
|
||
- **Impact:** Thirty-one lines of recursive, type-specific ownership code imply
|
||
a boundary that does not exist, add an untested maintenance surface, and can
|
||
mislead later changes into choosing the dead helper instead of the encoder's
|
||
active clone/serialization paths.
|
||
- **Recommendation:** Delete the two unreachable helpers. Do not replace them
|
||
with a shared abstraction unless a live metadata owner later demonstrates a
|
||
caller.
|
||
- **Preserve:** Keep request non-mutation, owned output bytes, exact preservation
|
||
of chunk-map annotation numbers, cloned normalized artifacts, and caller
|
||
mutation isolation after `Encode` returns.
|
||
- **Validation:** Retain the output encoder's request-mutation and annotation
|
||
number tests, run `go test ./internal/modules/generic/output/json`, and
|
||
confirm no production caller is removed with graph and compiler checks.
|
||
- **Grouping:** Independent.
|
||
|
||
### Shared D&D Types, Codecs, And Family Mechanics
|
||
|
||
### DND-CORE-001 — Remove the unused lossy unit-reference constructor
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Simplicity
|
||
- **Evidence:** `internal/modules/dnd/shared/unit_refs.go:24`–`27` exports
|
||
`UnitRefFromString`, discards the error from `parseUnitRefNumber`, and returns
|
||
a zero-valued `UnitRef` for every blank, whitespace-padded, non-integer, or
|
||
non-positive value. Knowledge-graph inbound-call tracing and a scoped source
|
||
search found no production or test caller. The live external boundary is
|
||
`UnitRef.UnmarshalJSON` at lines 47–71, which calls the same parser and
|
||
correctly propagates its error; `UnitRefFromInt` remains used by that decoder
|
||
and focused tests.
|
||
- **Impact:** The exported helper advertises a supported string-construction
|
||
path whose only distinguishing behavior is to erase invalid-input context.
|
||
A future caller could turn malformed model output or test data into a later,
|
||
less specific `must be positive` error, while the dead API adds maintenance
|
||
surface beside the authoritative strict decoder.
|
||
- **Recommendation:** Delete `UnitRefFromString`. If a non-JSON string boundary
|
||
later needs construction, expose a parsing function that returns
|
||
`(UnitRef, error)` and reuse `parseUnitRefNumber` without discarding errors.
|
||
- **Preserve:** Keep string-or-integer JSON compatibility, exact positive source
|
||
unit IDs rather than ordinal fallback, original string-versus-number JSON
|
||
rendering, and the contextual checks in `ResolveUnitID`.
|
||
- **Validation:** Confirm graph and compiler checks find no removed caller;
|
||
retain the string/integer/malformed JSON and exact-ID tests in
|
||
`internal/modules/dnd/shared/unit_refs_test.go`; run
|
||
`go test ./internal/modules/dnd/shared/...`.
|
||
- **Grouping:** Independent.
|
||
|
||
### NPC, Item, And Location Registries
|
||
|
||
### DND-REG-001 — Reject reversed evidence ranges at durable D&D codec boundaries
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Correctness
|
||
- **Evidence:** The three registry, three occurrence, spell, and scene-
|
||
description integration contracts require that a source range's start not
|
||
follow its end
|
||
(`docs/integrations/dnd-npc-registry-artifacts.md:32`–`34`,
|
||
`dnd-item-registry-artifacts.md:33`–`35`, and
|
||
`dnd-location-registry-artifacts.md:34`–`36`; and
|
||
`dnd-npc-occurrence-artifacts.md:33`–`36`,
|
||
`dnd-item-occurrence-artifacts.md:41`–`43`,
|
||
`dnd-location-occurrence-artifacts.md:33`–`35`,
|
||
`dnd-spell-artifacts.md:31`–`35`, and
|
||
`dnd-scene-description-artifacts.md:33`–`35`). All eight reviewed durable
|
||
codec validators check only that both endpoints are positive
|
||
(`internal/modules/dnd/codec/npcregistry/codec.go:96`–`106`,
|
||
`codec/itemregistry/codec.go:97`–`107`, and
|
||
`codec/locationregistry/codec.go:97`–`107`; and
|
||
`codec/npcoccurrences/codec.go:99`–`109`,
|
||
`codec/itemoccurrences/codec.go:117`–`127`,
|
||
`codec/locationoccurrences/codec.go:100`–`110`,
|
||
`codec/spells/codec.go:89`–`99`, and
|
||
`codec/scenedescriptions/codec.go:101`–`109`). Direct registry references are
|
||
decoded and identity-checked by `npcs/registry.loadRegistry`,
|
||
`items/registry.loadRegistry`, and `locations/registry.loadRegistry`, but do
|
||
not pass through the generated-output source-reference validators. A durable
|
||
registry or occurrence artifact with
|
||
`{start_unit_id: 2, end_unit_id: 1}` and otherwise valid fields is therefore
|
||
accepted; for locations, `validIdentityReference` at
|
||
`internal/modules/dnd/locations/identity/identity.go:170`–`171` also treats
|
||
that reversed range as a valid identity anchor. The immutable scene-
|
||
eligibility registry also decodes approved scene artifacts directly before
|
||
projecting ID, range, and kind
|
||
(`internal/modules/dnd/scenedescriptions/registry/registry.go:174`–`223`).
|
||
- **Impact:** An externally supplied D&D artifact can be accepted as approved
|
||
even though its evidence cannot denote the documented forward source
|
||
interval.
|
||
NPC and item prompt projections then hide the malformed provenance, a
|
||
location can derive and retain a durable ID from it, and scene gating can
|
||
retain the impossible interval as eligibility metadata; only later consumers
|
||
with source context may reject it. Generated pipeline outputs remain
|
||
protected by their source-reference validator chains, which limits current
|
||
exposure.
|
||
- **Recommendation:** Add the order-independent structural condition
|
||
`start_unit_id <= end_unit_id` to all eight reviewed codecs' durable
|
||
validation, and make location identity reject reversed anchors as a defense
|
||
in depth.
|
||
Keep document membership and current-chunk coverage in the existing
|
||
validators, where the source document is available.
|
||
- **Preserve:** Retain positive exact source-unit identifiers, strict unknown-
|
||
field/trailing-value rejection, location identity's earliest canonical
|
||
anchor policy, and the separation between source-independent durable shape
|
||
validation and source-dependent evidence validation.
|
||
- **Validation:** Add codec encode/decode cases for reversed ranges in all eight
|
||
reviewed families, a location identity case that refuses a reversed-only
|
||
anchor, and direct registry, occurrence, spell, and scene-eligibility
|
||
reference cases proving malformed referenced JSON is rejected; run the
|
||
codec, identity, registry, spell, scene, occurrence, and assembled CLI suites.
|
||
- **Grouping:** Independent.
|
||
|
||
### DND-REG-002 — Keep NPC and spell extraction evidence inside the current chunk
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Correctness
|
||
- **Evidence:** The NPC, item, location, and spell extractors receive only the
|
||
current chunk and all four default extract chains run their family source-
|
||
reference validator. Item and location validators require a non-nil
|
||
extraction chunk and reject references whose endpoints are not in it
|
||
(`internal/modules/dnd/validate/itemregistry/source_refs/validator.go:37`–`55`
|
||
and `validate/locationregistry/source_refs/validator.go:37`–`55`). The NPC
|
||
validator at `validate/npcregistry/source_refs/validator.go:36`–`53` checks
|
||
only that a reference is valid somewhere in the complete source document; it
|
||
neither requires `req.Chunk` during extraction nor checks chunk membership.
|
||
The spell validator has the same document-only behavior
|
||
(`validate/spells/source_refs/validator.go:35`–`51`). Its tests exercise
|
||
valid, out-of-document, missing-document, malformed-shape, and bounded-
|
||
aggregate cases, but supply no stage or chunk and do not cover an existing
|
||
off-chunk range (`validator_test.go:15`–`73`).
|
||
- **Impact:** If an NPC or spell extraction response supplies a valid unit ID
|
||
from another chunk, the candidate can pass evidence validation despite the
|
||
model never receiving that passage. Relatedness can also succeed when the NPC
|
||
name or spell appears only at the cited off-chunk range, causing duplicated
|
||
or misattributed provenance across chunk results. The model usually copies
|
||
visible unit IDs, which limits the likelihood.
|
||
- **Recommendation:** Match the item/location extraction contract: require the
|
||
current chunk when `req.Stage` is extract and reject NPC and spell references
|
||
outside that chunk. Keep whole-document validation for normalize and other
|
||
non-extraction validation calls.
|
||
- **Preserve:** Retain full-document source-ID/range validation, shape deferral,
|
||
bounded aggregate diagnostics, normalization without a chunk, direct factual
|
||
third-party mentions, and source-relatedness as an advisory check rather than
|
||
an identity gate.
|
||
- **Validation:** Add NPC and spell source-reference cases for an existing off-
|
||
chunk range, a missing extraction chunk, an accepted in-chunk range, and
|
||
normalize-stage validation without a chunk; run both source-reference
|
||
validators, their extractors, and assembled pipeline tests.
|
||
- **Grouping:** Independent.
|
||
|
||
### DND-REG-003 — Index item and location duplicate groups
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Efficiency
|
||
- **Evidence:** NPC preprocessing indexes normalized comparison names in a map,
|
||
but `comparisonNameGroups` in
|
||
`internal/modules/dnd/normalize/itemregistry/normalizer.go:242`–`259` scans
|
||
every previously formed group for each item and recomputes the first member's
|
||
comparison key. `exactDuplicateGroups` in
|
||
`normalize/locationregistry/normalizer.go:229`–`246` repeats that nested scan
|
||
and performs `reflect.DeepEqual` over canonical source-reference slices for
|
||
each candidate group. With distinct records, both paths are quadratic before
|
||
any LLM reconciliation request is built.
|
||
- **Impact:** Large merged registries spend avoidable deterministic CPU in
|
||
normalization, with the location cost also proportional to citation-list
|
||
comparisons. Normal registry sizes and the later LLM request dominate today,
|
||
so the issue is low severity.
|
||
- **Recommendation:** Preserve first-seen group order while maintaining a local
|
||
index from the complete duplicate identity to its group position. Use the
|
||
existing comparison key for items and an exact, collision-safe key over the
|
||
comparison name plus canonical source-reference sequence for locations;
|
||
verify equality on any hash collision rather than using lossy concatenation.
|
||
- **Preserve:** Item duplicates remain name-identity duplicates regardless of
|
||
evidence; locations collapse only equal comparison names with exactly equal
|
||
canonical evidence; group/member order, input-index provenance, warning
|
||
scopes, and later proposal-only semantic reconciliation remain unchanged.
|
||
- **Validation:** Add many-distinct and repeated-key cases that compare output
|
||
groups and warning order with current fixtures, plus a focused benchmark or
|
||
comparison-count hook demonstrating linear expected grouping work; run the
|
||
item and location normalizer tests.
|
||
- **Grouping:** Independent.
|
||
|
||
### DND-REG-004 — Select location identity anchors without sorting
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Efficiency
|
||
- **Evidence:** Location identity depends only on the least valid source
|
||
reference. `earliestReference` nevertheless calls `canonicalReferences`,
|
||
which allocates a full copy, sorts it, and deduplicates it
|
||
(`internal/modules/dnd/locations/identity/identity.go:137`–`167`).
|
||
`ValidateRegistry` calls `earliestReference` once to check evidence and then
|
||
calls `DeriveID`, which invokes it again for every syntactically valid record
|
||
(lines 106–121). Normalization has already canonicalized each location's
|
||
references before deriving its ID, but pays the same second ordering pass.
|
||
- **Impact:** Each validation performs two `O(r log r)` allocations/sorts per
|
||
location to obtain one minimum, and normalization adds another sort after
|
||
source-order canonicalization. Citation counts are bounded in ordinary
|
||
transcripts, so this is a localized allocation and CPU issue.
|
||
- **Recommendation:** Implement `earliestReference` as a non-mutating linear
|
||
minimum selection using the exact current source-ID/start/end comparator, and
|
||
compute that anchor once per record during identity validation before ID
|
||
comparison. Do not reuse document-position `SourceRefOrder`, whose ordering
|
||
contract differs from durable location identity.
|
||
- **Preserve:** Keep order-independent IDs, filtering of malformed anchors,
|
||
lexicographic source-ID then numeric start/end selection, same-name/different-
|
||
anchor distinction, exact compact JSON hash input, and input non-mutation.
|
||
- **Validation:** Retain the permuted-reference and same-name/different-anchor
|
||
identity fixtures, add mixed valid/invalid and tie-order cases, and use an
|
||
allocation or benchmark assertion to demonstrate one linear pass per record;
|
||
run the location identity, registry, and normalizer tests.
|
||
- **Grouping:** Independent.
|
||
|
||
### NPC, Item, And Location Occurrences
|
||
|
||
### DND-OCC-001 — Align item occurrence evidence fields with the shared prompt
|
||
|
||
- **Severity:** Medium
|
||
- **Category:** Correctness
|
||
- **Evidence:** The shared model-visible evidence instruction tells every D&D
|
||
extractor to return integer `start_unit_id` and `end_unit_id` fields
|
||
(`assets/dnd/shared/prompts/common-dnd-extraction-evidence.md:1`–`4`). NPC and
|
||
location occurrence extraction follow that convention, but the item
|
||
occurrence private response DTO instead requires `start_segment` and
|
||
`end_segment`
|
||
(`internal/modules/dnd/extract/itemoccurrences/model.go:16`–`18`), and its
|
||
strict structured-output schema requires the same incompatible names
|
||
(`assets/dnd/item-occurrences/schemas/dnd_item_occurrences_llm.v1.json:20`–`29`).
|
||
The representative generated pipeline fixture must likewise emit the
|
||
segment-named fields to pass
|
||
(`internal/cli/dnd_enemy_events_contract_test.go:318`–`334`).
|
||
- **Impact:** The prompt and enforced response schema give the model
|
||
contradictory instructions at one structured-output boundary. A model that
|
||
follows the shared evidence instruction produces missing required fields and
|
||
disallowed unknown fields, causing an otherwise valid item occurrence
|
||
candidate to be rejected and retried. Provider-side schema steering may
|
||
conceal the mismatch for some models, but cannot make the prompt contract
|
||
coherent.
|
||
- **Recommendation:** Rename the item occurrence private response fields and
|
||
schema properties to `start_unit_id` and `end_unit_id`, update the adapter
|
||
and fixtures, and verify that the affected response-schema fingerprint
|
||
changes. Add an assembled-prompt contract proving every evidence field named
|
||
by the item response schema matches the shared instruction.
|
||
- **Preserve:** Keep source IDs out of model output, inject the current source
|
||
identity locally, retain exact positive transcript unit IDs, strict
|
||
structured decoding, current-chunk evidence validation, and the unchanged
|
||
durable occurrence wire shape.
|
||
- **Validation:** Retain item extractor malformed-response and citation tests,
|
||
update the generated multi-family CLI fixture to use unit-named endpoints,
|
||
add the prompt/schema alignment assertion, and run the item occurrence and
|
||
CLI suites.
|
||
- **Grouping:** Independent.
|
||
|
||
### DND-OCC-002 — Remove unused campaign references from NPC occurrence normalization
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Simplicity
|
||
- **Evidence:** NPC occurrence normalization declares the optional campaign
|
||
`glossary`, `party`, `players`, and deprecated `roster` slots as material used
|
||
for occurrence disambiguation
|
||
(`internal/modules/dnd/normalize/npcoccurrences/normalizer.go:38`–`43`) and
|
||
includes them beside the required registry in `ReferenceSlots` (lines
|
||
252–263). Construction retains only an NPC registry resolver (lines 51–67),
|
||
and normalization resolves and uses only that registry plus the source
|
||
document (lines 99–123). Item and location occurrence normalizers expose
|
||
only their required family registry.
|
||
- **Impact:** Bindings that cannot affect NPC normalization are nevertheless
|
||
admitted into resolved profiles, materialized, recorded in manifests and
|
||
dependency identities, and can invalidate checkpoints when their content
|
||
changes. The descriptions also imply a disambiguation behavior the
|
||
deterministic normalizer does not implement.
|
||
- **Recommendation:** Remove the four campaign slots and their descriptions
|
||
from the NPC occurrence normalizer, leaving only required `npc_registry`.
|
||
Keep campaign context on extraction, where it is model-visible and can
|
||
actually influence selection.
|
||
- **Preserve:** Retain exact registry ID/name validation and canonicalization,
|
||
operation-time registry overrides, source-reference ordering, exact duplicate
|
||
collapse, bounded warnings, and all extraction reference slots.
|
||
- **Validation:** Update module-spec, reference-binding, manifest, and
|
||
checkpoint dependency tests to require exactly the registry slot during
|
||
normalization; prove extraction still accepts campaign context; run NPC
|
||
occurrence and assembled CLI tests.
|
||
- **Grouping:** Independent.
|
||
|
||
### DND-OCC-003 — Reuse canonical item occurrence evidence
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Efficiency
|
||
- **Evidence:** Item normalization canonicalizes each occurrence's evidence
|
||
once in `normalizeOccurrence`
|
||
(`internal/modules/dnd/normalize/itemoccurrences/normalizer.go:174`–`195`),
|
||
then sorts those records through `itemoccurrences.Less` (lines 155–157) and
|
||
builds duplicate keys through `ExactIdentity` (lines 220–229). `Less`
|
||
canonicalizes both reference slices again at its final tie-breaker
|
||
(`internal/modules/dnd/itemoccurrences/itemoccurrences.go:95`–`125`), while
|
||
`ExactIdentity` canonicalizes each slice again at lines 145–163. The
|
||
invariants validator first verifies canonical order and uniqueness, then
|
||
calls both helpers again
|
||
(`internal/modules/dnd/validate/itemoccurrences/invariants/validator.go:65`–`95`).
|
||
- **Impact:** Tie-heavy merged lists repeatedly allocate, sort, and deduplicate
|
||
evidence that the normalizer has already canonicalized; normalized artifact
|
||
validation repeats the same work. Transcript and LLM costs dominate ordinary
|
||
runs, so this is a localized low-severity deterministic cost.
|
||
- **Recommendation:** Add canonical-input comparator and identity-key paths, or
|
||
precompute canonical sort/dedup keys per normalized record, and use them only
|
||
after the caller has established canonical evidence. Retain the current
|
||
defensive public helpers for arbitrary or malformed candidates that still
|
||
need literal-fallback ordering.
|
||
- **Preserve:** Keep exact canonical ordering, nil and invalid-reference
|
||
diagnostic behavior, holder and quantity distinctions, collision-safe
|
||
identity keys, stable input provenance for warnings, and non-mutating helper
|
||
contracts.
|
||
- **Validation:** Compare optimized normalization and invariant results against
|
||
the existing permuted/duplicate/invalid evidence fixtures; add a tie-heavy
|
||
benchmark or canonicalization-count hook showing one evidence canonicalization
|
||
per normalized record; run item occurrence normalize/validator tests.
|
||
- **Grouping:** Independent.
|
||
|
||
### Spells, Scene Chunking, And Scene Descriptions
|
||
|
||
### DND-SCENE-001 — Expose catalog aliases to spell extraction
|
||
|
||
- **Severity:** Medium
|
||
- **Category:** Correctness
|
||
- **Evidence:** The overlay contract says optional aliases contribute to spell
|
||
recognition (`docs/integrations/dnd-spell-catalog-overlays.md:3`–`6` and
|
||
57–68), and the effective catalog correctly maps each accepted alias to its
|
||
canonical spell (`internal/modules/dnd/spells/catalog/effective.go:215`–`247`).
|
||
The extractor's model-visible projection serializes only
|
||
`EffectiveCatalog.CanonicalNames()` as `spell_names`
|
||
(`internal/modules/dnd/extract/spells/catalog_prompt_input.go:13`–`28`), while
|
||
its prompt explicitly says aliases are absent and requires exact canonical
|
||
spelling (`assets/dnd/spells/prompts/spell-catalog.md:1`–`6`). Focused and
|
||
assembled tests codify the omission by rejecting `Emberfall Aegis` from the
|
||
prompt even though the configured overlay maps it to `Aegis of Emberfall`
|
||
(`internal/modules/dnd/extract/spells/extractor_test.go:88`–`103` and
|
||
`internal/cli/spell_catalog_identity_contract_test.go:412`–`414`). The
|
||
downstream catalog validator and normalizer can resolve an alias only if the
|
||
model already returns it; they cannot teach the model that alias-only
|
||
transcript language denotes the configured spell.
|
||
- **Impact:** A real cast expressed only with a campaign alias can be omitted or
|
||
rejected because the extraction model receives neither the alias nor its
|
||
canonical mapping. This silently defeats a documented purpose of overlay
|
||
configuration and can lose durable spell-cast occurrences; canonical names
|
||
and base-catalog casts remain unaffected.
|
||
- **Recommendation:** Project a deterministic source-free list of canonical
|
||
spell names with their recognized aliases into `spell_catalog`, and instruct
|
||
the model to return the associated canonical spelling. Continue to omit
|
||
catalog source metadata, paths, licenses, rules text, and provenance. Treat
|
||
the projection shape and prompt change as semantic fingerprint changes.
|
||
- **Preserve:** Keep overlays recognition-only rather than evidence, retain
|
||
strict collision rejection and canonical display names, require current-
|
||
transcript cast evidence and an in-world caster, preserve NPC provenance
|
||
separation, and keep the existing prompt order and cache boundaries.
|
||
- **Validation:** Add base and overlay projection tests for stable canonical-
|
||
alias mappings, repeated aliases, and metadata exclusion; update the
|
||
assembled prompt contract to require `Emberfall Aegis` only as an alias of
|
||
`Aegis of Emberfall`; retain catalog rejection, normalization, checkpoint-
|
||
invalidation, and retry tests; run the spell extractor, catalog, normalizer,
|
||
validator, and CLI suites.
|
||
- **Grouping:** Independent.
|
||
|
||
### DND-SCENE-002 — Make scene and spell normalization diagnostics complete and bounded
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Correctness
|
||
- **Evidence:** The maintained D&D convention says deterministic normalizers
|
||
issue bounded warnings for changes and collapsed duplicates
|
||
(`docs/internal/dnd.md:129`–`132`). Spell normalization emits a warning for
|
||
every canonicalized or unresolved name and changed reference list, then one
|
||
for every duplicate group, and returns the concatenated slice directly
|
||
(`internal/modules/dnd/normalize/spells/normalizer.go:89`–`105`, 108–149,
|
||
and 177–226); unlike the occurrence and registry normalizers, it never calls
|
||
`diagnostics.LimitWarnings`. Scene-description normalization has the inverse
|
||
gap: it trims title/summary, reorders records, and removes exact duplicates
|
||
(`normalize/scenedescriptions/normalizer.go:70`–`124`) but always returns the
|
||
value without any warnings (lines 53–68). Its primary test performs all three
|
||
mutations without asserting diagnostic provenance
|
||
(`normalizer_test.go:15`–`43`).
|
||
- **Impact:** A large spell list can publish an unbounded number of warnings,
|
||
inflating manifests, debug artifacts, output files, and checkpoint payloads.
|
||
Scene normalization silently changes accepted durable content, so operators
|
||
cannot distinguish an unchanged scene list from repaired whitespace/order or
|
||
collapsed duplicates. Artifact values remain deterministic and valid.
|
||
- **Recommendation:** Pass spell warnings through the shared deterministic
|
||
limiter with a stable omission reason. Add bounded scene warnings that
|
||
distinguish prose trimming, canonical reordering, and exact duplicate
|
||
collapse while retaining stable input scopes and order. Emit no warning for
|
||
an already canonical value.
|
||
- **Preserve:** Keep current normalized values, source-position ordering,
|
||
conflict rejection, duplicate identities, input immutability, per-record
|
||
diagnostic order below the limit, and idempotent warning-free second passes.
|
||
- **Validation:** Add spell cases beyond `diagnostics.MaxWarnings` that assert
|
||
the stable omission summary, and scene cases for each mutation, combined
|
||
overflow, canonical input, and idempotence; run both normalizer suites and
|
||
assembled output/checkpoint warning tests.
|
||
- **Grouping:** Independent.
|
||
|
||
<!--
|
||
Finding template for later audit stages:
|
||
|
||
### AREA-001 — Concise title
|
||
|
||
- **Severity:** High, Medium, or Low
|
||
- **Category:** Correctness, Efficiency, Duplication, Simplicity, Test Quality,
|
||
or Documentation/Comments
|
||
- **Evidence:** Exact files, symbols, call paths, and observed behavior
|
||
- **Impact:** The concrete risk or cost
|
||
- **Recommendation:** A bounded implementation direction
|
||
- **Preserve:** Invariants and contracts that remediation must retain
|
||
- **Validation:** Focused checks that would demonstrate success
|
||
- **Grouping:** Independent, or the IDs with which this should be implemented
|
||
-->
|
||
|
||
## D&D Convention Matrix
|
||
|
||
The matrix records the current production convention before the lane-specific
|
||
reviews. `E`, `M`, and `N` mean extract, merge, and normalize. Every extractor
|
||
is LLM-backed, every merger is the deterministic typed `appendorder` merger,
|
||
and every family also registers the typed deterministic `noop` normalizer as an
|
||
explicit alternate. `B` is the ordered default chain
|
||
`valid_json → shape → source_refs → valid_json_schema → source_relatedness`;
|
||
notations such as `B + identity` name the check inserted after `shape` and
|
||
before `source_refs`. `C` means the optional campaign-context slots `glossary`,
|
||
`party`, `players`, and deprecated `roster`. Every durable codec is
|
||
family-owned, uses media type `application/json`, embeds its own v1 durable
|
||
schema, publishes count metadata, and has a registered direct-source-reference
|
||
evidence projector.
|
||
|
||
| Family | Durable kind / Go type | Modules and execution class | Default validator chains (E / N) | Reference dependencies (E / N) | Codec, prompt, and schema ownership | Documented exception and later confirmation |
|
||
| --- | --- | --- | --- | --- | --- | --- |
|
||
| Spells | `dnd/spell-list` / `dnd.SpellList` | `dnd/spells` (LLM) → typed `appendorder` (deterministic) → `dnd/spells` (deterministic) | `B + catalog` / `B + catalog` | `C` plus optional `spell_catalog` and `npc_registry` / optional `spell_catalog` | `codec/spells`; extractor owns its prompt and private response schema; normalizer has no prompt | Confirmed: catalog and NPC grounding remain source-free recognition context; canonicalization, retry, and checkpoint identities are deterministic. DND-REG-001, DND-REG-002, DND-SCENE-001, and DND-SCENE-002 record codec, chunk-evidence, alias-projection, and warning gaps. |
|
||
| NPC registry | `dnd/npc-registry` / `dnd.NPCRegistry` | `dnd/npc-registry` (LLM) → typed `appendorder` (deterministic) → `dnd/npc-registry` (LLM) | `B` / `B + identity` | `C` / none | `codec/npcregistry`; extractor and normalizer own prompts; extractor owns its response schema, while normalize uses the shared private entity-reconciliation schema | Confirmed: normalized comparison-name identity, deterministic exact duplicate consolidation, proposal-only semantic groups, collision-safe application, retry/fallback, and immutable name/identity projections; DND-REG-001 and DND-REG-002 record evidence-boundary gaps. |
|
||
| Combat turns | `dnd/combat-turn-list` / `dnd.CombatTurnList` | `dnd/combat-turns` (LLM) → typed `appendorder` (deterministic) → `dnd/combat-turns` (deterministic) | `B` / `B + invariants` | `C`, optional `npc_registry`, required `scene_descriptions` / optional `npc_registry` | `codec/combatturns`; extractor owns its prompt and private response schema; normalizer has no prompt | Scene descriptions gate LLM execution and NPC grounding is not evidence; confirm gate and empty-result semantics in the combat/enemy review. |
|
||
| Item occurrences | `dnd/item-occurrence-list` / `dnd.ItemOccurrenceList` | `dnd/item-occurrences` (LLM) → typed `appendorder` (deterministic) → `dnd/item-occurrences` (deterministic) | `B + registry` / `B + registry + invariants` | `C` plus required `item_registry` / required `item_registry` | `codec/itemoccurrences`; extractor owns its prompt and private response schema; normalizer has no prompt | Confirmed: exact names-only registry resolution is all-or-nothing, registry provenance never becomes evidence, and quantity/holder rules survive canonical ordering and duplicate collapse. DND-REG-001, DND-OCC-001, and DND-OCC-003 record bounded codec, prompt/schema, and repeated-work gaps. |
|
||
| Item registry | `dnd/item-registry` / `dnd.ItemRegistry` | `dnd/item-registry` (LLM) → typed `appendorder` (deterministic) → `dnd/item-registry` (LLM) | `B` / `B + identity` | `C` / none | `codec/itemregistry`; extractor and normalizer own prompts; extractor owns its response schema, while normalize uses the shared private entity-reconciliation schema | Confirmed: name identity, exact duplicate evidence union, proposal-only aliases, collision safety, and denomination/type-preserving currency gate; DND-REG-001 and DND-REG-003 record boundary/grouping gaps. |
|
||
| NPC occurrences | `dnd/npc-occurrence-list` / `dnd.NPCOccurrenceList` | `dnd/npc-occurrences` (LLM) → typed `appendorder` (deterministic) → `dnd/npc-occurrences` (deterministic) | `B + registry` / `B + registry + invariants` | `C` plus required `npc_registry` / `C` plus required `npc_registry` | `codec/npcoccurrences`; extractor owns its prompt and private response schema; normalizer has no prompt | Confirmed: names-only selection resolves to exact registry pairs, registry provenance cannot become occurrence evidence, `mentioned` remains factual, and current-source evidence is canonicalized before exact duplicate collapse. DND-REG-001 and DND-OCC-002 record the codec and unused-normalizer-reference gaps. |
|
||
| Scene descriptions | `dnd/scene-description-list` / `dnd.SceneDescriptionList` | `dnd/scene-descriptions` (LLM) → typed `appendorder` (deterministic) → `dnd/scene-descriptions` (deterministic) | `B` / `B + invariants` | optional `glossary`, `party`, and `players` / none | `codec/scenedescriptions`; extractor owns its prompt and private response schema; normalizer has no prompt | Confirmed: one model classification is mapped onto the current chunk's local ID and exact range, while an immutable source-free ID/range/kind projection alone controls later eligibility. DND-REG-001 and DND-SCENE-002 record codec and warning gaps. |
|
||
| Enemy events | `dnd/enemy-event-list` / `dnd.EnemyEventList` | `dnd/enemy-events` (LLM) → typed `appendorder` (deterministic) → `dnd/enemy-events` (deterministic) | `B + engagements` / `B + invariants` | `C` plus required `npc_registry`, `scene_descriptions`, `combat_turns`, and `npc_occurrences` / required `npc_registry` | `codec/enemyevents`; extractor owns its prompt and private response schema; normalizer has no prompt | Four generated artifacts ground extraction without becoming event evidence; durable decode separately proves required JSON-field presence. Confirm combat gating, engagement uniqueness, and observation ordering in the combat/enemy review. |
|
||
| Location registry | `dnd/location-registry` / `dnd.LocationRegistry` | `dnd/location-registry` (LLM) → typed `appendorder` (deterministic) → `dnd/location-registry` (LLM) | `B` / `B + identity` | `C` / none | `codec/locationregistry`; extractor and normalizer own prompts; extractor owns its response schema, while normalize uses the shared private entity-reconciliation schema | Confirmed: comparison name plus earliest canonical evidence identity, same-name/different-anchor preservation, proposal-only aliases, and immutable context-qualified selectors without durable IDs; DND-REG-001, DND-REG-003, and DND-REG-004 record bounded gaps. |
|
||
| Location occurrences | `dnd/location-occurrence-list` / `dnd.LocationOccurrenceList` | `dnd/location-occurrences` (LLM) → typed `appendorder` (deterministic) → `dnd/location-occurrences` (deterministic) | `B + registry` / `B + registry + invariants` | `C` plus required `location_registry` / required `location_registry` | `codec/locationoccurrences`; extractor owns its prompt and private response schema; normalizer has no prompt | Confirmed: source-free contextual selectors disambiguate same-name locations without exposing durable IDs, unresolved selections reject the whole response, registry evidence remains separate, and explicit kind precedence preserves speculation/mention distinctions. DND-REG-001 records the codec gap. |
|
||
|
||
Shared convention review classified the codec surface as safe typed adapters,
|
||
not a missing artifact-codec framework. `candidatejson` is the natural owner for
|
||
single-value JSON encoding and strict decoding (including unknown-field and
|
||
trailing-value rejection), while each codec retains exact Go type, kind,
|
||
metadata, embedded durable schema, and artifact-specific required-field or
|
||
semantic validation. Candidate encode/decode clones are limited to the two
|
||
families with retained pointer/slice ownership (`ItemOccurrenceList` and
|
||
`EnemyEventList`); schema bytes are owned because every `embed.FS.ReadFile`
|
||
call returns a fresh byte slice. The generic candidate helper intentionally
|
||
does not absorb family validation or durable-schema policy.
|
||
|
||
Source-reference responsibilities are likewise separated by semantics:
|
||
`SourceRefOrder` owns document-position ordering, exact equality-based
|
||
deduplication, invalid-reference preservation for diagnostics, and nil versus
|
||
owned-empty behavior; `UnitRef.UnmarshalJSON` owns strict model-facing string
|
||
or integer parsing; `CitationResolver` owns current-document range expansion;
|
||
family validators own admissibility and evidence rules; and evidence projectors
|
||
copy only direct artifact references. Extractor-local response adapters remain
|
||
typed because their private response DTOs differ; replacing them with
|
||
reflection or callbacks would hide field ownership without consolidating
|
||
policy. DND-CORE-001 records the one dead constructor outside these live paths.
|
||
|
||
Registration is explicit and complete for all ten codecs, extractors, mergers,
|
||
normalizers, noop alternates, evidence projectors, validator capabilities, and
|
||
default chains, plus the separate scene chunker. Prompt registration covers
|
||
every LLM extractor, the scene chunker, the three LLM registry normalizers, the
|
||
shared reconciliation schema, and the fallback profile. The registrar validates
|
||
every registry and the asset registry before mutation, registers in a fixed
|
||
order, and wraps failures with the exact D&D component name; a later failure
|
||
can leave the caller-supplied registries partially populated, but production
|
||
composition treats registration failure as terminal and no rollback contract
|
||
is documented.
|
||
|
||
The shared registry resolver correctly distinguishes absent construction views,
|
||
empty generated-reference placeholders, operation-time overrides, raw-content
|
||
cache identity, and semantic identity while retaining owned loaded values. The
|
||
entity-reconciliation helper keeps durable IDs out of prompts, rejects invalid
|
||
or colliding selectors and overlapping groups, bounds transcript context, and
|
||
returns owned safe groups. Shared comparison policy is versioned, and shared
|
||
diagnostics bound displayed issues and warning counts. The registry and
|
||
occurrence reviews now confirm the noun-family identity, projection, category,
|
||
and merge decisions; scene/spell policy and combat/enemy behavior remain
|
||
assigned to their subsequent reviews.
|
||
|
||
Finally, repeated `ManifestMetadata` and `CheckpointFingerprints` methods remain
|
||
module-local because their exact policy names, prompt/schema hashes, reference
|
||
projection digests, and ordering are checkpoint semantics. Small `Register`,
|
||
`New`, and `DecodeOptions` methods preserve typed registry construction,
|
||
module-local error context, dependency validation, and strict option surfaces;
|
||
parameterizing them would trade visible policy for callback-heavy helpers. The
|
||
shared prompt manifest/hash, candidate JSON, registry resolver, diagnostics,
|
||
source ordering, and entity reconciliation packages already own the exact
|
||
mechanics that recur without erasing those distinctions.
|
||
|
||
## Intentional Complexity And Duplication To Preserve
|
||
|
||
- `internal/modules/generic/register.Register`,
|
||
`internal/modules/seriatim/register.Register`, and
|
||
`internal/modules/dnd/register.Register` deliberately expose the same small
|
||
registrar shape while retaining family-local registration policy and
|
||
diagnostics. Combining them would move extension ownership out of the domain
|
||
registrars and weaken the composition boundary established by ADR-0004.
|
||
- Seriatim's `decodeOptionalSegmentID`, `decodeOptionalNumber`, and timestamp
|
||
checks deliberately keep separate external number handling at the input
|
||
adapter. Segment IDs accept only positive canonical decimal JSON numbers or
|
||
strings, while optional timestamps accept finite non-negative decimals and
|
||
preserve their lexical `json.Number` form in generic metadata. Replacing
|
||
these with a permissive generic number coercer would weaken the documented
|
||
source contract and leak an external-format decision into `core/source`.
|
||
- `generic/valid_json` and `generic/valid_json_schema` both detect malformed
|
||
JSON, but remain useful separate serialized-validator capabilities. The
|
||
former is a cheap format-only boundary; the latter must parse an instance to
|
||
apply an independently supplied schema and can be selected without the
|
||
former. MOD-002 concerns repeated compilation within one schema validator,
|
||
not merging the two module identities or requiring a particular chain.
|
||
- `internal/modules/dnd/register.registerModules`, `registerEvidence`,
|
||
`registerValidators`, and `registerDefaultChains` use explicit typed
|
||
registration lists. At this architectural pass, that repetition preserves
|
||
artifact Go types, module-specific validator order, and registrar-owned
|
||
production policy. Later D&D stages may evaluate individual shared
|
||
mechanics, but should not replace these lists with a dynamically typed
|
||
registration engine.
|
||
- `internal/framework/pipeline.RegisterArtifactCodec` and
|
||
`exactTypedValue` perform apparently repetitive exact-type checks around
|
||
private erasure. The checks deliberately turn incompatible values into
|
||
errors at each erased boundary rather than permitting a panic or accepting a
|
||
near-matching type, preserving ADR-0003.
|
||
- `internal/cli.runPipelineCommand` is a large linear orchestrator, but its
|
||
ordering is policy: syntax and config rejection precede run identity and
|
||
debug allocation; resolution and profile inspection precede module
|
||
preparation and input parsing; framework success precedes durable output;
|
||
and terminal debug publication precedes the optional JSON receipt. Existing
|
||
helpers isolate reference selection, recomputation, stores, output, result
|
||
encoding, and terminal error precedence. A generic lifecycle abstraction
|
||
would hide physical-state allocation and publication boundaries; bounded
|
||
parsing fixes such as CFGCLI-002 should not reorganize that lifecycle.
|
||
- `internal/core/config.validatePipelineProfiles` explicitly walks pipelines,
|
||
ordered steps, lanes, bindings, and references. Its nested structure mirrors
|
||
the public configuration shape and retains the nearest pipeline/step/lane
|
||
context in errors. Replacing it with a reflection-driven validator would
|
||
weaken those diagnostics and the presence-aware file-model boundary.
|
||
- `internal/cli.selectedReferenceTargets` and `recomputePolicy` perform
|
||
explicit resolved-shape traversals for distinct CLI policies: disambiguating
|
||
reference selectors against selected module capabilities, and computing the
|
||
forward forced/backward reusable checkpoint closure. Keeping these typed
|
||
traversals separate avoids adding command syntax or checkpoint policy to the
|
||
framework resolver.
|
||
- `pipeline.ResolvePipeline` and `resolveArtifactLane` are long, but their
|
||
linear sections retain the authoritative composition order: normalize
|
||
identity, select lanes, prove capabilities and exact artifact variants,
|
||
resolve stage-local references and validator chains, apply effective LLM
|
||
profiles, validate options, and only then compute the digest. Splitting these
|
||
checks into a generic stage engine would erase the different input, chunk,
|
||
typed-lane, validator, and output contracts. PIPE-001 is a bounded
|
||
normalization fix and should not reorganize this sequence.
|
||
- `pipeline.validateGeneratedBindings` has deeply nested traversal because it
|
||
proves a cross-step selector against ordered producer identity, the target's
|
||
declared slot, accepted artifact kinds, registered codec, and accepted media
|
||
types in one pass. Those checks are distinct static composition invariants;
|
||
materialized bytes, runtime handoff construction, and checkpoint hydration
|
||
remain separate runtime owners.
|
||
- The stage, validator, codec, and evidence registries intentionally use
|
||
private typed entries and small stage-specific lookup methods. Their
|
||
repetition preserves compile-time generic types until a narrow erased
|
||
closure, exact artifact-kind variant selection, and stage-specific
|
||
diagnostics. PIPE-002 concerns redundant request copies around those
|
||
closures, not the typed registry split itself.
|
||
- Generated-reference handoff deliberately decodes and re-encodes an accepted
|
||
normalized artifact through the registered producer codec even though fresh
|
||
outputs were already serialized. The same boundary also receives hydrated
|
||
checkpoint artifacts, so canonicalizing once per unique producer verifies
|
||
exact kind, schema, media type, and bytes before fanout. REF-002 removes only
|
||
repeated output discovery; it must not bypass this codec check.
|
||
- Operation paths clone reference sets for each module request while retaining
|
||
a separate pristine set for dependency fingerprints and validators. That
|
||
apparent duplication isolates module mutation across chunks and retries and
|
||
keeps validators on authoritative inputs. PIPE-002 applies only to adjacent
|
||
construction-builder copies and should not remove these operation-time
|
||
ownership boundaries.
|
||
- `Runner.Run` is a long linear coordinator, but its sequence is the runtime
|
||
contract: source state precedes chunk planning; a rejected chunk plan skips
|
||
lanes but remains an encodable ordinary outcome; ordered steps are barriers;
|
||
evidence is built only from accepted normalized artifacts; and logical output
|
||
encoding is last. RUN-001 requires narrow cancellation gates and should not
|
||
replace that readable lifecycle with a generic stage engine.
|
||
- Extract jobs and lane continuations deliberately use separate pools, each
|
||
bounded by the resolved worker count. This permits a completed lane to enter
|
||
its serial merge/normalize continuation while other lanes are still
|
||
extracting, without letting either class grow with lane or chunk count. The
|
||
two pools should not be collapsed merely to make the configured limit a
|
||
process-wide semaphore; provider calls have their own shared scheduler.
|
||
- Chunk, extract, merge, and normalize retry closures repeat candidate debug,
|
||
rejection, and warning mechanics around different typed operations and
|
||
checkpoint transitions. Their common retry counter and cancellation policy
|
||
appropriately live in `runWithRetry`, while stage-specific serialization,
|
||
normalize fallback, and state recording remain explicit. RUN-003 calls for a
|
||
small terminal-warning result, not a parameter-heavy generic stage runner.
|
||
- `synchronizedCheckpointRecorder`, `synchronizedDebugRecorder`, and
|
||
`synchronizedLLMDebugRecorder` intentionally serialize collaborators whose
|
||
contracts do not promise concurrent safety. Each adapter owns one lock and
|
||
invokes only its wrapped collaborator, so there is no framework lock-order
|
||
cycle; removing them would push concurrency requirements into filesystem and
|
||
test implementations.
|
||
- Filesystem checkpoint recorder and loader methods intentionally repeat the
|
||
stage-shaped interface surface. Each small adapter fixes a stage, manifest
|
||
status, dependency set, and payload codec before delegating to shared
|
||
manifest validation or publication. Collapsing them into a reflection- or
|
||
string-driven state engine would weaken typed call sites and contextual
|
||
validation; STATE-001 concerns only their shared path-component encoding.
|
||
- The chunk-plan store intentionally does not use the general `fileio` writer.
|
||
Its source-addressed directory is an independently replaceable cache entry,
|
||
and its `os.Root`-relative implementation rejects non-directory digest
|
||
entries, non-regular or symlinked plans, named pipes, and replacement races,
|
||
while syncing a private temporary file before rename. Those stronger
|
||
cache-entry mechanics should remain local rather than being generalized into
|
||
output/debug/checkpoint publication.
|
||
- Source metadata, accepted chunk maps, and evidence contexts retain separate
|
||
clone helpers because their ownership graphs and canonical invariants differ:
|
||
source metadata handles nested dynamic values and cycles, chunk maps own
|
||
annotations and chunk slices, and evidence contexts own source units and
|
||
metadata. STATE-002 removes only copies of an already-owned decoded graph; it
|
||
should not replace these helpers with reflection or remove encode-time
|
||
isolation.
|
||
- `PromptKitClient.CompleteStructured` deliberately keeps prompt preparation,
|
||
the frozen prepared-detail snapshot, provider execution, PromptKit schema
|
||
validation, application decode, profile recording, and debug material
|
||
assembly explicit. Their order proves that debug and generation describe one
|
||
prepared request and that malformed provider output cannot become a typed
|
||
result. LLM-001 changes only the outward error chain; it should not hide this
|
||
sequence behind a generic transport pipeline.
|
||
- The CLI profile inspector and production PromptKit client intentionally own
|
||
separate engines while sharing the same profile-source option construction.
|
||
Inspection must resolve configured, fallback, built-in, and local-backend
|
||
policy without credentials, provider contact, or capacity admission;
|
||
reusing the runtime engine would blur that preflight boundary.
|
||
- Module prompt manifests and PromptKit YAML message lists explicitly enumerate
|
||
module-owned files, shared fragments, stable reference blocks, transcript
|
||
blocks, lane-specific grounding, and final instructions. This duplication is
|
||
content policy: it makes exact model-visible byte order and cache-control
|
||
breakpoints reviewable per workload. LLM-003 adds collision rejection, and
|
||
LLM-004 shares only the underlying filesystem mechanics; neither should
|
||
generate manifests dynamically or infer prompt order from filenames.
|
||
- The application scheduler and PromptKit backend limits intentionally remain
|
||
layered. The scheduler is the one process-wide provider-call bound across
|
||
profiles and modules, while PromptKit owns selected-backend capacity and maps
|
||
its exhaustion through a provider-neutral sentinel. LLM-002 closes an
|
||
admission/cancellation race without combining these limits or moving provider
|
||
capacity policy into the pipeline worker pools.
|
||
- `internal/modules/dnd/chunk/scenes.planFromResponse` intentionally keeps the
|
||
whole-source coverage policy explicit in one pass: it resolves opaque unit
|
||
IDs through the shared document index, compares document positions rather
|
||
than numeric IDs, rejects reversed ranges, gaps, and overlaps, and requires
|
||
exact first-to-last coverage before the framework independently canonicalizes,
|
||
validates, and materializes the plan. Its repeated-looking resolve/position
|
||
calls are constant-time index lookups that preserve contextual diagnostics;
|
||
a generic range collector would hide scene-specific full-coverage policy.
|
||
- `internal/modules/dnd/spells/catalog.composeEffectiveCatalog` intentionally
|
||
uses separate canonical-name, per-spell alias, and complete lookup maps. The
|
||
function validates base and overlay collisions while applying overlays in
|
||
sorted ID order, then produces stable names and a semantic digest. Its nested
|
||
loops traverse catalog entries and each entry's aliases without a linear scan
|
||
inside the loop. DND-SCENE-001 concerns the later model-visible projection,
|
||
not combining catalog composition with prompting or weakening collision
|
||
checks.
|
||
|
||
## Areas Reviewed Without Findings
|
||
|
||
### Architecture And Dependency Boundaries
|
||
|
||
- **Composition root:** `internal/cli.newProductionComponents` constructs the
|
||
complete registry set and asset registry, then invokes only the generic,
|
||
Seriatim, and D&D family registrars. Direct production imports confirm that
|
||
`internal/cli` is the only layer importing those registrar packages.
|
||
- **Dependency direction:** A direct production import map found no core or
|
||
framework package importing `internal/modules`, no module importing
|
||
`internal/cli`, no concrete generic or Seriatim module importing D&D, and no
|
||
module importing the file-backed checkpoint, chunk-plan, debug, file-I/O, or
|
||
debug-bundle implementations. PromptKit is imported directly only by
|
||
`internal/framework/llm`.
|
||
- **Graph cross-layer calls:** The refreshed graph reported one
|
||
framework-to-module edge from `pipeline.Prepare` to a symbol named `request`
|
||
in a D&D validator test. Tracing it showed a confidence `0.06` suffix match
|
||
from the local closure call in `Prepare`; `trace_path` classified the target
|
||
as test-only, and the production import map disproved a dependency. The graph
|
||
reported no module-to-CLI calls.
|
||
- **Assets leaf:** `assets/package.go` imports only `embed` and `io/fs`, embeds
|
||
content, and exposes the read-only `FS() fs.FS` accessor. It contains no
|
||
business logic and has no `internal` or PromptKit dependency.
|
||
- **Fixed pipeline shape:** `pipeline.ResolvePipeline` resolves input and
|
||
chunk once, fixed extract/merge/normalize bindings per artifact lane, and one
|
||
output binding. Ordered steps are barriers around those fixed lanes rather
|
||
than arbitrary graph topology. `pipeline.Prepare`, `Runner.Run`,
|
||
`runPreparedSteps`, and `runLanes` retain that shape through construction and
|
||
execution.
|
||
- **Typed artifact boundary:** Typed registrations retain the exact Go type for
|
||
codecs and lane operations. Private erasure in `RegisterArtifactCodec` and
|
||
`exactTypedValue` verifies exact types and returns contextual errors;
|
||
normalized values cross into output through serialized artifacts.
|
||
- **Physical-state ownership:** The CLI owns root selection, store factories,
|
||
and durable file placement (`chunkPlanStoreForRun`, checkpoint/debug setup,
|
||
and `writeOutputFiles`). The framework receives collaborator interfaces and
|
||
returns logical output files. The generic JSON output module's direct import
|
||
of `internal/framework/chunkmap` validates and republishes the accepted
|
||
serialized chunk-map contract; it neither chooses a physical root nor writes
|
||
files.
|
||
- **Accepted architectural decisions:** ADRs 0001–0005 and 0007–0012 were read
|
||
against the current high-level composition. Apart from ARCH-001, the
|
||
composition root, fixed ordered pipeline, typed boundary, domain packaging,
|
||
canonical chunk-plan policy, separate state surfaces, checkpoint policy,
|
||
evidence rules, workload profile ownership, centralized asset leaf, and
|
||
deterministic entity identity boundary have corresponding current owners.
|
||
|
||
### Configuration And CLI Composition
|
||
|
||
- **End-to-end command path:** `RunWithOptions` normalizes injectable process
|
||
collaborators once and dispatches to `runPipelineCommand`. The run command
|
||
parses and normalizes command input, discovers and loads configuration,
|
||
applies command overrides, builds the effective catalog, resolves reference
|
||
selectors and the pipeline, inspects effective profiles, materializes
|
||
references, constructs runtime/state collaborators, invokes `pipeline.Run`,
|
||
publishes output files, terminalizes debug state, and only then publishes a
|
||
requested JSON receipt.
|
||
- **Precedence and resolution:** `loadConfig` enforces explicit `--config` over
|
||
`NOTARIUS_CONFIG` over the system default, then applies `Default`, file
|
||
configuration, supported environment overrides, and run-only CLI overrides
|
||
in order. `Config.Resolve` recomputes derived worker defaults, validates,
|
||
clones the selected profile, and delegates catalog-dependent composition to
|
||
the framework resolver. Apart from CFGCLI-001 and CFGCLI-002, unknown fields,
|
||
malformed values, normalized-key collisions, unknown command flags, and
|
||
invalid selected modules/options are rejected at their owning boundary.
|
||
- **Profile-source equality:** Validation-time
|
||
`validateExplicitPromptKitProfiles` and runtime
|
||
`buildProductionLLMClient` pass the same profile directory, profile file,
|
||
mapped local backend, and shared fallback asset registry. Effective profile
|
||
collection is sorted, deduplicated, and limited to selected LLM-backed
|
||
modules and validators, so inspection and runtime selection use the resolved
|
||
profile values rather than recomputing inheritance.
|
||
- **Session identity:** `resolvePromptSessionID` uses a versioned SHA-256 value
|
||
over the trimmed resolved input-module key, a separator, and exact raw input
|
||
bytes. It contains no pipeline ID, reference, profile, retry, input path,
|
||
working directory, or run ID; an explicit non-empty session replaces the
|
||
generated value. Run contracts verify the same effective session reaches all
|
||
prompt-facing requests, manifests, debug metadata, and checkpoint identity.
|
||
- **Reference and recomputation controls:** CLI reference selectors are
|
||
resolved only against selected chunk/extract/merge/normalize capabilities
|
||
before the authoritative effective resolution. Recompute policy is derived
|
||
after reference materialization, forces the requested step and transitive
|
||
consumers, and requires reusable checkpoints for non-forced transitive
|
||
producers. Focused contract tests exercise selector ambiguity, lane
|
||
selection, ordered dependency closure, and execution behavior.
|
||
- **Publication and terminal outcomes:** Syntax/config failures before run
|
||
identity allocate no output or debug state. After debug allocation,
|
||
resolution, profile, preparation, input, framework, partial-summary, and
|
||
output failures all pass through `failPipelineCommand`. `terminalize` writes
|
||
a run report once, preserves an existing primary failure over report/error-log
|
||
failures, promotes a success-report failure to primary, and reports other
|
||
persistence failures secondarily. Framework cancellation follows the same
|
||
wrapped primary-error path. Durable outputs are attempted only after runner
|
||
success; a JSON result is encoded before output publication but written to
|
||
stdout only after output and debug terminalization. A receipt-delivery
|
||
failure leaves already published bundles intact and returns failure.
|
||
- **Test ownership:** Configuration tests own strict file/env application,
|
||
structural validation, effective cloning/digests, and redaction. CLI command,
|
||
run, reference, recomputation, session, production, example, result, and state
|
||
contracts assert process-level ordering and side effects rather than merely
|
||
repeating lower-level resolver assertions. The two uncovered command/parser
|
||
cases are recorded as CFGCLI-001 and CFGCLI-002.
|
||
|
||
### Pipeline Resolution, Preparation, And Typed Registries
|
||
|
||
- **Static composition:** `ResolvePipeline` rejects mixed legacy/ordered
|
||
shapes, empty and duplicate normalized step/lane identities, invalid
|
||
invocation filtering, unknown modules, missing capabilities, incompatible
|
||
artifact variants, unsupported lane-level validators, invalid generated
|
||
selectors, and missing output capabilities before producing a resolved
|
||
value. Apart from PIPE-001's direct-call collision, selected lanes and steps
|
||
have stable sorted/order-preserving identities.
|
||
- **Effective policy and identity:** Execution classes come from normalized
|
||
registry specs. Command override → binding → pipeline LLM-profile precedence
|
||
applies only to selected LLM-backed modules and validators; deterministic
|
||
bindings reject explicit profiles. Module and validator option validators
|
||
receive owned maps before `resolvedPipelineDigest` hashes the complete
|
||
effective composition. Digest tests cover map canonicalization, validator
|
||
policy, artifact schema identity, effective profiles, and exclusion of the
|
||
digest field itself.
|
||
- **Typed registry boundary:** Extractor registrations retain one exact Go type;
|
||
merger, normalizer, and typed-validator registrations select an exact
|
||
module/artifact-kind variant; codecs validate complete schema/media identity;
|
||
and evidence projectors must match the active codec type. Every erased
|
||
operation checks the implementation or value type and returns an error rather
|
||
than asserting or panicking. Kind-neutral `Spec` methods are confined to
|
||
catalog inspection, while behavior-sensitive resolution uses exact variant
|
||
lookups.
|
||
- **Construction boundary:** `Prepare` validates the resolved shape and needed
|
||
registries, clones retained bindings, options, validator chains, reference
|
||
targets, schemas, and bytes, then constructs input, chunker, chunk validators,
|
||
every ordered typed lane and local validator chain, output, and the optional
|
||
evidence plan before returning. Implementations and operations remain
|
||
private; public prepared bindings/lanes are separate clones. Focused tests
|
||
verify deterministic construction order, late failure before input parsing,
|
||
nil and identity rejection, generated-selector retention, and independent
|
||
builder reference inputs. PIPE-002 records only the extra adjacent copies.
|
||
- **Checkpoint supplements:** Preparation collects component-provided semantic
|
||
fingerprints only after the complete implementation set exists. Scopes
|
||
include stage, globally unique lane identity, module, and validator position;
|
||
empty or duplicate values fail preparation, results are sorted, and the
|
||
accessor returns a defensive copy. Scheduling limits and diagnostics are not
|
||
included. Resolved composition—including options, reference selectors,
|
||
effective profiles, retries, and validator order—remains owned by the
|
||
resolved digest rather than being redundantly restated as component
|
||
fingerprints.
|
||
- **Registry comparison:** Input, chunker, and output registries consistently
|
||
normalize keys/specs, reject nil validators/builders, validate options on
|
||
owned maps, clone stored specs, sort discovery output, and verify constructed
|
||
identity. Typed stage and validator registries add only the exact-type and
|
||
artifact-variant mechanics their contracts require. Validator-chain lookup
|
||
distinguishes absent, default, explicit replacement, and explicit empty
|
||
chains while returning defensive copies. No dead compatibility path or safe
|
||
consolidation was found beyond the copy reduction in PIPE-002.
|
||
- **Test ownership:** Contract tests cover clone/serialization boundaries;
|
||
registry tests cover invalid registration, sorted/defensive discovery,
|
||
strict options, exact types, schema compatibility, and evidence ownership;
|
||
resolution tests cover heterogeneous variants and effective identity; and
|
||
preparation tests own all-before-parse construction and fingerprint
|
||
collection. The missing module-reference collision case is recorded in
|
||
PIPE-001 rather than as a separate test-only finding.
|
||
|
||
### References And Ordered Handoffs
|
||
|
||
- **External path:** Pipeline defaults are filtered to declared slots; local
|
||
bindings override eligible defaults; CLI overrides apply to one exact final
|
||
target; and unbind removes only an external binding before required-slot
|
||
enforcement. Selected legacy lanes determine eligible targets, while
|
||
explicit ordered steps reject `--only`. Config-relative and CLI-relative
|
||
paths remain distinct, materialized values retain digest/media/size/origin,
|
||
preparation supplies owned construction inputs, and operation requests get
|
||
independent content. REF-001 records the only missing bound in this path.
|
||
- **Generated path:** Pipeline-level selectors are rejected. Ordered local or
|
||
step bindings must name a declared earlier step and selected lane whose exact
|
||
artifact kind, registered codec media type, and target slot constraints are
|
||
compatible. Because all edges point strictly backward, forward references
|
||
and cycles are rejected during resolution. At the step barrier, exactly one
|
||
accepted normalized output is decoded and re-encoded through its codec;
|
||
missing, duplicate, wrong-kind, wrong-schema/media, oversized, rejected, or
|
||
unavailable producer state stops the consumer before execution. REF-002
|
||
concerns only the repeated lookup used to establish that cardinality.
|
||
- **Resume and recomputation:** Ordinary resume progressively compares
|
||
generated dependency fingerprints on extract, merge, and normalize
|
||
checkpoints. Selective recomputation instead forces the selected step and
|
||
transitive consumers while requiring unforced transitive producers to supply
|
||
accepted normalize state without extract/merge files or dependency matches.
|
||
Hydration validates stored provenance and canonical bytes through the active
|
||
codec before publishing an owned normalized output. REF-003 records the
|
||
omitted size field in the ordinary consumer fingerprint.
|
||
- **Provenance and evidence:** External manifests contain paths, digests,
|
||
media, sizes, and binding sources; generated manifests contain bounded
|
||
producer, codec/schema, digest, and size identity without content or a fake
|
||
path. Runtime stage debug envelopes omit reference sets, contract JSON omits
|
||
`ReferenceItem.Content`, and focused assembled-module tests confirm generated
|
||
references ground operations without becoming source evidence.
|
||
- **Complexity ownership:** Static target resolution keeps precedence,
|
||
unbinding, and required-slot policy together for contextual errors;
|
||
`validateGeneratedBindings` owns cross-step static compatibility; and
|
||
`buildStepReferenceSets` owns the runtime barrier. Apart from REF-002's
|
||
repeated full-output scan, their explicit traversals preserve distinct
|
||
invariants more clearly than a generic graph or reflection engine.
|
||
- **Test ownership:** Profile and CLI reference contracts cover defaults,
|
||
local precedence, unbinding, required slots, selected targets, ambiguity,
|
||
and typed producer compatibility. Reference materialization tests cover
|
||
origin, UTF-8, media, size diagnostics, warnings, and provenance; handoff and
|
||
runner tests cover canonical fanout, invalid/missing producers, ordering,
|
||
fingerprints, and accepted checkpoint hydration; assembled integration tests
|
||
cover semantic generated-reference consumers. REF-001 and REF-003 identify
|
||
the two unproved identity/resource details.
|
||
|
||
### Execution, Validation, Retry, And Concurrency
|
||
|
||
- **Runtime state diagram:** The audited framework lifecycle is:
|
||
|
||
```text
|
||
input validation / owned metadata
|
||
-> source load or parse -> document validation
|
||
-> chunk load or plan -> materialize -> whole-plan validator chain
|
||
-> rejected: terminal ordinary outcome ----------------------+
|
||
-> accepted: for each ordered step |
|
||
-> generated handoff barrier |
|
||
-> initialize lane checkpoint states |
|
||
-> dispatch extract jobs (chunk first, lane second) |
|
||
-> operation -> canonical candidate -> validators |
|
||
-> retry | rejected chunk | accepted chunk |
|
||
-> completed lane enters bounded continuation |
|
||
-> merge -> validators -> retry/reject |
|
||
-> normalize -> validators -> retry/reject |
|
||
-> await all started work -> stable lane merge -> barrier |
|
||
+---------------------------------------------------------------+
|
||
-> manifest/evidence -> logical output encoder
|
||
|
||
any framework error or parent cancellation
|
||
-> cancel derived work -> stop dispatch -> drain/await started work
|
||
-> stable error selection -> failed output (no logical files)
|
||
```
|
||
|
||
- **Ordering and bounded work:** `dispatchExtractJobs` enumerates chunks first
|
||
and lanes second. One fixed extract pool and one fixed continuation pool are
|
||
each bounded by `ExtractWorkers`; a lane continuation is serial merge then
|
||
normalize, while different completed lanes may overlap remaining extracts.
|
||
Results are indexed by lane and chunk, extracts are sorted by chunk index,
|
||
lane outputs merge in prepared order, and framework errors sort by stage,
|
||
lane, and chunk after child cancellation errors are filtered. Reverse-
|
||
completion, continuation-overlap, continuation-bound, provider-bound, and
|
||
stable-error tests confirm these contracts independently of goroutine finish
|
||
order.
|
||
- **Cancellation and liveness:** The lane engine checks cancellation before
|
||
starting queued extract and continuation work, stops dispatch, clears
|
||
unlaunched continuations, drains worker results, receives every launched
|
||
completion, and waits for both pools. The retry loop checks before an attempt
|
||
and after failed/rejected attempts. Parent cancellation takes precedence over
|
||
collected child cancellation, while an actual framework failure is selected
|
||
deterministically when the parent remains live. RUN-001 records the two
|
||
unguarded outer dispatch boundaries; no additional worker leak or deadlock
|
||
path was found. RUN-004 records the missing collector invariant comment.
|
||
- **Terminal outcomes:** Chunk, extract, merge, and normalize validators inspect
|
||
whole candidates, not partial streams. A rejection is collected as ordinary
|
||
output: a rejected chunk plan skips all lanes, a rejected extract chunk does
|
||
not prevent accepted chunks in that lane from merging, and merge/normalize
|
||
rejection terminates only that lane. A framework error cancels sibling work,
|
||
blocks later ordered steps, and returns through `failOutput`, which leaves no
|
||
logical output files. RUN-002 records the typed-value ownership violation;
|
||
serialized validators receive owned canonical bytes.
|
||
- **Retries and diagnostics:** Operation and complete validator-chain execution
|
||
share one retry budget. Debug persistence failure is terminal rather than
|
||
retried; cancellation stops retries; rejections record their final attempt;
|
||
and normalize retry directives use the same budget before validating their
|
||
final fallback. Candidate serialization precedes validation and accepted
|
||
checkpoint serialization follows it, so invalid candidates are not recorded
|
||
as success. Intermediate-attempt warnings remain only in attempt debug as
|
||
intended; RUN-003 records loss of the final rejected attempt's warnings.
|
||
- **Collaborator synchronization:** Runner-local wrappers serialize checkpoint,
|
||
debug, and LLM-debug collaborators without acquiring multiple framework
|
||
locks at once. Module/provider concurrency remains separately bounded by the
|
||
shared LLM scheduler. No duplicate recording or permit leak was found in the
|
||
audited execution layer; durable checkpoint transitions and filesystem
|
||
atomicity remain in the next audit area.
|
||
- **Focused test review:** Concurrency tests exercise chunk-first dispatch,
|
||
reverse completion, extract and continuation bounds, intended overlap,
|
||
provider limits, stable stage/lane error priority, ordinary rejection, parent
|
||
cancellation, and stopped queued extracts. Retry, rejection, candidate-
|
||
encoding, session, manifest, typed-checkpoint, debug-attempt, accepted-
|
||
checkpoint, and generated-handoff tests cover the remaining execution
|
||
branches. The consequential missing runtime cases are included in RUN-001
|
||
through RUN-003 rather than duplicated as test-only findings.
|
||
|
||
### State, Checkpoints, Debugging, And File Safety
|
||
|
||
- **Independent state lifecycles:** `runPipelineCommand` selects output,
|
||
chunk-plan, checkpoint, and debug roots only in the CLI. Bypass returns before
|
||
resolving or constructing a chunk-plan store; disabled checkpoints construct
|
||
only no-op collaborators and reject resume; enabled recording always creates
|
||
a recorder but creates a loader only for resume; and debug allocation occurs
|
||
only after an explicit debug request. None of these roots enters a module or
|
||
another state family's identity, and debug records are never read by reuse
|
||
code.
|
||
- **Chunk-plan cache:** The store accepts only a canonical lowercase SHA-256
|
||
source digest as its key, opens the exact digest directory through `os.Root`,
|
||
rejects symlink/non-directory digest entries and symlink/non-regular plan
|
||
files, strictly decodes one schema-versioned JSON value, revalidates plan
|
||
provenance and digest, and returns typed missing/invalid/hit decisions. A
|
||
reused plan is cloned, materialized against the current document, and passed
|
||
through the whole-plan validators. Only a newly accepted plan is published;
|
||
random 0600 temporary files are synced and renamed within the confined 0700
|
||
entry. Missing, invalid, or deleted entries are reconstructible.
|
||
- **Checkpoint identity and ordinary resume:** Identity hashes the complete
|
||
resolved pipeline, raw input, selected lanes, runtime overrides, external and
|
||
generated reference provenance, observed profile/runtime provenance, and
|
||
prepared component fingerprints. The directory uses readable/prefix
|
||
components, while every manifest retains and validates the complete identity
|
||
digest and exact stage/step/lane/module/dependencies. Payload publication
|
||
precedes a succeeded manifest, and the manifest retains payload digests, so
|
||
interruption or a changed dependency produces execution/invalidation rather
|
||
than stale reuse. STATE-001 records the remaining non-injective step/lane
|
||
component mapping.
|
||
- **Selective recomputation:** The CLI computes a forward forced closure and a
|
||
backward set of required reusable producers. The runner applies
|
||
`forced_recompute` before decoding an otherwise reusable artifact. Required
|
||
producers use only an accepted workspace-v3 normalize manifest with exact
|
||
invocation, producer provenance, artifact digest, active codec/schema/media,
|
||
and canonical bytes; they deliberately do not require extract/merge state or
|
||
current consumer dependencies. A failed required load records its typed
|
||
decision and stops before producer or consumer execution.
|
||
- **Decisions and diagnostics:** Checkpoint readers assign category and reason
|
||
code at the validation branch for missing, path, read, decode, schema,
|
||
identity, stage, dependency, payload, digest, and reuse outcomes. Runner
|
||
hydration adds codec and canonicality codes, forced policy adds the recompute
|
||
code, and required-predecessor errors name only stable step/lane identity and
|
||
code. Event detail is selected from code-owned text, UTF-8 normalized, and
|
||
bounded rather than copied from a cache error or caller payload.
|
||
- **File safety and recovery:** Output names are all validated before exclusive
|
||
run-directory creation; existing output/debug run directories are refused;
|
||
later output failure intentionally retains the new partial bundle. General
|
||
state writes use same-directory temporary files and atomic rename after
|
||
narrow relative-path checks, with 0700/0600 cache/debug modes and 0755/0644
|
||
output modes. General file I/O rejects existing symlink components; the
|
||
chunk-plan store additionally uses root-relative no-follow opens and
|
||
race-oriented entry rechecks. No production path automatically moves,
|
||
deletes, or cleans output, cache, or completed debug state; debug allocation
|
||
removes only its just-created partial leaf on setup failure.
|
||
- **Debug and terminalization:** A bundle owns distinct summary and trace
|
||
directories. Summary records use redacted invocation/configuration and
|
||
bounded checkpoint decisions; trace requests receive cloned application
|
||
payloads and no environment enumeration. Debug write failures fail the
|
||
requested run but cannot affect checkpoint identity or reuse. Guarded
|
||
terminalization writes one report, preserves an existing primary error over
|
||
report failure, attempts one error log, and joins persistence failures only
|
||
as secondary diagnostics. STATE-001 records trace-name collisions, not a
|
||
cache dependency or redaction leak.
|
||
- **Canonical ownership:** Source digesting uses canonical document/plan
|
||
serialization and cycle-aware metadata cloning. Chunk-map and evidence-
|
||
context codecs strictly validate schema, reject unknown fields and trailing
|
||
values, canonicalize nested identities/annotations/context, and return owned
|
||
graphs. Their encode-time clones are required ownership boundaries;
|
||
STATE-002 records only the redundant copies after decoding already-owned
|
||
JSON. STATE-003 records the one extractable duplicate writer; checkpoint
|
||
stage adapters, cache-specific `os.Root` mechanics, and type-specific clone
|
||
helpers remain intentional.
|
||
- **Focused test review:** File-I/O and debug-bundle tests cover confinement,
|
||
atomic replacement, symlinks, modes, refusal, cleanup, and redaction.
|
||
Checkpoint tests cover identity, manifests, interrupted publication,
|
||
categories/reason bounds, dependency invalidation, accepted normalize, and
|
||
codec canonicality. Chunk-plan tests cover corrupt entries, races, named
|
||
pipes, strict decode, materialization, and permissions; chunk-map and
|
||
evidence-context tests cover canonical round trips and ownership. CLI cache,
|
||
recomputation, state-hardening, run-contract, and terminal tests cover the
|
||
composed lifecycles and primary-error precedence. The uncovered identity,
|
||
copy, and writer issues are recorded as STATE-001 through STATE-003.
|
||
|
||
### LLM Runtime, Prompt Filesystems, And Assets
|
||
|
||
- **Completion path:** Every production structured completion crosses the
|
||
scheduled client and the shared scheduler before the PromptKit adapter. The
|
||
adapter forwards the stable session ID directly, renders inputs and variables
|
||
once, prepares one frozen execution target, snapshots its details, invokes
|
||
that exact prepared target, accepts only PromptKit schema-valid structured
|
||
output, decodes into the caller's target, records normalized profile
|
||
provenance, and emits caller-owned debug material. Context and capacity are
|
||
adapted to provider-neutral categories; LLM-001 records the raw error chain
|
||
that still escapes on other provider failures.
|
||
- **Scheduling:** Production constructs one scheduler around the one PromptKit
|
||
client, so all LLM-backed bindings and validators share the configured
|
||
application limit. Queue insertion and grant are FIFO, active counts change
|
||
under one mutex, canceled queued entries are removed or return an already
|
||
granted permit, and idempotent deferred release covers success and error.
|
||
LLM-002 records only the simultaneous grant/cancel dispatch window; no permit
|
||
leak or second application scheduler was found.
|
||
- **Profiles and cache identity:** Preflight inspection and runtime construction
|
||
share configured directory/file, embedded fallback, built-in catalog, and
|
||
optional local-backend option builders while retaining separate engines.
|
||
Inspection does not load credentials or contact a provider. Checkpoint
|
||
fingerprints cover the compiled catalog identity, every configured profile
|
||
file's content identity, fallback asset identity, reasoning override, and a
|
||
hashed local endpoint without publishing profile contents, paths, endpoint,
|
||
environment values, or concurrency limits. Successful manifests record only
|
||
selected non-secret profile/provider/model/backend/reasoning provenance.
|
||
- **Sessions, bytes, and cache points:** The CLI resolves one stable session ID
|
||
before physical run construction and the same value flows through pipeline
|
||
requests into PromptKit's native `SessionID`; it is not reconstructed from a
|
||
transcript. All maintained extraction prompts render a stable shared system,
|
||
identity/reference prefix before the transcript, then lane-specific evidence
|
||
and grounding, with the final instructions last. Focused composition tests
|
||
assert identical cached prefixes, exact-once input placement, suffix order,
|
||
and ephemeral cache controls at reference, transcript, and final-instruction
|
||
boundaries.
|
||
- **Asset ownership:** The root `assets` package imports only `embed` and
|
||
`io/fs` and exposes one read-only filesystem. Module manifests explicitly
|
||
flatten only selected module files and shared fragments; the asset registry
|
||
clones bytes, rejects duplicate flattened prompt/schema/profile roots,
|
||
validates schema manifests, and hashes selected semantic assets. All fourteen
|
||
D&D prompt manifests currently use unique normalized virtual names; LLM-003
|
||
records that `ModulePromptFS` does not enforce this invariant itself.
|
||
- **Model-visible content:** Every maintained PromptKit YAML file, shared
|
||
fragment, module instruction/grounding file, private response schema, and
|
||
fallback profile was searched and representative complete compositions were
|
||
read. The only provider/model-named content is the intentional fallback
|
||
profile target. No bearer token, credential variable/value, endpoint,
|
||
provider-generated opaque identifier, UUID, digest-copy instruction, or
|
||
duplicated model instruction was found in model-visible assets. Durable
|
||
private-schema `$id` values remain schema metadata rather than requests to
|
||
reproduce internal identifiers.
|
||
- **Filesystem mechanics and tests:** Asset and module prompt filesystems
|
||
enforce valid scoped paths, sorted directory entries, read-only metadata, and
|
||
owned bytes, while prompt/schema loaders are exercised through real
|
||
PromptKit preparation. LLM-004 records the duplicated map-filesystem
|
||
implementation and its drift. Focused client tests cover direct sessions,
|
||
prepared-snapshot consistency, default/explicit/fallback/local profiles,
|
||
response validation/decode, capacity mapping, debug redaction, profile
|
||
provenance, and scheduled concurrency; scheduler and promptfs tests cover
|
||
limits, queued cancellation, release, path scoping, reads, and ownership.
|
||
|
||
### Generic And Seriatim Modules
|
||
|
||
- **Seriatim source boundary:** The transcript adapter strictly decodes one
|
||
top-level JSON object, requires metadata and a non-empty segment array,
|
||
validates unique positive canonical IDs, non-empty speaker/text values, and
|
||
finite ordered timestamps, then chooses a deterministic document ID from the
|
||
request, metadata, or raw-byte digest. It constructs generic units with
|
||
exact self-references, computes the canonical source digest, and runs the
|
||
framework document validator. Transcript-only speaker/start/end helpers and
|
||
external metadata remain in the adapter package; the resulting document
|
||
exposes only generic source fields and metadata.
|
||
- **Generic chunk planning:** The unit chunker validates the complete source
|
||
document, computes source-addressed ranges by unit position with bounded
|
||
overlap, returns only the source digest and unit-ID ranges, and leaves
|
||
materialization, chunk identity, and plan validation to the framework. Tests
|
||
cover defaults, exact boundaries, overlap, invalid/unknown options, empty
|
||
sources, and non-retention of source units. MOD-001 records the only
|
||
avoidable option-decoder surface found.
|
||
- **Merge, normalize, and validation ownership:** Typed append-order combines
|
||
values in framework order through a family-supplied typed combiner, while
|
||
typed no-op normalization preserves the exact artifact type; D&D registrars
|
||
instantiate those domain-neutral mechanics without introducing D&D imports
|
||
into the generic packages. Always-accept/reject, JSON, and JSON-Schema
|
||
validators retain distinct selectable targets and strict empty option sets.
|
||
Schema mismatch remains an ordinary rejection and malformed schema remains
|
||
an operational error. MOD-002 records only repeated schema compilation.
|
||
- **JSON output and evidence:** Output decoding rejects unknown and
|
||
context-incompatible fields, normalizes/deduplicates/sorts evidence lane
|
||
allowlists, and validates configured lanes before preparation. Preparation
|
||
retains the full configured policy while selecting active exact-typed
|
||
evidence projectors; runtime evidence is built only from compatible accepted
|
||
normalized outputs. The encoder sorts lanes and logical files, rejects
|
||
unsafe or colliding lane filenames, emits no physical paths, validates and
|
||
republishes opt-in chunk-map and prepared evidence-context artifacts, and
|
||
preserves annotation number bytes. MOD-003 records only unreachable clone
|
||
code outside those live ownership paths.
|
||
- **Registration and dependency direction:** Generic registration checks all
|
||
required registries before mutation and registers the unit chunker, four
|
||
validators, and JSON output; Seriatim independently registers only its input
|
||
adapter. Direct import inspection found no D&D package in either family and
|
||
no Seriatim package below the input registrar. Typed append/no-op libraries
|
||
are instantiated by the D&D family rather than acquiring domain concepts.
|
||
- **Composed contracts:** Focused module tests cover strict registration,
|
||
source parsing/identity/metadata, chunk planning, typed combination,
|
||
validator outcomes, JSON bundle determinism, chunk-map/evidence validation,
|
||
and mutation isolation. Framework evidence-preparation/output tests prove
|
||
invocation filtering and exact codec/projector compatibility; maintained CLI
|
||
example contracts prove Seriatim input through generic JSON publication and
|
||
opt-in chunk-map provenance.
|
||
|
||
### Shared D&D Types, Codecs, And Family Mechanics
|
||
|
||
- **Durable family inventory:** `internal/modules/dnd/types.go`, the ten codec
|
||
packages, and D&D registration agree on ten exact kind/type variants. Each
|
||
has one LLM extractor, one typed append-order merger, a concrete normalizer,
|
||
a typed noop alternate, an evidence projector, artifact-specific validators,
|
||
and extract/normalize default chains. The convention matrix above records the
|
||
three intentionally LLM-backed registry normalizers and every reference
|
||
dependency; no missing or extra durable family registration was found.
|
||
- **Codec boundary:** All family codecs publish `application/json`, codec-local
|
||
v1 durable schemas and count metadata, strictly decode through the shared
|
||
candidate JSON helper, reject unknown fields and trailing values, and apply
|
||
family required-value validation only at durable encode/decode. Ownership
|
||
clones are present for the two mutable nested shapes that need them, and
|
||
schema reads return owned bytes. The common adapter surface is already at its
|
||
natural owner; a further generic codec would have to parameterize the exact
|
||
behavior the typed boundary is meant to expose.
|
||
- **Reference mechanics:** Shared campaign slots return owned media-type and
|
||
slot declarations; source ordering uses document positions with literal-ID
|
||
fallback, stable exact deduplication, and diagnostic preservation of invalid
|
||
refs. Strict unit-reference JSON parsing, current-document citation
|
||
expansion, typed family canonicalization, validator admissibility, and direct
|
||
evidence projection remain distinct owners. DND-CORE-001 records the only
|
||
dead/lossy API found in this layer.
|
||
- **Shared registry and reconciliation mechanics:** The generic registry
|
||
resolver validates one typed JSON artifact, bounds it by the family-declared
|
||
maximum, copies content at retention boundaries, and caches raw and semantic
|
||
identities behind a mutex. Entity reconciliation renders source-free
|
||
selectors and bounded transcript windows, keeps durable IDs model-invisible,
|
||
discards unsafe or overlapping proposals, and returns defensive copies.
|
||
Diagnostics and Unicode comparison policy are shared and bounded/versioned;
|
||
registry-specific identity and merge outcomes remain deferred.
|
||
- **Registration and assets:** The registrar validates all collaborators before
|
||
mutation and gives every failed component contextual identity. All ten
|
||
durable schemas, fourteen family prompt manifests, the shared reconciliation
|
||
schema, evidence projectors, generic validator capabilities, typed
|
||
always-accept/reject capabilities, default chains, and the fallback profile
|
||
are registered. Sequential registration is intentionally fail-stop rather
|
||
than transactional; production discards the composition on any error.
|
||
- **Repeated method classification:** Module-local metadata and fingerprint
|
||
methods enumerate semantic cache/checkpoint inputs; constructors and
|
||
registrars preserve exact dependencies and typed error context; strict empty
|
||
option decoders make supported surfaces locally visible. Existing shared
|
||
helpers already consolidate prompt assets, candidate JSON, reference slots,
|
||
comparison, diagnostics, resolution, and reconciliation. No additional
|
||
callback- or reflection-driven helper reduced demonstrated drift.
|
||
- **Deferred lane questions:** Registry identity, reconciliation, immutable
|
||
projections, occurrence-category/grounding policy, spell catalogs, scene
|
||
chunking, scene identity, and scene classification are now confirmed,
|
||
with DND-REG-001 through DND-REG-004 and DND-OCC-001 through DND-OCC-003
|
||
plus DND-SCENE-001 and DND-SCENE-002 recording the bounded gaps. Combat gates,
|
||
engagement uniqueness, collective labels, and enemy observation ordering
|
||
remain assigned to the combat/enemy review. This area therefore remains
|
||
`Revisit` until that review confirms the remaining matrix exceptions.
|
||
|
||
### NPC, Item, And Location Registries
|
||
|
||
- **End-to-end artifact path:** Each LLM extractor decodes a private
|
||
name/evidence response, injects the current source ID, canonicalizes direct
|
||
citations, orders candidates by earliest evidence, and derives family-owned
|
||
IDs before the typed append-order merger. Normalizers own display cleanup,
|
||
exact duplicate consolidation, proposal-only semantic reconciliation,
|
||
deterministic fallback, final identity derivation, and owned output. Shape,
|
||
identity, current-source reference, schema, and advisory relatedness checks
|
||
remain ordered in the declared chains before canonical codec publication.
|
||
Generated references cross the strict typed codec boundary; external
|
||
approved references use the immutable family registry loaders. DND-REG-001
|
||
records the structural rule missing from that direct path, and DND-REG-002
|
||
records the NPC extractor's missing current-chunk evidence gate.
|
||
- **Identity and lookup policy:** NPC and item identity is the versioned Unicode
|
||
comparison name hashed through compact JSON, with duplicate comparison names
|
||
rejected after normalization. Location identity adds the lexicographically
|
||
earliest valid source anchor, intentionally permitting equal display names at
|
||
distinct anchors. Family registries validate IDs, clone retained records and
|
||
citations, build exact ID/name lookups where supported, publish semantic
|
||
durable and projection digests, and return defensive copies. DND-REG-004
|
||
concerns only repeated work in selecting the location anchor.
|
||
- **Reconciliation safety:** All three normalizers render source-free candidate
|
||
keys with bounded transcript context, invoke the shared strict reconciliation
|
||
schema only when at least two eligible candidates exist, reject unknown,
|
||
colliding, duplicate, or overlapping groups, apply only disjoint safe groups,
|
||
and retry discarded proposals before deterministic warning-bearing fallback.
|
||
NPCs consolidate aliases under the selected canonical name; items additionally
|
||
refuse groups that mix currency with non-currency or conflicting
|
||
denominations; locations union citations and recompute anchored identity.
|
||
DND-REG-003 records only the pre-reconciliation exact-group scan.
|
||
- **Proper-name and currency behavior:** NPC extraction requires a proper name,
|
||
stable title, or individually identifying alias and excludes players,
|
||
generic roles, groups, and hypotheticals. Location extraction requires a
|
||
stable physical proper or uniquely identifying designation and excludes
|
||
generic, relative, transient, and merely descriptive references. Item
|
||
extraction keeps named/unique concrete reusable objects, materially distinct
|
||
stable types, and each currency denomination while excluding generic or
|
||
vague mentions. Normalization proposes equivalence but cannot invent records,
|
||
source evidence, or unsupported canonical names.
|
||
- **Location grounding:** Immutable location grounding groups comparison names,
|
||
exposes name-only selectors for unique names, and exposes exact canonical
|
||
ranges plus cited context only when same-name records need disambiguation.
|
||
Durable IDs and source IDs remain model-invisible, selectors resolve by exact
|
||
contextual identity, collisions and invalid contextual citations fail
|
||
construction, and prompt/resolved values are defensively copied.
|
||
- **Test ownership:** Extractor tests own prompt manifests, strict response
|
||
decoding, citation adaptation, order, and fingerprints; identity/registry
|
||
tests own compact hash inputs, same-name behavior, lookup/digest projections,
|
||
direct resolution, empty placeholders, grounding, and mutation isolation;
|
||
normalizer tests own exact/semantic consolidation, currency and location
|
||
safety gates, diagnostics, retry/exhaustion fallback, context, and prompt
|
||
fingerprints; validator tests own shape deferral, identity diagnostics,
|
||
current-source/chunk evidence, relatedness warnings, and bounded messages.
|
||
The prescribed package suite passes.
|
||
|
||
### NPC, Item, And Location Occurrences
|
||
|
||
- **End-to-end selection path:** Each extractor renders semantic registry
|
||
selectors without durable IDs, decodes a private factual occurrence
|
||
response, resolves every selection against the immutable required registry,
|
||
rejects the whole candidate on an unknown or ambiguous selector, injects the
|
||
current source ID, canonicalizes current-chunk citations, and attaches the
|
||
exact durable ID/name pair before the typed append-order merger. NPC and item
|
||
selectors are exact names because those registries prohibit comparison-name
|
||
collisions; location selectors add exact canonical evidence and bounded
|
||
cited context only for same-name records. DND-OCC-001 records the one private
|
||
response vocabulary that contradicts the shared evidence instruction.
|
||
- **Evidence and identity separation:** Registry prompt and grounding
|
||
projections omit durable/source IDs, and registry provenance is never copied
|
||
into occurrence evidence. Extract validators require direct current-source,
|
||
current-chunk evidence; normalizers and registry validators independently
|
||
re-establish exact ID/name pairs for artifacts entering through other
|
||
boundaries. Generated-reference checkpoint dependencies bind the complete
|
||
producer content/provenance while semantic registry projection digests bind
|
||
selection behavior, so an approved registry change cannot silently reuse a
|
||
stale occurrence result.
|
||
- **Canonicalization and family policy:** All three normalizers preserve nil
|
||
versus present-empty list meaning, retain invalid data long enough for
|
||
ordered diagnostics, canonicalize evidence without mutating input, sort
|
||
deterministically, collapse only exact valid-evidence duplicates, and bound
|
||
warnings. Their local policies intentionally remain distinct: NPC categories
|
||
encode factual presence/dialogue/combat roles; item identity includes
|
||
quantity and directionally constrained holders; location ordering uses an
|
||
explicit semantic kind precedence and retains same-name records with
|
||
different durable IDs. A generic occurrence normalizer would obscure these
|
||
contracts. DND-OCC-003 concerns only avoidable repeated canonicalization
|
||
inside the item implementation.
|
||
- **Boundary and reference review:** Durable codecs strictly reject unknown
|
||
fields, trailing values, absent lists, empty required values, unsupported
|
||
kinds, and item holder/quantity violations. DND-REG-001 now also covers the
|
||
shared reversed-range structural omission at the occurrence codecs. Extract
|
||
campaign context remains legitimate model grounding, while deterministic
|
||
normalizers require only their immutable registry; DND-OCC-002 records the
|
||
NPC normalizer's four inert campaign slots rather than changing extraction.
|
||
Family-specific shape, registry, source, relatedness, and invariant
|
||
validators remain separate because their ordered deferral and source-aware
|
||
responsibilities differ from durable decoding.
|
||
- **Generated handoffs and tests:** Production resolution requires an earlier
|
||
compatible generated producer or a strictly decoded compatible external
|
||
registry for each family. The representative assembled D&D flow proves exact
|
||
registry IDs, independent occurrence evidence, and distinct same-name
|
||
location selection through extraction and normalization. Extractor tests own
|
||
prompt/schema adaptation and all-or-nothing resolution; normalizer tests own
|
||
ordering, exact duplicates, holders/quantities, same-name locations, warning
|
||
order, and ownership; validator tests own pair, source, relatedness, and
|
||
invariant defenses. The prescribed occurrence and CLI suites pass.
|
||
|
||
### Spells, Scene Chunking, And Scene Descriptions
|
||
|
||
- **Accepted scene-plan path:** The scene chunker validates the complete source,
|
||
supplies the whole transcript after optional cached campaign context, and
|
||
decodes only ordered start/end unit IDs. `planFromResponse` resolves those
|
||
IDs against document positions and enforces non-empty, contiguous,
|
||
non-overlapping first-to-last coverage. The framework then canonicalizes,
|
||
validates, and materializes the plan before storing it. Only an accepted plan
|
||
is serialized into the distinct chunk-map artifact with exact source,
|
||
requested-chunker, producer, plan-digest, chunk-ID/range/count, and cloned
|
||
annotation metadata; lane rejection does not erase that accepted structural
|
||
artifact, while plan rejection publishes no chunk map. The D&D scene planner
|
||
deliberately emits empty annotations rather than duplicating classification
|
||
or source text, and the production contract verifies that separation.
|
||
- **Scene-description identity and eligibility:** Extraction maps exactly one
|
||
private kind/title/summary response onto the local current chunk ID and exact
|
||
range, so the model cannot select or invent durable identity. Extract
|
||
validation rechecks that exact chunk match; normalization uses source-document
|
||
positions, rejects ID/range conflicts, and removes only exact duplicates.
|
||
The immutable registry strictly decodes one approved artifact and projects
|
||
only ID, complete range, and kind into a semantic eligibility digest. Its
|
||
exact/missing/mismatched match state prevents title, summary, source prose,
|
||
or registry provenance from becoming later combat evidence. DND-REG-001 and
|
||
DND-SCENE-002 record the bounded codec and diagnostic exceptions.
|
||
- **Spell catalog, grounding, and evidence:** Construction strictly decodes the
|
||
optional single JSON overlay, composes it with the embedded 2014 SRD catalog
|
||
in sorted overlay-ID order, rejects cross-spell canonical/alias collisions,
|
||
and binds a semantic catalog digest to extractor, catalog-validator, and
|
||
normalizer checkpoints. Optional NPC grounding is resolved independently and
|
||
projected as source-free names; neither NPC nor catalog provenance becomes
|
||
cast evidence. The extractor injects the current source ID, canonicalizes
|
||
references and cast order, and the normalizer canonicalizes catalog names and
|
||
reference order before collapsing only exact valid-evidence duplicates.
|
||
DND-SCENE-001 records the missing model-visible alias mapping, DND-REG-002 the
|
||
extraction chunk gate, and PIPE-002 the redundant construction copy.
|
||
- **Validation, retries, and cache identity:** Default spell and scene chains
|
||
preserve shape deferral, strict private and durable schemas, source-aware
|
||
rejection, advisory relatedness, and normalization invariants. Catalog
|
||
rejection participates in the framework's bounded retry path, terminal
|
||
accepted-attempt warnings are retained, and rejected-attempt warnings are
|
||
suppressed as intended. Raw reference dependencies bind provenance/content,
|
||
while semantic catalog and NPC/scene projections bind behavior; focused CLI
|
||
tests prove catalog, mapping-policy, prompt/profile, and generated-reference
|
||
changes prevent stale checkpoint reuse.
|
||
- **Prompt and test ownership:** Scene planning follows the intentional D&D
|
||
prompt order of system, cacheable campaign references, uncached task
|
||
instructions/schema, and final ephemeral whole transcript. Per-chunk scene
|
||
descriptions and spells use the shared current-chunk evidence fragments;
|
||
local response schemas expose no source IDs or chunk IDs. Focused assets,
|
||
codecs, extractors, normalizers, validators, registries, catalog composition,
|
||
checkpoint/retry, and production chunk-map tests cover the reviewed contracts,
|
||
and the prescribed package suite passes.
|
||
|
||
## Validation Record
|
||
|
||
| Date | Scope | Command or check | Result |
|
||
| --- | --- | --- | --- |
|
||
| 2026-08-08 | Initial worktree | `git status --short` | Pass; no output |
|
||
| 2026-08-08 | Knowledge graph | Full index as `notarius-audit-92e8907` | Pass; 8,322 nodes and 47,887 edges; branch/head matched the production target |
|
||
| 2026-08-08 | Baseline tests | `go test ./...` | Pass |
|
||
| 2026-08-08 | Baseline static analysis | `go vet ./...` | Pass |
|
||
| 2026-08-08 | Baseline build | `go build ./cmd/notarius` | Pass |
|
||
| 2026-08-08 | Baseline whitespace | `git diff --check` | Pass |
|
||
| 2026-08-08 | Production imports | Direct `go list` import-edge audit plus graph call tracing | Pass; no production dependency inversion found |
|
||
| 2026-08-08 | Accepted ADR links | Relative Markdown-link target scan under `docs/adr/` | Two unresolved targets recorded as ARCH-001 |
|
||
| 2026-08-08 | Audit target integrity before configuration/CLI review | `git diff --quiet 92e89076a268089e703978fb9d7176200e93344c..HEAD -- . ':(exclude)docs/roadmap/**'` | Pass; production target unchanged |
|
||
| 2026-08-08 | YAML decoder contract | `go doc gopkg.in/yaml.v3.Decoder.Decode` and `ParseFileConfigYAML` call trace | Decode consumes the next document; no EOF/second-document check, recorded as CFGCLI-001 |
|
||
| 2026-08-08 | Focused configuration and CLI tests | `go test ./internal/core/config ./internal/cli` | Pass |
|
||
| 2026-08-08 | Focused configuration and CLI static analysis | `go vet ./internal/core/config ./internal/cli` | Pass |
|
||
| 2026-08-08 | Audit target integrity before pipeline composition review | `git diff --quiet 92e89076a268089e703978fb9d7176200e93344c..HEAD -- . ':(exclude)docs/roadmap/**'` | Pass; production target unchanged |
|
||
| 2026-08-08 | Pipeline graph review | Architecture, complexity query, exact symbol reads, and call traces for `ResolvePipeline`, binding normalization, typed registries, `Prepare`, and checkpoint fingerprints | Pass; PIPE-001 and PIPE-002 recorded; generated handoff execution deferred to the next area |
|
||
| 2026-08-08 | Focused contracts and pipeline tests | `go test ./internal/framework/contracts ./internal/framework/pipeline` | Pass |
|
||
| 2026-08-08 | Focused contracts and pipeline static analysis | `go vet ./internal/framework/contracts ./internal/framework/pipeline` | Pass |
|
||
| 2026-08-08 | Audit target integrity before references/handoffs review | `git diff --quiet 92e89076a268089e703978fb9d7176200e93344c..HEAD -- . ':(exclude)docs/roadmap/**'` | Pass; production target unchanged |
|
||
| 2026-08-08 | Reference and handoff graph review | Architecture, exact symbol reads, and call traces across external materialization, target resolution, operation cloning, generated codec handoff, consumer fingerprints, ordered execution, and accepted-checkpoint hydration | REF-001, REF-002, and REF-003 recorded; no content/evidence leak found |
|
||
| 2026-08-08 | Focused pipeline and CLI tests | `go test ./internal/framework/pipeline ./internal/cli` | Pass |
|
||
| 2026-08-08 | Assembled integration tests | `go test ./internal/modules/integration/...` | Pass |
|
||
| 2026-08-08 | Audit target integrity before runtime review | `git diff --quiet 92e89076a268089e703978fb9d7176200e93344c..HEAD -- . ':(exclude)docs/roadmap/**'` | Pass; production target unchanged |
|
||
| 2026-08-08 | Runtime graph and state-machine review | Exact symbol reads, complexity inspection, call traces, cancellation-search coverage, and focused concurrency/retry/rejection/session/checkpoint/debug/manifest/handoff test review | RUN-001 through RUN-004 recorded; bounded work, stable assembly, and worker draining otherwise confirmed |
|
||
| 2026-08-08 | Pipeline race tests | `go test -race ./internal/framework/pipeline` | Pass |
|
||
| 2026-08-08 | Fresh pipeline tests | `go test -count=1 ./internal/framework/pipeline` | Pass |
|
||
| 2026-08-08 | Shuffled pipeline tests | `go test -shuffle=on ./internal/framework/pipeline` | Pass |
|
||
| 2026-08-08 | Audit target integrity before state review | `git diff --quiet 92e89076a268089e703978fb9d7176200e93344c..HEAD -- . ':(exclude)docs/roadmap/**'` | Pass; production target unchanged |
|
||
| 2026-08-08 | State graph and lifecycle review | Exact symbol reads, complexity and call traces across CLI root construction, core file I/O/debug allocation, chunk-plan storage, checkpoint identity/loading/recording, runner reuse decisions, trace/summary writers, terminalization, and canonical codecs | STATE-001 through STATE-003 recorded; independent lifecycle, typed decisions, accepted hydration, debug isolation, and primary-error precedence otherwise confirmed |
|
||
| 2026-08-08 | State collaborator race tests | `go test -race ./internal/core/fileio ./internal/core/debugbundle ./internal/framework/checkpoint ./internal/framework/chunkplan ./internal/framework/chunkmap ./internal/framework/debug ./internal/framework/evidencecontext` | Pass |
|
||
| 2026-08-08 | CLI state and terminal tests | `go test ./internal/cli` | Pass |
|
||
| 2026-08-08 | Audit target integrity before LLM/assets review | `git diff --quiet 92e89076a268089e703978fb9d7176200e93344c..HEAD -- . ':(exclude)docs/roadmap/**'` | Pass; production target unchanged |
|
||
| 2026-08-08 | LLM and asset graph/content review | Exact symbol reads and call traces across scheduled completion, PromptKit adaptation, profiles/preflight/fingerprints, sessions, debug, asset flattening/hashing, prompt filesystems, all fourteen manifests, every prompt YAML, shared/model-visible content, private schemas, and the root asset leaf | LLM-001 through LLM-004 recorded; profile parity, exact prompt order/cache points, asset ownership, and secret-free model-visible content otherwise confirmed |
|
||
| 2026-08-08 | LLM and prompt filesystem race tests | `go test -race ./internal/framework/llm ./internal/framework/promptfs` | Pass |
|
||
| 2026-08-08 | Composed CLI and D&D prompt-cache tests | `go test ./internal/cli ./internal/modules/dnd/register` | Pass |
|
||
| 2026-08-08 | Audit target integrity before generic/Seriatim review | `git diff --quiet 92e89076a268089e703978fb9d7176200e93344c..HEAD -- . ':(exclude)docs/roadmap/**'` | Pass; production target unchanged |
|
||
| 2026-08-08 | Generic and Seriatim graph/code review | Architecture clusters, exact symbol reads, call traces, scoped caller/search checks, and complete implementation/test inspection across input parsing, chunk planning, typed merge/normalize helpers, validators, JSON output, registration, evidence preparation, and output publication | MOD-001 through MOD-003 recorded; source/domain boundaries and live ownership paths otherwise confirmed |
|
||
| 2026-08-08 | Generic and Seriatim module tests | `go test ./internal/modules/generic/... ./internal/modules/seriatim/...` | Pass |
|
||
| 2026-08-08 | Published source-artifact codec tests | `go test ./internal/framework/chunkmap ./internal/framework/evidencecontext` | Pass |
|
||
| 2026-08-08 | Composed evidence/output contracts | Focused `go test` runs in `./internal/framework/pipeline` and `./internal/cli` for evidence preparation/publication, maintained minimal JSON output, and production chunk-map provenance | Pass |
|
||
| 2026-08-08 | Generic and Seriatim import boundaries | Direct `go list` import-edge audit plus scoped transcript-adapter import search | Pass; no D&D dependency in generic/Seriatim packages and no production import of the transcript adapter outside Seriatim registration |
|
||
| 2026-08-08 | Audit target integrity before shared D&D review | `git diff --quiet 92e89076a268089e703978fb9d7176200e93344c..HEAD -- . ':(exclude)docs/roadmap/**'` | Pass; production target unchanged |
|
||
| 2026-08-08 | Shared D&D graph/code review | Scoped architecture, exact symbol reads, inbound traces, and complete implementation/test inspection across ten durable types/codecs, candidate JSON, source-reference utilities, resolver/reconciliation/diagnostics, typed mergers, evidence, registration, default chains, shared assets, and representative family prompt/schema owners | DND-CORE-001 recorded; family registration, codec ownership, reference/evidence separation, shared reconciliation safety, and typed adapter boundaries otherwise confirmed |
|
||
| 2026-08-08 | Required shared D&D tests | `go test ./internal/modules/dnd/codec/... ./internal/modules/dnd/shared/... ./internal/modules/dnd/register` | Pass |
|
||
| 2026-08-08 | Audit target integrity before registry-family review | `git diff --quiet 92e89076a268089e703978fb9d7176200e93344c..HEAD -- . ':(exclude)docs/roadmap/**'` | Pass; production target unchanged |
|
||
| 2026-08-08 | Registry-family graph/code review | Exact symbol reads, complexity queries, and call traces across NPC, item, and location extraction, canonicalization, identity, immutable registries and projections, reconciliation eligibility/application/retry/fallback, normalization, validation, codecs, prompt assets, schemas, and focused tests | DND-REG-001 through DND-REG-004 recorded; name/anchor identity, same-name location, currency, proposal safety, diagnostics, fallback, and immutable projection policies otherwise confirmed |
|
||
| 2026-08-08 | Required NPC, item, and location registry tests | `go test ./internal/modules/dnd/extract/npcregistry ./internal/modules/dnd/extract/itemregistry ./internal/modules/dnd/extract/locationregistry ./internal/modules/dnd/npcs/... ./internal/modules/dnd/items/... ./internal/modules/dnd/locations/... ./internal/modules/dnd/normalize/npcregistry ./internal/modules/dnd/normalize/itemregistry ./internal/modules/dnd/normalize/locationregistry ./internal/modules/dnd/validate/npcregistry/... ./internal/modules/dnd/validate/itemregistry/... ./internal/modules/dnd/validate/locationregistry/...` | Pass |
|
||
| 2026-08-08 | Audit target integrity before occurrence-family review | `git diff --quiet 92e89076a268089e703978fb9d7176200e93344c..HEAD -- . ':(exclude)docs/roadmap/**'` | Pass; production target unchanged |
|
||
| 2026-08-08 | Occurrence-family graph/code review | Exact symbol reads, complexity queries, and call traces across NPC, item, and location semantic registry projections/resolution, response adaptation, current-source evidence attachment, canonical ordering/deduplication, normalization, validation, codecs, prompt assets, private schemas, and generated-reference tests | DND-OCC-001 through DND-OCC-003 recorded and DND-REG-001 broadened; all-or-nothing identity attachment, evidence separation, category/holder/quantity/same-name policy, nil/empty ownership, and handoff behavior otherwise confirmed |
|
||
| 2026-08-08 | Required NPC, item, and location occurrence tests | `go test ./internal/modules/dnd/extract/npcoccurrences ./internal/modules/dnd/extract/itemoccurrences ./internal/modules/dnd/extract/locationoccurrences ./internal/modules/dnd/normalize/npcoccurrences ./internal/modules/dnd/normalize/itemoccurrences ./internal/modules/dnd/normalize/locationoccurrences ./internal/modules/dnd/validate/npcoccurrences/... ./internal/modules/dnd/validate/itemoccurrences/... ./internal/modules/dnd/validate/locationoccurrences/...` | Pass |
|
||
| 2026-08-08 | Generated occurrence handoff tests | `go test ./internal/cli` | Pass |
|
||
| 2026-08-08 | Audit target integrity before spell/scene-family review | `git diff --quiet 92e89076a268089e703978fb9d7176200e93344c..HEAD -- . ':(exclude)docs/roadmap/**'` | Pass; production target unchanged |
|
||
| 2026-08-08 | Spell and scene graph/code review | Scoped architecture, exact symbol reads, complexity inspection, and call traces across whole-source scene planning, framework plan canonicalization/materialization, chunk-map serialization/publication, per-chunk scene extraction, normalization, validation and eligibility projection, effective spell catalog composition, NPC grounding, response canonicalization, normalization, validation, checkpoint identity, retry, codecs, prompt assets, schemas, and focused tests | DND-SCENE-001 and DND-SCENE-002 recorded; DND-REG-001, DND-REG-002, and PIPE-002 broadened; structural/evidence separation, exact scene eligibility, and catalog/checkpoint behavior otherwise confirmed |
|
||
| 2026-08-08 | Required spell, scene chunking, and scene-description tests | `go test ./internal/modules/dnd/chunk/scenes ./internal/modules/dnd/extract/scenedescriptions ./internal/modules/dnd/normalize/scenedescriptions ./internal/modules/dnd/validate/scenedescriptions/... ./internal/modules/dnd/scenedescriptions/... ./internal/modules/dnd/extract/spells ./internal/modules/dnd/normalize/spells ./internal/modules/dnd/validate/spells/... ./internal/modules/dnd/spells/...` | Pass |
|
||
|
||
## Coverage Matrix
|
||
|
||
| Audit area | Status | Packages and documents inspected | Validation run | Finding IDs |
|
||
| --- | --- | --- | --- | --- |
|
||
| Architecture and dependency boundaries | Reviewed | Architecture, documentation, and testing policies; internal overview; accepted ADRs; `internal/cli/catalog.go`; production registrars; root `assets` package; representative pipeline, typed-codec, output, and state-owner symbols | Full baseline, fresh graph, direct import map, cross-layer call traces, ADR link scan | ARCH-001 |
|
||
| Configuration and CLI composition | Reviewed | `docs/config.md`, `docs/cli.md`, `docs/operations.md`, internal configuration/CLI docs; `internal/core/config/`; CLI run, catalog, session, profile, result, and terminal owners; focused config, command, run, reference, recomputation, session, production, example, result, and state tests | Target-integrity check, graph call/data-owner traces, YAML decoder contract, focused tests and vet | CFGCLI-001, CFGCLI-002 |
|
||
| Pipeline resolution, preparation, and typed registries | Reviewed | Internal pipeline/module docs; `internal/framework/contracts/`; pipeline profile, options, module, construction, preparation, fingerprint, stage/validator/chain/codec/evidence registry implementations and focused tests | Target-integrity check, graph architecture/complexity/call traces, focused tests and vet | PIPE-001, PIPE-002 |
|
||
| References and ordered handoffs | Reviewed | Reference and ordered-step sections of configuration, internal pipeline, and state docs; pipeline reference resolution/materialization, preparation ownership, generated handoff, consumer fingerprint, checkpoint hydration, and runner barriers; CLI selector/recomputation owners; focused profile, reference, handoff, checkpoint, recomputation, and assembled integration tests | Target-integrity check, graph architecture/complexity/call traces, focused pipeline/CLI tests, assembled integration tests | REF-001, REF-002, REF-003 |
|
||
| Execution, validation, retry, and concurrency | Reviewed | Internal pipeline runtime contract; runner, chunk planning/validation, concurrent lane engine, typed execution/validation, retry/normalize, synchronization, output suppression, and focused concurrency, retry, rejection, session, typed-checkpoint, debug, manifest, candidate-encoding, and handoff tests | Target-integrity check, graph state-machine/call/complexity review, race tests, fresh tests, shuffled tests | RUN-001, RUN-002, RUN-003, RUN-004 |
|
||
| State, checkpoints, debugging, and file safety | Reviewed | State and operations docs plus architecture state/security policy; CLI output, cache-root, checkpoint identity, recomputation, debug allocation, and terminal owners; core file I/O, debug bundle, and source digest/clone helpers; framework checkpoint, chunk-plan, chunk-map, debug, evidence-context implementations and focused tests | Target-integrity check, graph architecture/complexity/call traces, state collaborator race tests, CLI tests | STATE-001, STATE-002, STATE-003 |
|
||
| LLM runtime, prompt filesystems, and assets | Reviewed | LLM internal/integration/configuration/operations docs and related ADRs; scheduled client, scheduler, PromptKit adapter/profile inspector, profile/source fingerprints, redaction, debug, asset registry, schema loader, promptfs implementations and focused tests; root assets, fallback profile, all fourteen module manifests and prompt YAML files, shared/model-visible fragments, private response schemas, and representative composed loaders | Target-integrity check, graph architecture/symbol/call review, content scan and exact prompt-order comparison, focused race tests, composed CLI and D&D prompt-cache tests | LLM-001, LLM-002, LLM-003, LLM-004 |
|
||
| Generic and Seriatim modules | Reviewed | Internal module guide; Seriatim, JSON output, chunk-map, and evidence-context integration contracts; all implementation/tests under `internal/modules/generic/` and `internal/modules/seriatim/`; framework evidence preparation/output paths and focused tests; maintained CLI example and production output contracts | Target-integrity check, graph architecture/symbol/call/caller review, direct import map, required module and codec tests, focused composed evidence/output tests | MOD-001, MOD-002, MOD-003 |
|
||
| Shared D&D types, codecs, and family mechanics | Revisit | D&D internal/module docs and all D&D integration contracts; root durable types; all ten codec packages and candidate JSON; shared references, ordering, citations, inputs, comparison, diagnostics, registry resolver, entity reconciliation, and assets; typed merger/evidence/default-chain/fallback/family registration; representative module asset declarations and focused tests | Target-integrity check, scoped graph architecture/search/traces, ten-family convention matrix, required codec/shared/register tests | DND-CORE-001 |
|
||
| NPC, item, and location registries | Reviewed | NPC/item/location registry integration contracts and relevant identity/evidence ADRs; extractors, candidate models and response schemas; identity, immutable registry, prompt/identity projection, location grounding, reconciliation, normalizer, validators, codecs, prompt assets, and focused tests for all three families | Target-integrity check, scoped graph architecture/complexity/search/traces, full family comparison, required extractor/identity/registry/normalizer/validator tests | DND-REG-001, DND-REG-002, DND-REG-003, DND-REG-004 |
|
||
| NPC, item, and location occurrences | Reviewed | NPC/item/location occurrence integration contracts and deterministic opaque-ID ADR; extractors, private response models/schemas, semantic registry projections and resolvers, canonicalizers, normalizers, validator chains, codecs, prompt assets, and generated/external registry handoff tests for all three families | Target-integrity check, scoped graph architecture/complexity/search/traces, full family comparison, required extractor/normalizer/validator tests, generated CLI handoff tests | DND-REG-001, DND-OCC-001, DND-OCC-002, DND-OCC-003 |
|
||
| Spells, scene chunking, and scene descriptions | Reviewed | Spell, overlay, scene-description, and accepted chunk-map integration contracts; D&D scene chunker and prompt/schema assets; framework plan canonicalization, validation, materialization, accepted chunk-map serialization/publication and production tests; scene-description extractor, normalizer, validators, immutable eligibility registry, codec, prompts, schemas, and tests; spell extractor, effective/base/overlay catalogs, NPC grounding, canonicalizer, normalizer, validators, codec, prompts, schemas, checkpoint/retry/CLI tests | Target-integrity check, scoped graph architecture/complexity/symbol/call review, full family comparison, required chunker/extractor/normalizer/validator/registry/catalog tests | PIPE-002, DND-REG-001, DND-REG-002, DND-SCENE-001, DND-SCENE-002 |
|
||
| Combat turns and enemy events | Pending | — | — | — |
|
||
| Test ownership, comments, and final synthesis | Pending | — | — | — |
|