121 KiB
Codebase Audit
Audit Metadata
- Production target:
92e89076a268089e703978fb9d7176200e93344c - Branch at target:
main - Audit date: 2026-08-08
- Go version:
go1.26.5 linux/amd64 - PromptKit version:
gitea.maximumdirect.net/eric/promptkit v0.5.0 - Knowledge-graph project:
notarius-audit-92e8907 - Knowledge-graph target: branch
main, head92e89076a268089e703978fb9d7176200e93344c - Initial worktree: Clean. There were no pre-existing production or roadmap changes to record.
The commit above is the production snapshot under audit. Later commits that change only roadmap audit documents do not change that production target.
Executive Summary
Pending final synthesis. The initial baseline is healthy. The architecture, configuration/CLI, pipeline composition, reference/handoff, runtime, state, LLM, generic/Seriatim, and shared D&D reviews have found three High findings, four Medium findings, and sixteen Low findings, with no production dependency inversion, unbounded framework worker pool, completion-order-dependent result assembly, debug-to-cache coupling, model-visible credential material in the embedded LLM assets, or domain leakage across the Seriatim and generic module boundaries.
Finding Index
Final cross-area ordering is pending synthesis.
| ID | Severity | Category | Title |
|---|---|---|---|
| ARCH-001 | Low | Documentation/Comments | Repair broken ADR cross-references |
| CFGCLI-001 | Medium | Correctness | Reject additional YAML documents |
| CFGCLI-002 | Low | Correctness | Reject a blank command-level LLM profile |
| PIPE-001 | Low | Correctness | Reject normalized module-reference collisions in the resolver |
| PIPE-002 | Low | Efficiency | Clone construction inputs once per builder boundary |
| REF-001 | Medium | Efficiency | Bound reference reads before allocating the file |
| REF-002 | Low | Efficiency | Index accepted outputs once per ordered handoff |
| REF-003 | Low | Correctness | Include canonical size in generated-reference fingerprints |
| RUN-001 | High | Correctness | Check cancellation at unguarded dispatch boundaries |
| RUN-002 | High | Correctness | Isolate typed validator values from stage output |
| RUN-003 | Medium | Correctness | Preserve warnings from the terminal rejected attempt |
| RUN-004 | Low | Documentation/Comments | Document the lane collector's liveness invariant |
| STATE-001 | Medium | Correctness | Preserve distinct identities in state paths |
| STATE-002 | Low | Efficiency | Remove redundant post-decode clones from canonical codecs |
| STATE-003 | Low | Duplication | Publish output through the confined file writer |
| LLM-001 | High | Correctness | Keep raw provider errors inside the adapter |
| LLM-002 | Low | Correctness | Recheck cancellation after scheduler admission |
| LLM-003 | Low | Correctness | Reject duplicate virtual prompt names |
| LLM-004 | Low | Duplication | Share the read-only in-memory filesystem mechanics |
| MOD-001 | Low | Simplicity | Narrow generic integer option decoding |
| MOD-002 | Low | Efficiency | Reuse compiled response schemas within a prepared validator |
| MOD-003 | Low | Simplicity | Remove unreachable JSON metadata clone helpers |
| DND-CORE-001 | Low | Simplicity | Remove the unused lossy unit-reference constructor |
Findings
Architecture And Dependency Boundaries
ARCH-001 — Repair broken ADR cross-references
- Severity: Low
- Category: Documentation/Comments
- Evidence:
docs/adr/0012-resolve-opaque-entity-identifiers-deterministically.md:15links ADR-0003 as0003-strongly-typed-stage-interfaces.md, and line 17 links ADR-0009 as0009-prefer-minimal-evidence-grounded-extraction-artifacts.md. Neither file exists. The maintained files are0003-typed-interfaces-with-two-zone-data-model.mdand0009-minimal-evidence-grounded-extraction-artifacts.md. - Impact: Readers and documentation tooling cannot follow ADR-0012 to the two architectural decisions it explicitly relies on. Runtime behavior is unaffected.
- Recommendation: Correct only the two relative link targets in ADR-0012.
- Preserve: Keep the accepted decision text and its intended references to ADR-0003 and ADR-0009 unchanged.
- Validation: Run a relative Markdown-link check across
docs/adr/and confirm both targets resolve; verify the change contains no decision-text edits. - Grouping: Independent.
Configuration And CLI Composition
CFGCLI-001 — Reject additional YAML documents
- Severity: Medium
- Category: Correctness
- Evidence:
internal/core/config/file_config.go:327–353constructs ayaml.Decoder, enablesKnownFields, and callsDecodeonly once.yaml.Decoder.Decodereads the next YAML document, so input such as a valid version 4 configuration followed by---and another configuration is accepted with the second document ignored. The strict file tests ininternal/core/config/file_config_contract_test.gocover malformed values, unknown fields, and duplicate normalized identifiers, but not an additional document. - Impact: An operator can append a syntactically valid configuration document, receive a successful validation result, and then run with only the first document. Settings in the ignored document—including operational or pipeline settings—have no effect without a diagnostic, contradicting the documented strict single-file model.
- Recommendation: After decoding
FileConfig, decode once more and requireio.EOF; reject any second document, including an empty or malformed one, with a contextual configuration error. - Preserve: Keep version gating before the full strict decode, unknown-field and duplicate-key rejection, and the existing defaults → file → environment precedence unchanged.
- Validation: Add focused parser cases for a second valid document, a
second malformed document, and ordinary trailing whitespace/comments; run
go test ./internal/core/config ./internal/cli. - Grouping: Independent.
CFGCLI-002 — Reject a blank command-level LLM profile
- Severity: Low
- Category: Correctness
- Evidence:
internal/cli/run.go:149–166registers--llm-profileas a plain string flag, while the command's presence-aware empty-value checks cover session ID, reasoning effort, output/debug directories, and recompute step but not this flag.runPipelineCommandpasses the resulting string toConfig.Resolve;internal/framework/pipeline/profile.go:1255–1277trims an empty override and treats it as absent. The run contract tests cover a valid override and an unknown non-empty profile, but not an explicitly supplied blank value. - Impact: A shell expansion such as
--llm-profile "$PROFILE"with an unset or blank value succeeds by silently using binding, pipeline, or PromptKit defaults. The run can therefore use a different model/profile than the operator explicitly intended to select. - Recommendation: Make the CLI flag presence-aware and reject an explicitly supplied empty or whitespace-only profile ID as command syntax before config loading or physical-state allocation.
- Preserve: Keep a genuinely omitted override optional, trim non-empty IDs, retain command → binding → pipeline → PromptKit precedence, and continue to apply overrides only to selected LLM-backed bindings and validators.
- Validation: Add command-contract cases for blank and whitespace-only
values that assert exit status 2 and no state allocation, plus retain the
valid and unknown-profile run cases; run
go test ./internal/cli. - Grouping: Independent.
Pipeline Resolution, Preparation, And Typed Registries
PIPE-001 — Reject normalized module-reference collisions in the resolver
- Severity: Low
- Category: Correctness
- Evidence:
internal/framework/pipeline/profile.go:1239–1252sends every module binding's reference map throughnormalizeReferenceMap. That helper, at lines 1361–1377, trims each key but overwritesrawByNormalized[trimmedKey]without checking whether another raw key already produced the same identity. A programmatic binding containing bothslotandslottherefore retains whichever raw key is visited last by Go's map iteration, then silently emits only one resolved binding. The later strict reference resolver sees only the collapsed map and cannot diagnose the collision.internal/core/config/validation.go:256–266correctly rejects this shape for validated file/config flows, but directResolvePipelinecallers do not pass through that owner and there is no focused framework regression case. - Impact: A programmatically assembled profile can resolve successfully to different external paths or generated selectors across processes from the same ambiguous input. The normal CLI configuration path is protected by upstream validation, which limits current production exposure, but the resolver's own contract is nondeterministic.
- Recommendation: Make binding reference normalization return an error for
empty or duplicate trimmed keys before constructing the normalized map, and
propagate stage/lane context through
resolveBindingcallers. Avoid relying on the config layer to make the framework resolver deterministic. - Preserve: Keep whitespace normalization, exact external-versus-generated source validation, sorted resolved bindings, local-over-pipeline precedence, and the config layer's earlier contextual diagnostics.
- Validation: Add focused
ResolvePipelinecases for whitespace-equivalent chunk, extract, merge, and normalize reference keys, including different source forms, and assert deterministic contextual rejection; rungo test ./internal/framework/pipeline ./internal/core/config. - Grouping: Independent.
PIPE-002 — Clone construction inputs once per builder boundary
- Severity: Low
- Category: Efficiency
- Evidence:
PrepareandprepareLaneclone binding option maps while forming requests (internal/framework/pipeline/prepare.go:102–104and 202–206), andprepareValidatorChaindoes the same at lines 258–263. Registry/build boundaries then clone the complete request again. Typed extractors, mergers, normalizers, and validators add another clone inside their registered erased-builder adapters (extractor_registry.go:56,merger_registry.go:68–74,normalizer_registry.go:63–69, andvalidator_registry.go:101–108), afterbuildErasedModuleorbuildPreparedValidatoralready calledcloneBuildRequest(prepare.go:272–299and 325–327). Each request clone deep-copies materialized reference content as well as options, so typed builders receive two reference copies and as many as three option copies; untyped stage and validator builders use fewer copies. - Impact: Every preparation repeats allocation and byte copying for bounded external references and nested options, with the highest cost and a different ownership path specifically for typed lanes and validators. The work is run-construction-time rather than a concurrent operation hot path, so the issue is low severity.
- Recommendation: Designate one private construction invocation as the
ownership boundary and clone the complete
BuildRequestexactly there. Store raw builders or remove the caller-side clone consistently so all stage and validator registry variants follow the same single-copy rule. - Preserve: Builders must continue to receive independently owned options, reference maps, slot slices, metadata, and content bytes; preparation must retain its own immutable resolved/reference state; nil, key/name, execution class, and exact artifact-type checks must remain contextual errors.
- Validation: Extend construction hooks to mutate nested options and
reference bytes for typed and untyped modules/validators, assert no aliasing
with resolved or sibling requests, and use allocation/byte-copy observations
or a focused benchmark to confirm a single defensive copy; run
go test ./internal/framework/contracts ./internal/framework/pipeline. - Grouping: Independent.
References And Ordered Handoffs
REF-001 — Bound reference reads before allocating the file
- Severity: Medium
- Category: Efficiency
- Evidence:
internal/framework/pipeline/references.go:88–164implements the external-reference materialization boundary. At lines 128–146,materializeReferenceTargetcallsos.ReadFile(path)before comparing the resulting allocation withReferenceSlot.MaxBytes. The focusedTestMaterializeReferencesEnforcesMaxBytescase uses a nine-byte file and verifies the post-read diagnostic, but does not prove that reads are bounded by the declared three-byte limit. - Impact: A mistakenly selected very large file, growing file, device, or named pipe can consume memory far beyond the slot's advertised bound before the framework rejects it. CLI reference overrides expose the same path, so a local operator error can terminate the process instead of producing the intended bounded validation failure.
- Recommendation: Open the path and read through a limit of
MaxBytes + 1when a positive maximum is declared, rejecting an extra byte before retaining or cloning content. A regular-file size precheck may improve diagnostics, but the bounded reader must remain authoritative for changing or non-regular inputs. Preserve the existing unbounded behavior only for slots that explicitly declare no maximum. - Preserve: Keep config-relative versus working-directory-relative path resolution, UTF-8 and media-type validation, empty-file warnings, canonical digest/size/origin metadata, contextual errors without content, and owned reference bytes.
- Validation: Add a reader or file fixture that proves no more than
MaxBytes + 1bytes are consumed, including a non-regular or growing-input case, while retaining the current UTF-8, media, empty, and ordinary oversize diagnostics; rungo test ./internal/framework/pipeline ./internal/cli. - Grouping: Independent.
REF-002 — Index accepted outputs once per ordered handoff
- Severity: Low
- Category: Efficiency
- Evidence:
buildStepReferenceSetswalks every generated binding in the receiving step (internal/framework/pipeline/handoff.go:37–89). For each previously unseen producer,generatedReferenceItemallocates amatchesslice and scans the complete cumulativeoutputsslice to find that step/lane (handoff.go:104–133). Its cache avoids rescanning when several targets fan out from the same producer, but a step consumingPdistinct producers fromOearlier outputs still performsP * Ocomparisons and up toPtemporary allocations. - Impact: Ordered pipelines with many distinct generated dependencies pay quadratic handoff preparation work before the consumer step can start. The current production profiles are small and the scan is outside the lane worker hot path, which limits present impact.
- Recommendation: Build one map from normalized
(step ID, lane ID)to an explicit zero/one/many accepted-output state at the start ofbuildStepReferenceSets, then letgeneratedReferenceItemperform a direct lookup. Keep ambiguity as data in the index so duplicate producer outputs are still rejected rather than overwritten. - Preserve: Retain exact accepted-output cardinality, codec decode/re-encode canonicalization, producer lookup, complete schema/media validation, per-target byte ownership, deterministic contextual errors, and the rule that no consumer lane starts after a failed handoff.
- Validation: Add a many-producer/fanout case that retains missing and
duplicate rejection, then use a focused benchmark or comparison counter to
demonstrate one output-index pass plus direct producer lookups; run
go test ./internal/framework/pipeline. - Grouping: Independent; do not combine with PIPE-002, which concerns construction-request copying rather than runtime producer lookup.
REF-003 — Include canonical size in generated-reference fingerprints
- Severity: Low
- Category: Correctness
- Evidence: The generated
ReferenceItemrecords canonical content length inSizeBytes(internal/framework/pipeline/handoff.go:154–177), and the manifest provenance also retains that value at lines 210–232. However,generatedReferenceFingerprintIdentityandgeneratedReferenceDependencies(handoff.go:234–287) hash producer, kind, complete schema identity, media type, and content digest without the canonical size. This differs from the documented resume contract indocs/internal/state.md:49–54. The focused fingerprint test changes only canonical content and does not assert size participation. - Impact: Consumer checkpoint identity does not cover one field that the handoff and manifest declare part of canonical reference identity. Current construction derives size directly from canonical bytes, so a practical stale reuse also requires malformed internal metadata, a digest collision, or a future producer-path change; the immediate risk is therefore low.
- Recommendation: Add
SizeBytesto the private fingerprint identity and populate it from the canonical generated item before JSON hashing. Keep the existing normalized fingerprint name and all current identity fields. - Preserve: Do not weaken the content digest, producer provenance, artifact kind, complete schema digest/fields, media type, or canonical codec checks. Continue to keep content bytes out of checkpoints, manifests, debug summaries, and errors.
- Validation: Add a direct dependency-fingerprint case that holds the
other identity fields constant while changing size metadata, retain the
canonical-content sensitivity case, and run
go test ./internal/framework/pipeline ./internal/modules/integration/.... - Grouping: Independent.
Execution, Validation, Retry, And Concurrency
RUN-001 — Check cancellation at unguarded dispatch boundaries
- Severity: High
- Category: Correctness
- Evidence:
Runner.Runinvokes the input adapter atinternal/framework/pipeline/runner.go:159–177without checkingctx.Err()first; the first framework-owned cancellation check on that path is insiderunWithRetryat lines 389–418, after source work and its checkpoint/debug side effects. At the other end of the run, ordered lane work completes at lines 278–282, then evidence/debug assembly andencoder.Encoderun at lines 297–346 without another cancellation check. The worker and retry paths do check cancellation, butTestRunnerReturnsParentCancellationAndStopsQueuedExtractscovers only cancellation while two extract calls are active (runner_concurrency_test.go:559–585). A context-ignoring input or output module can therefore be called with an already-cancelled context; in the output case it can return files and make the cancelled run report success. - Impact: Work can start after cancellation, and a cancellation arriving after lane completion or during output debug assembly can still publish a successful logical output. This violates the documented guarantee that parent cancellation prevents queued work and output encoding, and it leaves correctness dependent on every module independently honoring an already cancelled context.
- Recommendation: Add framework-owned cancellation gates at run entry and
immediately before every module dispatch not already protected by
runWithRetryor a lane worker, especially input parsing and output encoding. Recheck after intervening collaborator/debug/evidence work so a cancellation cannot slip between the gate and the operation, and after an unguarded module returns so a module that did not observe mid-call cancellation cannot publish success. Return the parent context error through the existing failed-output path. - Preserve: Continue to pass the caller context into active operations, wait for all started lane work, suppress output files on framework failure, select an actual framework error deterministically when the parent remains live, and prefer the parent cancellation when it is set.
- Validation: Add a pre-cancelled run whose input adapter ignores context
and must not be called, plus a run whose debug recorder cancels immediately
before output dispatch while a context-ignoring encoder records calls, and an
encoder that cancels mid-call but returns files. Assert
context.Canceled, no invocation after pre-dispatch cancellation, and no output files in either output case; retain the active-extract cancellation case and run the package under-race. - Grouping: Independent.
RUN-002 — Isolate typed validator values from stage output
- Severity: High
- Category: Correctness
- Evidence: Extract, merge, and normalize serialize a canonical candidate
before validation, but pass the operation's original typed value alongside
it; representative extract code is
internal/framework/pipeline/runner_concurrent.go:424–439, and merge and normalize do the same inrunner_typed.go:264–277and 359–392.validateTypedArtifactclones source input, references, metadata, and chunks, butrequestTarget := targetleavestarget.valueshallow-copied (runner_typed.go:458–499). The registered typed adapter then exact-casts that value and passes it directly to the validator (validator_registry.go:109–119). Struct values containing slices, maps, or pointers therefore retain aliases to the stage output. A validator can mutate what later validators and final checkpoint serialization observe, while serialized validators and attempt debug continue to describe the pre-mutation canonical candidate. Candidate-encoding tests verify encode counts and rejection-before-checkpoint behavior, but no focused test mutates a typed validator request. - Impact: A buggy validator can corrupt an accepted output, cause validators in one chain to inspect different values, or make persisted output differ from the candidate that was serialized for validation and debug. This breaks the documented immutable whole-output validation contract at all three typed stages.
- Recommendation: Treat the already serialized canonical candidate as the clone boundary. Decode a fresh exact typed value through the active codec for each typed validator, or provide an equivalent codec-backed deep clone, and never expose the stage operation's retained value. Keep serialized validators on independent schema/content copies of the same candidate.
- Preserve: Retain exact Go-type checks, one canonical candidate encode per attempt, validator declaration order, isolated request metadata/references, serialized-validator byte ownership, contextual errors, and final encoding only after the entire chain approves.
- Validation: Use an artifact with both slice and map fields. Have the first
typed validator mutate both, then assert that the next typed validator, a
serialized validator, attempt debug, and accepted extract/merge/normalize
output all observe the original canonical value. Run the focused package
tests under
-raceas well as normally. - Grouping: Independent.
RUN-003 — Preserve warnings from the terminal rejected attempt
- Severity: Medium
- Category: Correctness
- Evidence: The runtime contract says warnings from the final accepted or
rejected attempt are preserved (
docs/internal/pipeline.md:122–126). Each retry closure assembles per-attempt operation and validator warnings, but stores them only on acceptance. Generated chunk-plan rejection returns atinternal/framework/pipeline/runner_chunk_plan.go:142–150before assigningresult.warningsat line 154, andRunner.Runpromotes chunk warnings only for acceptance or a cache hit (runner.go:228–233). Extract follows the same pattern atrunner_concurrent.go:425–457;finalizeLaneExtractskips directly from a rejected result to the next chunk at lines 469–476. Merge and normalize collectattemptWarningsbut return terminal rejections before appending them toRunOutput(runner_typed.go:264–301and 359–415).runWithRetryreturns only the last rejection, not its warnings (runner.go:389–428). Existing retry-warning tests prove that discarded attempts are not promoted, and final-rejection debug tests prove rejection recording, but neither asserts terminal rejection warnings. - Impact: Module warnings and warnings from validators that approved before the rejecting validator disappear exactly when the final candidate is rejected. Operators receive the rejection but lose diagnostics produced by that terminal attempt, contrary to the documented output contract.
- Recommendation: Carry the warnings associated with the last rejection through the retry result, then promote only that terminal attempt's warnings at chunk, extract, merge, and normalize rejection handling. Keep warnings from earlier failed or rejected attempts confined to attempt debug. Coordinate any persistence-format addition with the later checkpoint/state audit rather than silently changing state ownership here.
- Preserve: Do not promote warnings from attempts superseded by a later retry, do not turn rejection into a framework error, keep deterministic lane/chunk warning order, and retain accepted and reused checkpoint warning behavior.
- Validation: Add table-driven chunk, extract, merge, and normalize cases
with distinct first-attempt and final-rejection warning scopes. Assert that
only the final scopes reach
RunOutput, rejection attempt counts remain correct, intermediate warnings remain in attempt debug, and shuffled/race runs preserve order. - Grouping: Independent.
RUN-004 — Document the lane collector's liveness invariant
- Severity: Low
- Category: Documentation/Comments
- Evidence:
laneEngine.collectcoordinates a dynamically enabled continuation send, extract-result closure, cancellation, and thepending/launched/completedcounters ininternal/framework/pipeline/runner_concurrent.go:256–281, with state changes split acrosshandleExtractResultandhandleCompletionat lines 283–313. The loop has no invariant-level comment. Its non-obvious purpose is to discard only unlaunched continuations after cancellation while continuing to drain extract results and exactly one completion from every launched continuation before the continuation channel is closed and workers are awaited. - Impact: Current tests and the race run support the implementation, but a future change to channel buffering, cancellation cleanup, or a counter can introduce a deadlock or early close without the ownership/liveness rule being visible at the maintenance point.
- Recommendation: Add one short comment immediately above
laneEngine.collectstating that it is the sole owner of pending/launch/ completion accounting and must drain closed extract results plus every launched continuation completion even after cancellation. Do not narrate theselectcases. - Preserve: Keep chunk-first/lane-second extract dispatch, bounded extract and continuation pools, overlap between completed lanes and remaining extracts, cancellation of undispatched work, and the final worker wait.
- Validation: Review the comment against the collector termination predicate and retain the existing cancellation, reverse-completion, continuation-overlap, worker-bound, race, and shuffled tests.
- Grouping: Independent.
State, Checkpoints, Debugging, And File Safety
STATE-001 — Preserve distinct identities in state paths
- Severity: Medium
- Category: Correctness
- Evidence: The resolver accepts any non-empty trimmed ordered-step and
lane IDs (
internal/framework/pipeline/profile.go:341–351and 1382–1392). Checkpoint path construction then sends both IDs throughcheckpointPathComponent(internal/framework/checkpoint/recorder.go:476–508), which maps.,.., and every value containing..to the single component_. The trace side duplicates that encoder indebugPathComponent(internal/framework/pipeline/debug.go:37–62), andcleanDebugPathat lines 364–374 additionally appliespath.Cleanbefore encoding. Thus valid distinct identities such as.and_, ora..band_, select the same checkpoint component and can also collapse to the same debug path. Existing filesystem tests cover traversal and symlink rejection, but no focused test asserts that accepted identities map injectively. - Impact: Two valid lanes or steps in one resolved pipeline can overwrite each other's checkpoint manifests and payloads. Exact manifest validation prevents a mismatched artifact from being silently reused, but ordinary resume repeatedly invalidates and re-executes the colliding work, and a required selective-recompute predecessor can become unavailable after its sibling overwrites it. Requested debug records can likewise overwrite or be attributed to the wrong identity. Atomic rename does not protect against a logical-name collision.
- Recommendation: Introduce one shared, injective path-component encoding
for accepted application identities. Preserve already-safe components, but
encode reserved dot components and every unsafe rune rather than replacing a
whole value with
_; do not run identity-bearing debug paths through a lossy clean operation first. Keep path joining and confinement separate from identity encoding. - Preserve: Retain trimmed non-empty identity validation, human-readable ordinary IDs, exact step/lane checks inside checkpoint manifests, narrow relative state paths, symlink rejection, and recoverable cold execution for incompatible reconstructible cache entries.
- Validation: Add table and uniqueness cases covering
.,..,_, embedded.., separators, tildes, and Unicode, then run a two-identity filesystem checkpoint/resume and debug trace case that proves distinct files and successful accepted-normalize hydration. Include selective recomputation so a required predecessor remains reusable. - Grouping: Independent.
STATE-002 — Remove redundant post-decode clones from canonical codecs
- Severity: Low
- Category: Efficiency
- Evidence:
chunkmap.Codec.Decodeschema-validates and decodes JSON into a fresh value, canonicalizes that value, then deep-clones it again before returning (internal/framework/chunkmap/codec.go:149–171).evidencecontext.Codec.Decodefollows the same sequence (internal/framework/evidencecontext/codec.go:79–101), while itscanonicalizealready deep-clones the decoded document at lines 169–203; the finalcloneat lines 299–320 therefore makes a second complete copy of context/reference slices, source-unit structs, and nested metadata. Unlike an encode caller, the JSON decoder retains no caller-owned object graph that must be protected from canonicalization. - Impact: Hydrating or consuming canonical artifacts performs avoidable slice and nested-metadata allocations. The evidence-context path can clone a large source-derived object graph twice after JSON decoding, increasing peak allocation and latency without adding an ownership boundary.
- Recommendation: Separate canonicalization of an already-owned decoded value from the defensive-copy entry point used by encoders and external callers. Return the canonical decoded value directly once validation and normalization succeed; retain exactly one clone wherever caller-owned input could otherwise be mutated.
- Preserve: Keep schema validation, unknown-field and trailing-value rejection, digest reconstruction, annotation/context normalization, encode-time caller isolation, and a fully independently owned decode result.
- Validation: Retain and extend ownership tests that mutate both the input value after encode and the returned value after decode, then use focused allocation assertions or benchmarks for a nested chunk map and evidence context to confirm that decode no longer performs the redundant deep copy.
- Grouping: Independent.
STATE-003 — Publish output through the confined file writer
- Severity: Low
- Category: Duplication
- Evidence: The CLI's private
writeFileAtomic(internal/cli/run.go:826–856) independently implements the same create-temp, write, chmod, close, rename, and cleanup sequence asinternal/core/fileio.writeAtomic(internal/core/fileio/fileio.go:105–133).writeOutputFilescalls the CLI copy after its own path checks and directory creation (internal/cli/run.go:754–787), whilefileio.WriteBytesadditionally centralizes relative-path confinement and symlink-component rejection. Debug summaries, traces, and checkpoints already use that core primitive; durable output is the divergent state writer. - Impact: Atomic-publication fixes must be made and tested in two places, and the copies have already drifted in their confinement checks. The fresh, exclusively created run directory limits current output exposure, so this is a maintenance and defense-in-depth issue rather than an observed escape.
- Recommendation: After prevalidating all logical output names and
exclusively creating the run directory, publish each file through
fileio.WriteBytes(or a narrowly exported equivalent) using the run directory as root. Remove the CLI atomic-write copy rather than introducing another generic filesystem abstraction. - Preserve: Validate every logical name before allocating state, refuse an existing run directory unchanged, retain a newly created partial bundle on a later write failure, keep output directory/file modes at 0755/0644, and preserve logical-name context in errors.
- Validation: Retain the existing output ordering, collision, permission, and partial-bundle cases; add a symlink-component case at the shared writer boundary and verify no temporary file remains after write, chmod, close, or rename failure.
- Grouping: Independent.
LLM Runtime, Prompt Filesystems, And Assets
LLM-001 — Keep raw provider errors inside the adapter
- Severity: High
- Category: Correctness
- Evidence: Preparation and ordinary execution failures wrap
redactPromptKitErrorwith%watinternal/framework/llm/promptkit_client.go:140–145and 149–172. The returnedredactedProviderErrorreplaces bearer-token text only in itsErrormethod, then exposes the original upstream error unchanged throughUnwrapat lines 424–442. The outer error therefore prints a redacted diagnostic, buterrors.Unwrap,errors.Is, orerrors.Ascan recover the raw PromptKit/provider error, including its original text and concrete type. This contradicts the adapter contract indocs/internal/llm.md:177–190and 231–236, which requires provider-specific errors and credentials to stay behind the provider-neutral boundary. Capacity exhaustion avoids this particular chain by mapping the PromptKit sentinel explicitly and formatting the sanitized diagnostic with%v. - Impact: Any framework collaborator, logger, test helper, or future error serializer that walks the standard error chain can disclose a bearer credential that normal string formatting appeared to redact. It can also couple application code to PromptKit/provider error types despite the documented neutral transport boundary.
- Recommendation: Make sanitized upstream diagnostics opaque: do not
implement
Unwrapand do not wrap a provider error directly after crossing the adapter boundary. Classify context cancellation, capacity, invalid structured output, and any other intentionally supported neutral categories before sanitization, then expose only the corresponding contracts sentinel plus credential-redacted text. - Preserve: Keep caller-context precedence, prompt context, the provider-neutral capacity and invalid-output sentinels, full internal PromptKit error inspection before adaptation, and useful redacted operational diagnostics.
- Validation: Return a typed upstream error containing a bearer secret from
both preparation and generation. Assert that every reachable error-chain
level and formatted form omits the secret,
errors.Ascannot recover the upstream type, anderrors.Isstill recognizes only the documented context, capacity, and invalid-output categories. - Grouping: Independent.
LLM-002 — Recheck cancellation after scheduler admission
- Severity: Low
- Category: Correctness
- Evidence: A queued scheduler waiter selects between its closed
readychannel andctx.Done()atinternal/framework/llm/scheduler.go:50–73.grantQueuedLockedmarks the waiter granted and closesreadyat lines 103–111. If cancellation and that grant become observable together, Go may choose the ready case, soAcquirereturns a permit even though the context is already canceled.Scheduler.Runthen invokesfn(ctx)immediately at lines 77–86 without another cancellation check. The focused queued- cancellation test cancels before releasing the existing permit and therefore does not exercise the simultaneous grant/cancel branch. - Impact: A scheduler user that does not independently reject an already- canceled context can begin backend work after the caller canceled while it was queued. The production PromptKit path receives the canceled context and ordinarily stops downstream work, and the deferred release prevents a permit leak, which limits current severity; the scheduler's own cancellation contract is nevertheless incomplete.
- Recommendation: After admission and before dispatch, recheck
ctx.Err()while retaining the deferred permit release. KeepAcquire's locked grant-versus-remove accounting as the owner of queue state; the extra gate should only prevent the admitted callback from starting. - Preserve: Retain one process-wide FIFO queue, the fixed concurrency bound, idempotent release functions, cancellation removal for still-queued waiters, and permit transfer/release after all errors.
- Validation: Add a focused grant/cancel race case with a callback that
records invocation and intentionally ignores its context. Prove that an
already-canceled admitted request returns
context.Canceled, never invokes the callback, releases its permit, and allows the next FIFO waiter to run; retain the concurrency, cancellation, and idempotent-release tests under-race. - Grouping: Independent; this is the provider scheduler boundary rather than RUN-001's pipeline module-dispatch boundary.
LLM-003 — Reject duplicate virtual prompt names
- Severity: Low
- Category: Correctness
- Evidence:
promptfs.ModulePromptFSvalidates and reads every declared module file, then assigns its bytes directly into a map atinternal/framework/promptfs/prompt_fs.go:40–58; shared files use the same direct assignment at lines 60–80. Two names that are equal after trimming, leading-slash removal, and cleaning therefore select the same virtual path, and the later declaration silently replaces the earlier bytes. The asset registry rejects duplicate flattened roots, but the module manifest boundary has no equivalent check or focused duplicate test. All fourteen current D&D manifests declare unique normalized names, so no maintained asset is presently shadowed. - Impact: A future or programmatically supplied prompt manifest can hash, register, and execute successfully while using different prompt or shared fragment bytes than one of its declarations implies. List order silently decides the winner instead of producing the contextual asset-construction error expected at startup.
- Recommendation: Track each cleaned virtual destination while assembling
ModulePromptFSand reject a second declaration before overwriting it. Report whether the collision is module-owned or shared and include only the safe virtual name, not file content. - Preserve: Keep module and
sharedassetsnamespaces distinct, reject nested virtual names, read only explicitly declared files, return owned bytes, and retain manifest order for fingerprinting and diagnostics where it is semantically relevant. - Validation: Add exact and normalization-equivalent duplicate cases for
module files and for shared files backed by different filesystems. Assert
deterministic contextual rejection while distinct module/shared basenames
remain valid; run the package under
-race. - Grouping: Independent.
LLM-004 — Share the read-only in-memory filesystem mechanics
- Severity: Low
- Category: Duplication
- Evidence:
internal/framework/llm/asset_registry.go:279–430implements a complete map-backed read-only filesystem—path cleaning,Open,ReadFile,ReadDir, directory discovery, sorted entries, cursor reads, file metadata, and modes.internal/framework/promptfs/prompt_fs.go:84–240independently repeats the same mechanics with renamed types. The copies have already drifted: the asset filesystem represents an empty root as an existing empty directory while the prompt filesystem reports it missing, and directoryReadreturns a custom error in one implementation versusio.EOFin the other. - Impact: Filesystem-contract fixes and edge-case tests must be duplicated, while callers receive subtly different behavior from two byte maps assembled for the same PromptKit engine. The maintained registries are non-empty, so the observed drift is a maintenance risk rather than a current prompt load failure.
- Recommendation: Extract one narrow framework-owned read-only byte-map
filesystem with standard
io/fsbehavior and independently owned file handles. Keep asset flattening, duplicate/root validation, schema selection, and module/shared manifest construction in their existing domain owners. - Preserve: Retain valid-path enforcement, deterministic directory order,
0444/0555 metadata, immutable source bytes, independent reader offsets,
contextual domain errors before filesystem construction, and the asset
registry's direct owned
ReadFileresult. - Validation: Run
fstest.TestFSover empty, single-file, and nested trees; test root and directoryOpen/ReadDir/Readbehavior, file-handle independence, sorted entries, missing/invalid paths, modes, and mutation isolation. Retain both asset-registry andModulePromptFSintegration tests. - Grouping: Independent; implement after or alongside LLM-003 without moving manifest collision policy into the generic filesystem.
Generic And Seriatim Modules
MOD-001 — Narrow generic integer option decoding
- Severity: Low
- Category: Simplicity
- Evidence:
internal/modules/generic/chunk/units/chunker.go:138–220routes onlymax_unitsandoverlap_unitsthrough a 61-line numeric conversion family that accepts every signed and unsigned Go integer type, integralfloat64values, andjson.Number. Its only production callers are the two local positive/non-negative wrappers. In contrast, the other integer option in this family, JSON output'swindow_units, accepts the configuration-nativeintdirectly (internal/modules/generic/output/json/encoder.go:178–188). The focused chunker tests reject a fractional float and malformedjson.Number, but do not establish a caller or contract for any accepted non-intrepresentation. - Impact: The generic chunker's private option surface is substantially
larger than its two-option configuration contract and silently treats a
floating-point value such as
2.0as an integer while the adjacent generic output decoder rejects that representation. Additional numeric branches and architecture-dependent range logic can drift without providing behavior used by the maintained configuration path. - Recommendation: Define the supported module-option scalar type at the
configuration boundary and reduce these two options to that representation,
currently
int, with their existing positive/non-negative and overlap checks. If a second real option source requiresjson.Numberor another representation, normalize it once at that source rather than retaining a speculative all-Go-numeric converter in this leaf module. - Preserve: Keep defaults, strict unknown-option rejection, precise option
names in diagnostics,
overlap_units < max_units, platform-safe values, and validation/build decoding parity. - Validation: Add focused rejection cases for integral floats and other
unsupported numeric representations, retain boundary/overlap tests, and run
go test ./internal/modules/generic/chunk/units ./internal/core/config. - Grouping: Independent.
MOD-002 — Reuse compiled response schemas within a prepared validator
- Severity: Low
- Category: Efficiency
- Evidence: Every call to
internal/modules/generic/validate/valid_json_schema.Validator.Validatedelegates tovalidate, which parses the candidate, reparses the complete response schema, creates a newjsonschema.Compiler, adds the schema resource, and compiles it before validation (validator.go:29–56). The framework constructs one validator instance per prepared chain position (internal/framework/pipeline/prepare.go:258–299) and invokes that same prepared instance for each chunk/artifact candidate and retry (runner.go:430–489andrunner_typed.go:458–542). Within a chain position the active artifact schema is stable, while candidate content changes. - Impact: A run with many chunks or retries repeatedly pays schema parse, compiler construction, resource loading, and compilation for identical schema bytes. The cost is deterministic local CPU/allocation work on every validation attempt; current LLM-backed workloads limit its overall severity.
- Recommendation: Cache the compiled schema on the prepared validator, keyed by the complete schema identity including its JSON bytes or digest, and make concurrent first use safe because extract validators can run in parallel. Continue parsing each candidate independently and preserve support for a validator instance receiving a different schema rather than assuming one globally.
- Preserve: Retain operational errors for missing/malformed schemas,
ordinary
invalid_jsonandjson_schema_invalidrejections, support for both chunk and artifact targets, exact candidate ownership, and standalone use without a precedingvalid_jsonvalidator. - Validation: Add repeated-schema and changed-schema cases, exercise one
validator concurrently, verify malformed-schema errors are stable, and use a
focused benchmark or compiler hook to demonstrate one compilation per
distinct schema; run
go test -race ./internal/modules/generic/validate/valid_json_schema ./internal/framework/pipeline. - Grouping: Independent.
MOD-003 — Remove unreachable JSON metadata clone helpers
- Severity: Low
- Category: Simplicity
- Evidence:
cloneMetadataandcloneJSONMetadataValueatinternal/modules/generic/output/json/encoder.go:497–527form a mutually recursive deep-clone implementation. Knowledge-graph inbound-call inspection and a scoped source search find only the calls between those two functions; no encoder path, registration path, or test reaches either helper. Actual output ownership is handled bycloneNormalizeOutputs, serialized-artifact cloning, and immediate JSON serialization. - Impact: Thirty-one lines of recursive, type-specific ownership code imply a boundary that does not exist, add an untested maintenance surface, and can mislead later changes into choosing the dead helper instead of the encoder's active clone/serialization paths.
- Recommendation: Delete the two unreachable helpers. Do not replace them with a shared abstraction unless a live metadata owner later demonstrates a caller.
- Preserve: Keep request non-mutation, owned output bytes, exact preservation
of chunk-map annotation numbers, cloned normalized artifacts, and caller
mutation isolation after
Encodereturns. - Validation: Retain the output encoder's request-mutation and annotation
number tests, run
go test ./internal/modules/generic/output/json, and confirm no production caller is removed with graph and compiler checks. - Grouping: Independent.
Shared D&D Types, Codecs, And Family Mechanics
DND-CORE-001 — Remove the unused lossy unit-reference constructor
- Severity: Low
- Category: Simplicity
- Evidence:
internal/modules/dnd/shared/unit_refs.go:24–27exportsUnitRefFromString, discards the error fromparseUnitRefNumber, and returns a zero-valuedUnitReffor every blank, whitespace-padded, non-integer, or non-positive value. Knowledge-graph inbound-call tracing and a scoped source search found no production or test caller. The live external boundary isUnitRef.UnmarshalJSONat lines 47–71, which calls the same parser and correctly propagates its error;UnitRefFromIntremains used by that decoder and focused tests. - Impact: The exported helper advertises a supported string-construction
path whose only distinguishing behavior is to erase invalid-input context.
A future caller could turn malformed model output or test data into a later,
less specific
must be positiveerror, while the dead API adds maintenance surface beside the authoritative strict decoder. - Recommendation: Delete
UnitRefFromString. If a non-JSON string boundary later needs construction, expose a parsing function that returns(UnitRef, error)and reuseparseUnitRefNumberwithout discarding errors. - Preserve: Keep string-or-integer JSON compatibility, exact positive source
unit IDs rather than ordinal fallback, original string-versus-number JSON
rendering, and the contextual checks in
ResolveUnitID. - Validation: Confirm graph and compiler checks find no removed caller;
retain the string/integer/malformed JSON and exact-ID tests in
internal/modules/dnd/shared/unit_refs_test.go; rungo test ./internal/modules/dnd/shared/.... - Grouping: Independent.
D&D Convention Matrix
The matrix records the current production convention before the lane-specific
reviews. E, M, and N mean extract, merge, and normalize. Every extractor
is LLM-backed, every merger is the deterministic typed appendorder merger,
and every family also registers the typed deterministic noop normalizer as an
explicit alternate. B is the ordered default chain
valid_json → shape → source_refs → valid_json_schema → source_relatedness;
notations such as B + identity name the check inserted after shape and
before source_refs. C means the optional campaign-context slots glossary,
party, players, and deprecated roster. Every durable codec is
family-owned, uses media type application/json, embeds its own v1 durable
schema, publishes count metadata, and has a registered direct-source-reference
evidence projector.
| Family | Durable kind / Go type | Modules and execution class | Default validator chains (E / N) | Reference dependencies (E / N) | Codec, prompt, and schema ownership | Documented exception and later confirmation |
|---|---|---|---|---|---|---|
| Spells | dnd/spell-list / dnd.SpellList |
dnd/spells (LLM) → typed appendorder (deterministic) → dnd/spells (deterministic) |
B + catalog / B + catalog |
C plus optional spell_catalog and npc_registry / optional spell_catalog |
codec/spells; extractor owns its prompt and private response schema; normalizer has no prompt |
Catalog overlay and NPC caster grounding are optional and never evidence; confirm spell/catalog and scene-family behavior in the spells/scenes review. |
| NPC registry | dnd/npc-registry / dnd.NPCRegistry |
dnd/npc-registry (LLM) → typed appendorder (deterministic) → dnd/npc-registry (LLM) |
B / B + identity |
C / none |
codec/npcregistry; extractor and normalizer own prompts; extractor owns its response schema, while normalize uses the shared private entity-reconciliation schema |
Normalization is proposal-only LLM reconciliation; confirm identity, merge safety, and same-name policy in the registry review. |
| Combat turns | dnd/combat-turn-list / dnd.CombatTurnList |
dnd/combat-turns (LLM) → typed appendorder (deterministic) → dnd/combat-turns (deterministic) |
B / B + invariants |
C, optional npc_registry, required scene_descriptions / optional npc_registry |
codec/combatturns; extractor owns its prompt and private response schema; normalizer has no prompt |
Scene descriptions gate LLM execution and NPC grounding is not evidence; confirm gate and empty-result semantics in the combat/enemy review. |
| Item occurrences | dnd/item-occurrence-list / dnd.ItemOccurrenceList |
dnd/item-occurrences (LLM) → typed appendorder (deterministic) → dnd/item-occurrences (deterministic) |
B + registry / B + registry + invariants |
C plus required item_registry / required item_registry |
codec/itemoccurrences; extractor owns its prompt and private response schema; normalizer has no prompt |
Registry identity grounds current-transcript facts but never supplies evidence; confirm quantities, holders, and registry projection in the occurrence review. |
| Item registry | dnd/item-registry / dnd.ItemRegistry |
dnd/item-registry (LLM) → typed appendorder (deterministic) → dnd/item-registry (LLM) |
B / B + identity |
C / none |
codec/itemregistry; extractor and normalizer own prompts; extractor owns its response schema, while normalize uses the shared private entity-reconciliation schema |
Normalization is proposal-only and preserves denominations/types rather than inventing instances; confirm identity and merge policy in the registry review. |
| NPC occurrences | dnd/npc-occurrence-list / dnd.NPCOccurrenceList |
dnd/npc-occurrences (LLM) → typed appendorder (deterministic) → dnd/npc-occurrences (deterministic) |
B + registry / B + registry + invariants |
C plus required npc_registry / C plus required npc_registry |
codec/npcoccurrences; extractor owns its prompt and private response schema; normalizer has no prompt |
Registry provenance cannot become occurrence evidence and mentioned remains a factual category; confirm category/identity handling in the occurrence review. |
| Scene descriptions | dnd/scene-description-list / dnd.SceneDescriptionList |
dnd/scene-descriptions (LLM) → typed appendorder (deterministic) → dnd/scene-descriptions (deterministic) |
B / B + invariants |
optional glossary, party, and players / none |
codec/scenedescriptions; extractor owns its prompt and private response schema; normalizer has no prompt |
Each record has one source_ref rather than a slice; the separate dnd/scenes LLM chunker owns the full-transcript scene prompt. Confirm scene IDs, ordering, and classification in the spells/scenes review. |
| Enemy events | dnd/enemy-event-list / dnd.EnemyEventList |
dnd/enemy-events (LLM) → typed appendorder (deterministic) → dnd/enemy-events (deterministic) |
B + engagements / B + invariants |
C plus required npc_registry, scene_descriptions, combat_turns, and npc_occurrences / required npc_registry |
codec/enemyevents; extractor owns its prompt and private response schema; normalizer has no prompt |
Four generated artifacts ground extraction without becoming event evidence; durable decode separately proves required JSON-field presence. Confirm combat gating, engagement uniqueness, and observation ordering in the combat/enemy review. |
| Location registry | dnd/location-registry / dnd.LocationRegistry |
dnd/location-registry (LLM) → typed appendorder (deterministic) → dnd/location-registry (LLM) |
B / B + identity |
C / none |
codec/locationregistry; extractor and normalizer own prompts; extractor owns its response schema, while normalize uses the shared private entity-reconciliation schema |
Normalization is proposal-only and same-name locations require contextual evidence; confirm identity and merge policy in the registry review. |
| Location occurrences | dnd/location-occurrence-list / dnd.LocationOccurrenceList |
dnd/location-occurrences (LLM) → typed appendorder (deterministic) → dnd/location-occurrences (deterministic) |
B + registry / B + registry + invariants |
C plus required location_registry / required location_registry |
codec/locationoccurrences; extractor owns its prompt and private response schema; normalizer has no prompt |
Registry grounding cannot become evidence and speculation remains distinct from unsupported inference; confirm category and identity handling in the occurrence review. |
Shared convention review classified the codec surface as safe typed adapters,
not a missing artifact-codec framework. candidatejson is the natural owner for
single-value JSON encoding and strict decoding (including unknown-field and
trailing-value rejection), while each codec retains exact Go type, kind,
metadata, embedded durable schema, and artifact-specific required-field or
semantic validation. Candidate encode/decode clones are limited to the two
families with retained pointer/slice ownership (ItemOccurrenceList and
EnemyEventList); schema bytes are owned because every embed.FS.ReadFile
call returns a fresh byte slice. The generic candidate helper intentionally
does not absorb family validation or durable-schema policy.
Source-reference responsibilities are likewise separated by semantics:
SourceRefOrder owns document-position ordering, exact equality-based
deduplication, invalid-reference preservation for diagnostics, and nil versus
owned-empty behavior; UnitRef.UnmarshalJSON owns strict model-facing string
or integer parsing; CitationResolver owns current-document range expansion;
family validators own admissibility and evidence rules; and evidence projectors
copy only direct artifact references. Extractor-local response adapters remain
typed because their private response DTOs differ; replacing them with
reflection or callbacks would hide field ownership without consolidating
policy. DND-CORE-001 records the one dead constructor outside these live paths.
Registration is explicit and complete for all ten codecs, extractors, mergers, normalizers, noop alternates, evidence projectors, validator capabilities, and default chains, plus the separate scene chunker. Prompt registration covers every LLM extractor, the scene chunker, the three LLM registry normalizers, the shared reconciliation schema, and the fallback profile. The registrar validates every registry and the asset registry before mutation, registers in a fixed order, and wraps failures with the exact D&D component name; a later failure can leave the caller-supplied registries partially populated, but production composition treats registration failure as terminal and no rollback contract is documented.
The shared registry resolver correctly distinguishes absent construction views, empty generated-reference placeholders, operation-time overrides, raw-content cache identity, and semantic identity while retaining owned loaded values. The entity-reconciliation helper keeps durable IDs out of prompts, rejects invalid or colliding selectors and overlapping groups, bounds transcript context, and returns owned safe groups. Shared comparison policy is versioned, and shared diagnostics bound displayed issues and warning counts. Domain-specific identity, merge, category, and eligibility decisions remain assigned to the subsequent registry, occurrence, scene/spell, and combat/enemy reviews.
Finally, repeated ManifestMetadata and CheckpointFingerprints methods remain
module-local because their exact policy names, prompt/schema hashes, reference
projection digests, and ordering are checkpoint semantics. Small Register,
New, and DecodeOptions methods preserve typed registry construction,
module-local error context, dependency validation, and strict option surfaces;
parameterizing them would trade visible policy for callback-heavy helpers. The
shared prompt manifest/hash, candidate JSON, registry resolver, diagnostics,
source ordering, and entity reconciliation packages already own the exact
mechanics that recur without erasing those distinctions.
Intentional Complexity And Duplication To Preserve
internal/modules/generic/register.Register,internal/modules/seriatim/register.Register, andinternal/modules/dnd/register.Registerdeliberately expose the same small registrar shape while retaining family-local registration policy and diagnostics. Combining them would move extension ownership out of the domain registrars and weaken the composition boundary established by ADR-0004.- Seriatim's
decodeOptionalSegmentID,decodeOptionalNumber, and timestamp checks deliberately keep separate external number handling at the input adapter. Segment IDs accept only positive canonical decimal JSON numbers or strings, while optional timestamps accept finite non-negative decimals and preserve their lexicaljson.Numberform in generic metadata. Replacing these with a permissive generic number coercer would weaken the documented source contract and leak an external-format decision intocore/source. generic/valid_jsonandgeneric/valid_json_schemaboth detect malformed JSON, but remain useful separate serialized-validator capabilities. The former is a cheap format-only boundary; the latter must parse an instance to apply an independently supplied schema and can be selected without the former. MOD-002 concerns repeated compilation within one schema validator, not merging the two module identities or requiring a particular chain.internal/modules/dnd/register.registerModules,registerEvidence,registerValidators, andregisterDefaultChainsuse explicit typed registration lists. At this architectural pass, that repetition preserves artifact Go types, module-specific validator order, and registrar-owned production policy. Later D&D stages may evaluate individual shared mechanics, but should not replace these lists with a dynamically typed registration engine.internal/framework/pipeline.RegisterArtifactCodecandexactTypedValueperform apparently repetitive exact-type checks around private erasure. The checks deliberately turn incompatible values into errors at each erased boundary rather than permitting a panic or accepting a near-matching type, preserving ADR-0003.internal/cli.runPipelineCommandis a large linear orchestrator, but its ordering is policy: syntax and config rejection precede run identity and debug allocation; resolution and profile inspection precede module preparation and input parsing; framework success precedes durable output; and terminal debug publication precedes the optional JSON receipt. Existing helpers isolate reference selection, recomputation, stores, output, result encoding, and terminal error precedence. A generic lifecycle abstraction would hide physical-state allocation and publication boundaries; bounded parsing fixes such as CFGCLI-002 should not reorganize that lifecycle.internal/core/config.validatePipelineProfilesexplicitly walks pipelines, ordered steps, lanes, bindings, and references. Its nested structure mirrors the public configuration shape and retains the nearest pipeline/step/lane context in errors. Replacing it with a reflection-driven validator would weaken those diagnostics and the presence-aware file-model boundary.internal/cli.selectedReferenceTargetsandrecomputePolicyperform explicit resolved-shape traversals for distinct CLI policies: disambiguating reference selectors against selected module capabilities, and computing the forward forced/backward reusable checkpoint closure. Keeping these typed traversals separate avoids adding command syntax or checkpoint policy to the framework resolver.pipeline.ResolvePipelineandresolveArtifactLaneare long, but their linear sections retain the authoritative composition order: normalize identity, select lanes, prove capabilities and exact artifact variants, resolve stage-local references and validator chains, apply effective LLM profiles, validate options, and only then compute the digest. Splitting these checks into a generic stage engine would erase the different input, chunk, typed-lane, validator, and output contracts. PIPE-001 is a bounded normalization fix and should not reorganize this sequence.pipeline.validateGeneratedBindingshas deeply nested traversal because it proves a cross-step selector against ordered producer identity, the target's declared slot, accepted artifact kinds, registered codec, and accepted media types in one pass. Those checks are distinct static composition invariants; materialized bytes, runtime handoff construction, and checkpoint hydration remain separate runtime owners.- The stage, validator, codec, and evidence registries intentionally use private typed entries and small stage-specific lookup methods. Their repetition preserves compile-time generic types until a narrow erased closure, exact artifact-kind variant selection, and stage-specific diagnostics. PIPE-002 concerns redundant request copies around those closures, not the typed registry split itself.
- Generated-reference handoff deliberately decodes and re-encodes an accepted normalized artifact through the registered producer codec even though fresh outputs were already serialized. The same boundary also receives hydrated checkpoint artifacts, so canonicalizing once per unique producer verifies exact kind, schema, media type, and bytes before fanout. REF-002 removes only repeated output discovery; it must not bypass this codec check.
- Operation paths clone reference sets for each module request while retaining a separate pristine set for dependency fingerprints and validators. That apparent duplication isolates module mutation across chunks and retries and keeps validators on authoritative inputs. PIPE-002 applies only to adjacent construction-builder copies and should not remove these operation-time ownership boundaries.
Runner.Runis a long linear coordinator, but its sequence is the runtime contract: source state precedes chunk planning; a rejected chunk plan skips lanes but remains an encodable ordinary outcome; ordered steps are barriers; evidence is built only from accepted normalized artifacts; and logical output encoding is last. RUN-001 requires narrow cancellation gates and should not replace that readable lifecycle with a generic stage engine.- Extract jobs and lane continuations deliberately use separate pools, each bounded by the resolved worker count. This permits a completed lane to enter its serial merge/normalize continuation while other lanes are still extracting, without letting either class grow with lane or chunk count. The two pools should not be collapsed merely to make the configured limit a process-wide semaphore; provider calls have their own shared scheduler.
- Chunk, extract, merge, and normalize retry closures repeat candidate debug,
rejection, and warning mechanics around different typed operations and
checkpoint transitions. Their common retry counter and cancellation policy
appropriately live in
runWithRetry, while stage-specific serialization, normalize fallback, and state recording remain explicit. RUN-003 calls for a small terminal-warning result, not a parameter-heavy generic stage runner. synchronizedCheckpointRecorder,synchronizedDebugRecorder, andsynchronizedLLMDebugRecorderintentionally serialize collaborators whose contracts do not promise concurrent safety. Each adapter owns one lock and invokes only its wrapped collaborator, so there is no framework lock-order cycle; removing them would push concurrency requirements into filesystem and test implementations.- Filesystem checkpoint recorder and loader methods intentionally repeat the stage-shaped interface surface. Each small adapter fixes a stage, manifest status, dependency set, and payload codec before delegating to shared manifest validation or publication. Collapsing them into a reflection- or string-driven state engine would weaken typed call sites and contextual validation; STATE-001 concerns only their shared path-component encoding.
- The chunk-plan store intentionally does not use the general
fileiowriter. Its source-addressed directory is an independently replaceable cache entry, and itsos.Root-relative implementation rejects non-directory digest entries, non-regular or symlinked plans, named pipes, and replacement races, while syncing a private temporary file before rename. Those stronger cache-entry mechanics should remain local rather than being generalized into output/debug/checkpoint publication. - Source metadata, accepted chunk maps, and evidence contexts retain separate clone helpers because their ownership graphs and canonical invariants differ: source metadata handles nested dynamic values and cycles, chunk maps own annotations and chunk slices, and evidence contexts own source units and metadata. STATE-002 removes only copies of an already-owned decoded graph; it should not replace these helpers with reflection or remove encode-time isolation.
PromptKitClient.CompleteStructureddeliberately keeps prompt preparation, the frozen prepared-detail snapshot, provider execution, PromptKit schema validation, application decode, profile recording, and debug material assembly explicit. Their order proves that debug and generation describe one prepared request and that malformed provider output cannot become a typed result. LLM-001 changes only the outward error chain; it should not hide this sequence behind a generic transport pipeline.- The CLI profile inspector and production PromptKit client intentionally own separate engines while sharing the same profile-source option construction. Inspection must resolve configured, fallback, built-in, and local-backend policy without credentials, provider contact, or capacity admission; reusing the runtime engine would blur that preflight boundary.
- Module prompt manifests and PromptKit YAML message lists explicitly enumerate module-owned files, shared fragments, stable reference blocks, transcript blocks, lane-specific grounding, and final instructions. This duplication is content policy: it makes exact model-visible byte order and cache-control breakpoints reviewable per workload. LLM-003 adds collision rejection, and LLM-004 shares only the underlying filesystem mechanics; neither should generate manifests dynamically or infer prompt order from filenames.
- The application scheduler and PromptKit backend limits intentionally remain layered. The scheduler is the one process-wide provider-call bound across profiles and modules, while PromptKit owns selected-backend capacity and maps its exhaustion through a provider-neutral sentinel. LLM-002 closes an admission/cancellation race without combining these limits or moving provider capacity policy into the pipeline worker pools.
Areas Reviewed Without Findings
Architecture And Dependency Boundaries
- Composition root:
internal/cli.newProductionComponentsconstructs the complete registry set and asset registry, then invokes only the generic, Seriatim, and D&D family registrars. Direct production imports confirm thatinternal/cliis the only layer importing those registrar packages. - Dependency direction: A direct production import map found no core or
framework package importing
internal/modules, no module importinginternal/cli, no concrete generic or Seriatim module importing D&D, and no module importing the file-backed checkpoint, chunk-plan, debug, file-I/O, or debug-bundle implementations. PromptKit is imported directly only byinternal/framework/llm. - Graph cross-layer calls: The refreshed graph reported one
framework-to-module edge from
pipeline.Prepareto a symbol namedrequestin a D&D validator test. Tracing it showed a confidence0.06suffix match from the local closure call inPrepare;trace_pathclassified the target as test-only, and the production import map disproved a dependency. The graph reported no module-to-CLI calls. - Assets leaf:
assets/package.goimports onlyembedandio/fs, embeds content, and exposes the read-onlyFS() fs.FSaccessor. It contains no business logic and has nointernalor PromptKit dependency. - Fixed pipeline shape:
pipeline.ResolvePipelineresolves input and chunk once, fixed extract/merge/normalize bindings per artifact lane, and one output binding. Ordered steps are barriers around those fixed lanes rather than arbitrary graph topology.pipeline.Prepare,Runner.Run,runPreparedSteps, andrunLanesretain that shape through construction and execution. - Typed artifact boundary: Typed registrations retain the exact Go type for
codecs and lane operations. Private erasure in
RegisterArtifactCodecandexactTypedValueverifies exact types and returns contextual errors; normalized values cross into output through serialized artifacts. - Physical-state ownership: The CLI owns root selection, store factories,
and durable file placement (
chunkPlanStoreForRun, checkpoint/debug setup, andwriteOutputFiles). The framework receives collaborator interfaces and returns logical output files. The generic JSON output module's direct import ofinternal/framework/chunkmapvalidates and republishes the accepted serialized chunk-map contract; it neither chooses a physical root nor writes files. - Accepted architectural decisions: ADRs 0001–0005 and 0007–0012 were read against the current high-level composition. Apart from ARCH-001, the composition root, fixed ordered pipeline, typed boundary, domain packaging, canonical chunk-plan policy, separate state surfaces, checkpoint policy, evidence rules, workload profile ownership, centralized asset leaf, and deterministic entity identity boundary have corresponding current owners.
Configuration And CLI Composition
- End-to-end command path:
RunWithOptionsnormalizes injectable process collaborators once and dispatches torunPipelineCommand. The run command parses and normalizes command input, discovers and loads configuration, applies command overrides, builds the effective catalog, resolves reference selectors and the pipeline, inspects effective profiles, materializes references, constructs runtime/state collaborators, invokespipeline.Run, publishes output files, terminalizes debug state, and only then publishes a requested JSON receipt. - Precedence and resolution:
loadConfigenforces explicit--configoverNOTARIUS_CONFIGover the system default, then appliesDefault, file configuration, supported environment overrides, and run-only CLI overrides in order.Config.Resolverecomputes derived worker defaults, validates, clones the selected profile, and delegates catalog-dependent composition to the framework resolver. Apart from CFGCLI-001 and CFGCLI-002, unknown fields, malformed values, normalized-key collisions, unknown command flags, and invalid selected modules/options are rejected at their owning boundary. - Profile-source equality: Validation-time
validateExplicitPromptKitProfilesand runtimebuildProductionLLMClientpass the same profile directory, profile file, mapped local backend, and shared fallback asset registry. Effective profile collection is sorted, deduplicated, and limited to selected LLM-backed modules and validators, so inspection and runtime selection use the resolved profile values rather than recomputing inheritance. - Session identity:
resolvePromptSessionIDuses a versioned SHA-256 value over the trimmed resolved input-module key, a separator, and exact raw input bytes. It contains no pipeline ID, reference, profile, retry, input path, working directory, or run ID; an explicit non-empty session replaces the generated value. Run contracts verify the same effective session reaches all prompt-facing requests, manifests, debug metadata, and checkpoint identity. - Reference and recomputation controls: CLI reference selectors are resolved only against selected chunk/extract/merge/normalize capabilities before the authoritative effective resolution. Recompute policy is derived after reference materialization, forces the requested step and transitive consumers, and requires reusable checkpoints for non-forced transitive producers. Focused contract tests exercise selector ambiguity, lane selection, ordered dependency closure, and execution behavior.
- Publication and terminal outcomes: Syntax/config failures before run
identity allocate no output or debug state. After debug allocation,
resolution, profile, preparation, input, framework, partial-summary, and
output failures all pass through
failPipelineCommand.terminalizewrites a run report once, preserves an existing primary failure over report/error-log failures, promotes a success-report failure to primary, and reports other persistence failures secondarily. Framework cancellation follows the same wrapped primary-error path. Durable outputs are attempted only after runner success; a JSON result is encoded before output publication but written to stdout only after output and debug terminalization. A receipt-delivery failure leaves already published bundles intact and returns failure. - Test ownership: Configuration tests own strict file/env application, structural validation, effective cloning/digests, and redaction. CLI command, run, reference, recomputation, session, production, example, result, and state contracts assert process-level ordering and side effects rather than merely repeating lower-level resolver assertions. The two uncovered command/parser cases are recorded as CFGCLI-001 and CFGCLI-002.
Pipeline Resolution, Preparation, And Typed Registries
- Static composition:
ResolvePipelinerejects mixed legacy/ordered shapes, empty and duplicate normalized step/lane identities, invalid invocation filtering, unknown modules, missing capabilities, incompatible artifact variants, unsupported lane-level validators, invalid generated selectors, and missing output capabilities before producing a resolved value. Apart from PIPE-001's direct-call collision, selected lanes and steps have stable sorted/order-preserving identities. - Effective policy and identity: Execution classes come from normalized
registry specs. Command override → binding → pipeline LLM-profile precedence
applies only to selected LLM-backed modules and validators; deterministic
bindings reject explicit profiles. Module and validator option validators
receive owned maps before
resolvedPipelineDigesthashes the complete effective composition. Digest tests cover map canonicalization, validator policy, artifact schema identity, effective profiles, and exclusion of the digest field itself. - Typed registry boundary: Extractor registrations retain one exact Go type;
merger, normalizer, and typed-validator registrations select an exact
module/artifact-kind variant; codecs validate complete schema/media identity;
and evidence projectors must match the active codec type. Every erased
operation checks the implementation or value type and returns an error rather
than asserting or panicking. Kind-neutral
Specmethods are confined to catalog inspection, while behavior-sensitive resolution uses exact variant lookups. - Construction boundary:
Preparevalidates the resolved shape and needed registries, clones retained bindings, options, validator chains, reference targets, schemas, and bytes, then constructs input, chunker, chunk validators, every ordered typed lane and local validator chain, output, and the optional evidence plan before returning. Implementations and operations remain private; public prepared bindings/lanes are separate clones. Focused tests verify deterministic construction order, late failure before input parsing, nil and identity rejection, generated-selector retention, and independent builder reference inputs. PIPE-002 records only the extra adjacent copies. - Checkpoint supplements: Preparation collects component-provided semantic fingerprints only after the complete implementation set exists. Scopes include stage, globally unique lane identity, module, and validator position; empty or duplicate values fail preparation, results are sorted, and the accessor returns a defensive copy. Scheduling limits and diagnostics are not included. Resolved composition—including options, reference selectors, effective profiles, retries, and validator order—remains owned by the resolved digest rather than being redundantly restated as component fingerprints.
- Registry comparison: Input, chunker, and output registries consistently normalize keys/specs, reject nil validators/builders, validate options on owned maps, clone stored specs, sort discovery output, and verify constructed identity. Typed stage and validator registries add only the exact-type and artifact-variant mechanics their contracts require. Validator-chain lookup distinguishes absent, default, explicit replacement, and explicit empty chains while returning defensive copies. No dead compatibility path or safe consolidation was found beyond the copy reduction in PIPE-002.
- Test ownership: Contract tests cover clone/serialization boundaries; registry tests cover invalid registration, sorted/defensive discovery, strict options, exact types, schema compatibility, and evidence ownership; resolution tests cover heterogeneous variants and effective identity; and preparation tests own all-before-parse construction and fingerprint collection. The missing module-reference collision case is recorded in PIPE-001 rather than as a separate test-only finding.
References And Ordered Handoffs
- External path: Pipeline defaults are filtered to declared slots; local
bindings override eligible defaults; CLI overrides apply to one exact final
target; and unbind removes only an external binding before required-slot
enforcement. Selected legacy lanes determine eligible targets, while
explicit ordered steps reject
--only. Config-relative and CLI-relative paths remain distinct, materialized values retain digest/media/size/origin, preparation supplies owned construction inputs, and operation requests get independent content. REF-001 records the only missing bound in this path. - Generated path: Pipeline-level selectors are rejected. Ordered local or step bindings must name a declared earlier step and selected lane whose exact artifact kind, registered codec media type, and target slot constraints are compatible. Because all edges point strictly backward, forward references and cycles are rejected during resolution. At the step barrier, exactly one accepted normalized output is decoded and re-encoded through its codec; missing, duplicate, wrong-kind, wrong-schema/media, oversized, rejected, or unavailable producer state stops the consumer before execution. REF-002 concerns only the repeated lookup used to establish that cardinality.
- Resume and recomputation: Ordinary resume progressively compares generated dependency fingerprints on extract, merge, and normalize checkpoints. Selective recomputation instead forces the selected step and transitive consumers while requiring unforced transitive producers to supply accepted normalize state without extract/merge files or dependency matches. Hydration validates stored provenance and canonical bytes through the active codec before publishing an owned normalized output. REF-003 records the omitted size field in the ordinary consumer fingerprint.
- Provenance and evidence: External manifests contain paths, digests,
media, sizes, and binding sources; generated manifests contain bounded
producer, codec/schema, digest, and size identity without content or a fake
path. Runtime stage debug envelopes omit reference sets, contract JSON omits
ReferenceItem.Content, and focused assembled-module tests confirm generated references ground operations without becoming source evidence. - Complexity ownership: Static target resolution keeps precedence,
unbinding, and required-slot policy together for contextual errors;
validateGeneratedBindingsowns cross-step static compatibility; andbuildStepReferenceSetsowns the runtime barrier. Apart from REF-002's repeated full-output scan, their explicit traversals preserve distinct invariants more clearly than a generic graph or reflection engine. - Test ownership: Profile and CLI reference contracts cover defaults, local precedence, unbinding, required slots, selected targets, ambiguity, and typed producer compatibility. Reference materialization tests cover origin, UTF-8, media, size diagnostics, warnings, and provenance; handoff and runner tests cover canonical fanout, invalid/missing producers, ordering, fingerprints, and accepted checkpoint hydration; assembled integration tests cover semantic generated-reference consumers. REF-001 and REF-003 identify the two unproved identity/resource details.
Execution, Validation, Retry, And Concurrency
-
Runtime state diagram: The audited framework lifecycle is:
input validation / owned metadata -> source load or parse -> document validation -> chunk load or plan -> materialize -> whole-plan validator chain -> rejected: terminal ordinary outcome ----------------------+ -> accepted: for each ordered step | -> generated handoff barrier | -> initialize lane checkpoint states | -> dispatch extract jobs (chunk first, lane second) | -> operation -> canonical candidate -> validators | -> retry | rejected chunk | accepted chunk | -> completed lane enters bounded continuation | -> merge -> validators -> retry/reject | -> normalize -> validators -> retry/reject | -> await all started work -> stable lane merge -> barrier | +---------------------------------------------------------------+ -> manifest/evidence -> logical output encoder any framework error or parent cancellation -> cancel derived work -> stop dispatch -> drain/await started work -> stable error selection -> failed output (no logical files) -
Ordering and bounded work:
dispatchExtractJobsenumerates chunks first and lanes second. One fixed extract pool and one fixed continuation pool are each bounded byExtractWorkers; a lane continuation is serial merge then normalize, while different completed lanes may overlap remaining extracts. Results are indexed by lane and chunk, extracts are sorted by chunk index, lane outputs merge in prepared order, and framework errors sort by stage, lane, and chunk after child cancellation errors are filtered. Reverse- completion, continuation-overlap, continuation-bound, provider-bound, and stable-error tests confirm these contracts independently of goroutine finish order. -
Cancellation and liveness: The lane engine checks cancellation before starting queued extract and continuation work, stops dispatch, clears unlaunched continuations, drains worker results, receives every launched completion, and waits for both pools. The retry loop checks before an attempt and after failed/rejected attempts. Parent cancellation takes precedence over collected child cancellation, while an actual framework failure is selected deterministically when the parent remains live. RUN-001 records the two unguarded outer dispatch boundaries; no additional worker leak or deadlock path was found. RUN-004 records the missing collector invariant comment.
-
Terminal outcomes: Chunk, extract, merge, and normalize validators inspect whole candidates, not partial streams. A rejection is collected as ordinary output: a rejected chunk plan skips all lanes, a rejected extract chunk does not prevent accepted chunks in that lane from merging, and merge/normalize rejection terminates only that lane. A framework error cancels sibling work, blocks later ordered steps, and returns through
failOutput, which leaves no logical output files. RUN-002 records the typed-value ownership violation; serialized validators receive owned canonical bytes. -
Retries and diagnostics: Operation and complete validator-chain execution share one retry budget. Debug persistence failure is terminal rather than retried; cancellation stops retries; rejections record their final attempt; and normalize retry directives use the same budget before validating their final fallback. Candidate serialization precedes validation and accepted checkpoint serialization follows it, so invalid candidates are not recorded as success. Intermediate-attempt warnings remain only in attempt debug as intended; RUN-003 records loss of the final rejected attempt's warnings.
-
Collaborator synchronization: Runner-local wrappers serialize checkpoint, debug, and LLM-debug collaborators without acquiring multiple framework locks at once. Module/provider concurrency remains separately bounded by the shared LLM scheduler. No duplicate recording or permit leak was found in the audited execution layer; durable checkpoint transitions and filesystem atomicity remain in the next audit area.
-
Focused test review: Concurrency tests exercise chunk-first dispatch, reverse completion, extract and continuation bounds, intended overlap, provider limits, stable stage/lane error priority, ordinary rejection, parent cancellation, and stopped queued extracts. Retry, rejection, candidate- encoding, session, manifest, typed-checkpoint, debug-attempt, accepted- checkpoint, and generated-handoff tests cover the remaining execution branches. The consequential missing runtime cases are included in RUN-001 through RUN-003 rather than duplicated as test-only findings.
State, Checkpoints, Debugging, And File Safety
- Independent state lifecycles:
runPipelineCommandselects output, chunk-plan, checkpoint, and debug roots only in the CLI. Bypass returns before resolving or constructing a chunk-plan store; disabled checkpoints construct only no-op collaborators and reject resume; enabled recording always creates a recorder but creates a loader only for resume; and debug allocation occurs only after an explicit debug request. None of these roots enters a module or another state family's identity, and debug records are never read by reuse code. - Chunk-plan cache: The store accepts only a canonical lowercase SHA-256
source digest as its key, opens the exact digest directory through
os.Root, rejects symlink/non-directory digest entries and symlink/non-regular plan files, strictly decodes one schema-versioned JSON value, revalidates plan provenance and digest, and returns typed missing/invalid/hit decisions. A reused plan is cloned, materialized against the current document, and passed through the whole-plan validators. Only a newly accepted plan is published; random 0600 temporary files are synced and renamed within the confined 0700 entry. Missing, invalid, or deleted entries are reconstructible. - Checkpoint identity and ordinary resume: Identity hashes the complete resolved pipeline, raw input, selected lanes, runtime overrides, external and generated reference provenance, observed profile/runtime provenance, and prepared component fingerprints. The directory uses readable/prefix components, while every manifest retains and validates the complete identity digest and exact stage/step/lane/module/dependencies. Payload publication precedes a succeeded manifest, and the manifest retains payload digests, so interruption or a changed dependency produces execution/invalidation rather than stale reuse. STATE-001 records the remaining non-injective step/lane component mapping.
- Selective recomputation: The CLI computes a forward forced closure and a
backward set of required reusable producers. The runner applies
forced_recomputebefore decoding an otherwise reusable artifact. Required producers use only an accepted workspace-v3 normalize manifest with exact invocation, producer provenance, artifact digest, active codec/schema/media, and canonical bytes; they deliberately do not require extract/merge state or current consumer dependencies. A failed required load records its typed decision and stops before producer or consumer execution. - Decisions and diagnostics: Checkpoint readers assign category and reason code at the validation branch for missing, path, read, decode, schema, identity, stage, dependency, payload, digest, and reuse outcomes. Runner hydration adds codec and canonicality codes, forced policy adds the recompute code, and required-predecessor errors name only stable step/lane identity and code. Event detail is selected from code-owned text, UTF-8 normalized, and bounded rather than copied from a cache error or caller payload.
- File safety and recovery: Output names are all validated before exclusive run-directory creation; existing output/debug run directories are refused; later output failure intentionally retains the new partial bundle. General state writes use same-directory temporary files and atomic rename after narrow relative-path checks, with 0700/0600 cache/debug modes and 0755/0644 output modes. General file I/O rejects existing symlink components; the chunk-plan store additionally uses root-relative no-follow opens and race-oriented entry rechecks. No production path automatically moves, deletes, or cleans output, cache, or completed debug state; debug allocation removes only its just-created partial leaf on setup failure.
- Debug and terminalization: A bundle owns distinct summary and trace directories. Summary records use redacted invocation/configuration and bounded checkpoint decisions; trace requests receive cloned application payloads and no environment enumeration. Debug write failures fail the requested run but cannot affect checkpoint identity or reuse. Guarded terminalization writes one report, preserves an existing primary error over report failure, attempts one error log, and joins persistence failures only as secondary diagnostics. STATE-001 records trace-name collisions, not a cache dependency or redaction leak.
- Canonical ownership: Source digesting uses canonical document/plan
serialization and cycle-aware metadata cloning. Chunk-map and evidence-
context codecs strictly validate schema, reject unknown fields and trailing
values, canonicalize nested identities/annotations/context, and return owned
graphs. Their encode-time clones are required ownership boundaries;
STATE-002 records only the redundant copies after decoding already-owned
JSON. STATE-003 records the one extractable duplicate writer; checkpoint
stage adapters, cache-specific
os.Rootmechanics, and type-specific clone helpers remain intentional. - Focused test review: File-I/O and debug-bundle tests cover confinement, atomic replacement, symlinks, modes, refusal, cleanup, and redaction. Checkpoint tests cover identity, manifests, interrupted publication, categories/reason bounds, dependency invalidation, accepted normalize, and codec canonicality. Chunk-plan tests cover corrupt entries, races, named pipes, strict decode, materialization, and permissions; chunk-map and evidence-context tests cover canonical round trips and ownership. CLI cache, recomputation, state-hardening, run-contract, and terminal tests cover the composed lifecycles and primary-error precedence. The uncovered identity, copy, and writer issues are recorded as STATE-001 through STATE-003.
LLM Runtime, Prompt Filesystems, And Assets
- Completion path: Every production structured completion crosses the scheduled client and the shared scheduler before the PromptKit adapter. The adapter forwards the stable session ID directly, renders inputs and variables once, prepares one frozen execution target, snapshots its details, invokes that exact prepared target, accepts only PromptKit schema-valid structured output, decodes into the caller's target, records normalized profile provenance, and emits caller-owned debug material. Context and capacity are adapted to provider-neutral categories; LLM-001 records the raw error chain that still escapes on other provider failures.
- Scheduling: Production constructs one scheduler around the one PromptKit client, so all LLM-backed bindings and validators share the configured application limit. Queue insertion and grant are FIFO, active counts change under one mutex, canceled queued entries are removed or return an already granted permit, and idempotent deferred release covers success and error. LLM-002 records only the simultaneous grant/cancel dispatch window; no permit leak or second application scheduler was found.
- Profiles and cache identity: Preflight inspection and runtime construction share configured directory/file, embedded fallback, built-in catalog, and optional local-backend option builders while retaining separate engines. Inspection does not load credentials or contact a provider. Checkpoint fingerprints cover the compiled catalog identity, every configured profile file's content identity, fallback asset identity, reasoning override, and a hashed local endpoint without publishing profile contents, paths, endpoint, environment values, or concurrency limits. Successful manifests record only selected non-secret profile/provider/model/backend/reasoning provenance.
- Sessions, bytes, and cache points: The CLI resolves one stable session ID
before physical run construction and the same value flows through pipeline
requests into PromptKit's native
SessionID; it is not reconstructed from a transcript. All maintained extraction prompts render a stable shared system, identity/reference prefix before the transcript, then lane-specific evidence and grounding, with the final instructions last. Focused composition tests assert identical cached prefixes, exact-once input placement, suffix order, and ephemeral cache controls at reference, transcript, and final-instruction boundaries. - Asset ownership: The root
assetspackage imports onlyembedandio/fsand exposes one read-only filesystem. Module manifests explicitly flatten only selected module files and shared fragments; the asset registry clones bytes, rejects duplicate flattened prompt/schema/profile roots, validates schema manifests, and hashes selected semantic assets. All fourteen D&D prompt manifests currently use unique normalized virtual names; LLM-003 records thatModulePromptFSdoes not enforce this invariant itself. - Model-visible content: Every maintained PromptKit YAML file, shared
fragment, module instruction/grounding file, private response schema, and
fallback profile was searched and representative complete compositions were
read. The only provider/model-named content is the intentional fallback
profile target. No bearer token, credential variable/value, endpoint,
provider-generated opaque identifier, UUID, digest-copy instruction, or
duplicated model instruction was found in model-visible assets. Durable
private-schema
$idvalues remain schema metadata rather than requests to reproduce internal identifiers. - Filesystem mechanics and tests: Asset and module prompt filesystems enforce valid scoped paths, sorted directory entries, read-only metadata, and owned bytes, while prompt/schema loaders are exercised through real PromptKit preparation. LLM-004 records the duplicated map-filesystem implementation and its drift. Focused client tests cover direct sessions, prepared-snapshot consistency, default/explicit/fallback/local profiles, response validation/decode, capacity mapping, debug redaction, profile provenance, and scheduled concurrency; scheduler and promptfs tests cover limits, queued cancellation, release, path scoping, reads, and ownership.
Generic And Seriatim Modules
- Seriatim source boundary: The transcript adapter strictly decodes one top-level JSON object, requires metadata and a non-empty segment array, validates unique positive canonical IDs, non-empty speaker/text values, and finite ordered timestamps, then chooses a deterministic document ID from the request, metadata, or raw-byte digest. It constructs generic units with exact self-references, computes the canonical source digest, and runs the framework document validator. Transcript-only speaker/start/end helpers and external metadata remain in the adapter package; the resulting document exposes only generic source fields and metadata.
- Generic chunk planning: The unit chunker validates the complete source document, computes source-addressed ranges by unit position with bounded overlap, returns only the source digest and unit-ID ranges, and leaves materialization, chunk identity, and plan validation to the framework. Tests cover defaults, exact boundaries, overlap, invalid/unknown options, empty sources, and non-retention of source units. MOD-001 records the only avoidable option-decoder surface found.
- Merge, normalize, and validation ownership: Typed append-order combines values in framework order through a family-supplied typed combiner, while typed no-op normalization preserves the exact artifact type; D&D registrars instantiate those domain-neutral mechanics without introducing D&D imports into the generic packages. Always-accept/reject, JSON, and JSON-Schema validators retain distinct selectable targets and strict empty option sets. Schema mismatch remains an ordinary rejection and malformed schema remains an operational error. MOD-002 records only repeated schema compilation.
- JSON output and evidence: Output decoding rejects unknown and context-incompatible fields, normalizes/deduplicates/sorts evidence lane allowlists, and validates configured lanes before preparation. Preparation retains the full configured policy while selecting active exact-typed evidence projectors; runtime evidence is built only from compatible accepted normalized outputs. The encoder sorts lanes and logical files, rejects unsafe or colliding lane filenames, emits no physical paths, validates and republishes opt-in chunk-map and prepared evidence-context artifacts, and preserves annotation number bytes. MOD-003 records only unreachable clone code outside those live ownership paths.
- Registration and dependency direction: Generic registration checks all required registries before mutation and registers the unit chunker, four validators, and JSON output; Seriatim independently registers only its input adapter. Direct import inspection found no D&D package in either family and no Seriatim package below the input registrar. Typed append/no-op libraries are instantiated by the D&D family rather than acquiring domain concepts.
- Composed contracts: Focused module tests cover strict registration, source parsing/identity/metadata, chunk planning, typed combination, validator outcomes, JSON bundle determinism, chunk-map/evidence validation, and mutation isolation. Framework evidence-preparation/output tests prove invocation filtering and exact codec/projector compatibility; maintained CLI example contracts prove Seriatim input through generic JSON publication and opt-in chunk-map provenance.
Shared D&D Types, Codecs, And Family Mechanics
- Durable family inventory:
internal/modules/dnd/types.go, the ten codec packages, and D&D registration agree on ten exact kind/type variants. Each has one LLM extractor, one typed append-order merger, a concrete normalizer, a typed noop alternate, an evidence projector, artifact-specific validators, and extract/normalize default chains. The convention matrix above records the three intentionally LLM-backed registry normalizers and every reference dependency; no missing or extra durable family registration was found. - Codec boundary: All family codecs publish
application/json, codec-local v1 durable schemas and count metadata, strictly decode through the shared candidate JSON helper, reject unknown fields and trailing values, and apply family required-value validation only at durable encode/decode. Ownership clones are present for the two mutable nested shapes that need them, and schema reads return owned bytes. The common adapter surface is already at its natural owner; a further generic codec would have to parameterize the exact behavior the typed boundary is meant to expose. - Reference mechanics: Shared campaign slots return owned media-type and slot declarations; source ordering uses document positions with literal-ID fallback, stable exact deduplication, and diagnostic preservation of invalid refs. Strict unit-reference JSON parsing, current-document citation expansion, typed family canonicalization, validator admissibility, and direct evidence projection remain distinct owners. DND-CORE-001 records the only dead/lossy API found in this layer.
- Shared registry and reconciliation mechanics: The generic registry resolver validates one typed JSON artifact, bounds it by the family-declared maximum, copies content at retention boundaries, and caches raw and semantic identities behind a mutex. Entity reconciliation renders source-free selectors and bounded transcript windows, keeps durable IDs model-invisible, discards unsafe or overlapping proposals, and returns defensive copies. Diagnostics and Unicode comparison policy are shared and bounded/versioned; registry-specific identity and merge outcomes remain deferred.
- Registration and assets: The registrar validates all collaborators before mutation and gives every failed component contextual identity. All ten durable schemas, fourteen family prompt manifests, the shared reconciliation schema, evidence projectors, generic validator capabilities, typed always-accept/reject capabilities, default chains, and the fallback profile are registered. Sequential registration is intentionally fail-stop rather than transactional; production discards the composition on any error.
- Repeated method classification: Module-local metadata and fingerprint methods enumerate semantic cache/checkpoint inputs; constructors and registrars preserve exact dependencies and typed error context; strict empty option decoders make supported surfaces locally visible. Existing shared helpers already consolidate prompt assets, candidate JSON, reference slots, comparison, diagnostics, resolution, and reconciliation. No additional callback- or reflection-driven helper reduced demonstrated drift.
- Deferred lane questions: Registry identity and reconciliation policy is
assigned to the registry review; occurrence categories and registry
projections to the occurrence review; spell catalog, scene chunking, scene
IDs, and scene classification to the spells/scenes review; and combat gates,
engagement uniqueness, collective labels, and enemy observation ordering to
the combat/enemy review. This area therefore remains
Revisituntil those reviews confirm the matrix's documented exceptions.
Validation Record
| Date | Scope | Command or check | Result |
|---|---|---|---|
| 2026-08-08 | Initial worktree | git status --short |
Pass; no output |
| 2026-08-08 | Knowledge graph | Full index as notarius-audit-92e8907 |
Pass; 8,322 nodes and 47,887 edges; branch/head matched the production target |
| 2026-08-08 | Baseline tests | go test ./... |
Pass |
| 2026-08-08 | Baseline static analysis | go vet ./... |
Pass |
| 2026-08-08 | Baseline build | go build ./cmd/notarius |
Pass |
| 2026-08-08 | Baseline whitespace | git diff --check |
Pass |
| 2026-08-08 | Production imports | Direct go list import-edge audit plus graph call tracing |
Pass; no production dependency inversion found |
| 2026-08-08 | Accepted ADR links | Relative Markdown-link target scan under docs/adr/ |
Two unresolved targets recorded as ARCH-001 |
| 2026-08-08 | Audit target integrity before configuration/CLI review | git diff --quiet 92e89076a268089e703978fb9d7176200e93344c..HEAD -- . ':(exclude)docs/roadmap/**' |
Pass; production target unchanged |
| 2026-08-08 | YAML decoder contract | go doc gopkg.in/yaml.v3.Decoder.Decode and ParseFileConfigYAML call trace |
Decode consumes the next document; no EOF/second-document check, recorded as CFGCLI-001 |
| 2026-08-08 | Focused configuration and CLI tests | go test ./internal/core/config ./internal/cli |
Pass |
| 2026-08-08 | Focused configuration and CLI static analysis | go vet ./internal/core/config ./internal/cli |
Pass |
| 2026-08-08 | Audit target integrity before pipeline composition review | git diff --quiet 92e89076a268089e703978fb9d7176200e93344c..HEAD -- . ':(exclude)docs/roadmap/**' |
Pass; production target unchanged |
| 2026-08-08 | Pipeline graph review | Architecture, complexity query, exact symbol reads, and call traces for ResolvePipeline, binding normalization, typed registries, Prepare, and checkpoint fingerprints |
Pass; PIPE-001 and PIPE-002 recorded; generated handoff execution deferred to the next area |
| 2026-08-08 | Focused contracts and pipeline tests | go test ./internal/framework/contracts ./internal/framework/pipeline |
Pass |
| 2026-08-08 | Focused contracts and pipeline static analysis | go vet ./internal/framework/contracts ./internal/framework/pipeline |
Pass |
| 2026-08-08 | Audit target integrity before references/handoffs review | git diff --quiet 92e89076a268089e703978fb9d7176200e93344c..HEAD -- . ':(exclude)docs/roadmap/**' |
Pass; production target unchanged |
| 2026-08-08 | Reference and handoff graph review | Architecture, exact symbol reads, and call traces across external materialization, target resolution, operation cloning, generated codec handoff, consumer fingerprints, ordered execution, and accepted-checkpoint hydration | REF-001, REF-002, and REF-003 recorded; no content/evidence leak found |
| 2026-08-08 | Focused pipeline and CLI tests | go test ./internal/framework/pipeline ./internal/cli |
Pass |
| 2026-08-08 | Assembled integration tests | go test ./internal/modules/integration/... |
Pass |
| 2026-08-08 | Audit target integrity before runtime review | git diff --quiet 92e89076a268089e703978fb9d7176200e93344c..HEAD -- . ':(exclude)docs/roadmap/**' |
Pass; production target unchanged |
| 2026-08-08 | Runtime graph and state-machine review | Exact symbol reads, complexity inspection, call traces, cancellation-search coverage, and focused concurrency/retry/rejection/session/checkpoint/debug/manifest/handoff test review | RUN-001 through RUN-004 recorded; bounded work, stable assembly, and worker draining otherwise confirmed |
| 2026-08-08 | Pipeline race tests | go test -race ./internal/framework/pipeline |
Pass |
| 2026-08-08 | Fresh pipeline tests | go test -count=1 ./internal/framework/pipeline |
Pass |
| 2026-08-08 | Shuffled pipeline tests | go test -shuffle=on ./internal/framework/pipeline |
Pass |
| 2026-08-08 | Audit target integrity before state review | git diff --quiet 92e89076a268089e703978fb9d7176200e93344c..HEAD -- . ':(exclude)docs/roadmap/**' |
Pass; production target unchanged |
| 2026-08-08 | State graph and lifecycle review | Exact symbol reads, complexity and call traces across CLI root construction, core file I/O/debug allocation, chunk-plan storage, checkpoint identity/loading/recording, runner reuse decisions, trace/summary writers, terminalization, and canonical codecs | STATE-001 through STATE-003 recorded; independent lifecycle, typed decisions, accepted hydration, debug isolation, and primary-error precedence otherwise confirmed |
| 2026-08-08 | State collaborator race tests | go test -race ./internal/core/fileio ./internal/core/debugbundle ./internal/framework/checkpoint ./internal/framework/chunkplan ./internal/framework/chunkmap ./internal/framework/debug ./internal/framework/evidencecontext |
Pass |
| 2026-08-08 | CLI state and terminal tests | go test ./internal/cli |
Pass |
| 2026-08-08 | Audit target integrity before LLM/assets review | git diff --quiet 92e89076a268089e703978fb9d7176200e93344c..HEAD -- . ':(exclude)docs/roadmap/**' |
Pass; production target unchanged |
| 2026-08-08 | LLM and asset graph/content review | Exact symbol reads and call traces across scheduled completion, PromptKit adaptation, profiles/preflight/fingerprints, sessions, debug, asset flattening/hashing, prompt filesystems, all fourteen manifests, every prompt YAML, shared/model-visible content, private schemas, and the root asset leaf | LLM-001 through LLM-004 recorded; profile parity, exact prompt order/cache points, asset ownership, and secret-free model-visible content otherwise confirmed |
| 2026-08-08 | LLM and prompt filesystem race tests | go test -race ./internal/framework/llm ./internal/framework/promptfs |
Pass |
| 2026-08-08 | Composed CLI and D&D prompt-cache tests | go test ./internal/cli ./internal/modules/dnd/register |
Pass |
| 2026-08-08 | Audit target integrity before generic/Seriatim review | git diff --quiet 92e89076a268089e703978fb9d7176200e93344c..HEAD -- . ':(exclude)docs/roadmap/**' |
Pass; production target unchanged |
| 2026-08-08 | Generic and Seriatim graph/code review | Architecture clusters, exact symbol reads, call traces, scoped caller/search checks, and complete implementation/test inspection across input parsing, chunk planning, typed merge/normalize helpers, validators, JSON output, registration, evidence preparation, and output publication | MOD-001 through MOD-003 recorded; source/domain boundaries and live ownership paths otherwise confirmed |
| 2026-08-08 | Generic and Seriatim module tests | go test ./internal/modules/generic/... ./internal/modules/seriatim/... |
Pass |
| 2026-08-08 | Published source-artifact codec tests | go test ./internal/framework/chunkmap ./internal/framework/evidencecontext |
Pass |
| 2026-08-08 | Composed evidence/output contracts | Focused go test runs in ./internal/framework/pipeline and ./internal/cli for evidence preparation/publication, maintained minimal JSON output, and production chunk-map provenance |
Pass |
| 2026-08-08 | Generic and Seriatim import boundaries | Direct go list import-edge audit plus scoped transcript-adapter import search |
Pass; no D&D dependency in generic/Seriatim packages and no production import of the transcript adapter outside Seriatim registration |
| 2026-08-08 | Audit target integrity before shared D&D review | git diff --quiet 92e89076a268089e703978fb9d7176200e93344c..HEAD -- . ':(exclude)docs/roadmap/**' |
Pass; production target unchanged |
| 2026-08-08 | Shared D&D graph/code review | Scoped architecture, exact symbol reads, inbound traces, and complete implementation/test inspection across ten durable types/codecs, candidate JSON, source-reference utilities, resolver/reconciliation/diagnostics, typed mergers, evidence, registration, default chains, shared assets, and representative family prompt/schema owners | DND-CORE-001 recorded; family registration, codec ownership, reference/evidence separation, shared reconciliation safety, and typed adapter boundaries otherwise confirmed |
| 2026-08-08 | Required shared D&D tests | go test ./internal/modules/dnd/codec/... ./internal/modules/dnd/shared/... ./internal/modules/dnd/register |
Pass |
Coverage Matrix
| Audit area | Status | Packages and documents inspected | Validation run | Finding IDs |
|---|---|---|---|---|
| Architecture and dependency boundaries | Reviewed | Architecture, documentation, and testing policies; internal overview; accepted ADRs; internal/cli/catalog.go; production registrars; root assets package; representative pipeline, typed-codec, output, and state-owner symbols |
Full baseline, fresh graph, direct import map, cross-layer call traces, ADR link scan | ARCH-001 |
| Configuration and CLI composition | Reviewed | docs/config.md, docs/cli.md, docs/operations.md, internal configuration/CLI docs; internal/core/config/; CLI run, catalog, session, profile, result, and terminal owners; focused config, command, run, reference, recomputation, session, production, example, result, and state tests |
Target-integrity check, graph call/data-owner traces, YAML decoder contract, focused tests and vet | CFGCLI-001, CFGCLI-002 |
| Pipeline resolution, preparation, and typed registries | Reviewed | Internal pipeline/module docs; internal/framework/contracts/; pipeline profile, options, module, construction, preparation, fingerprint, stage/validator/chain/codec/evidence registry implementations and focused tests |
Target-integrity check, graph architecture/complexity/call traces, focused tests and vet | PIPE-001, PIPE-002 |
| References and ordered handoffs | Reviewed | Reference and ordered-step sections of configuration, internal pipeline, and state docs; pipeline reference resolution/materialization, preparation ownership, generated handoff, consumer fingerprint, checkpoint hydration, and runner barriers; CLI selector/recomputation owners; focused profile, reference, handoff, checkpoint, recomputation, and assembled integration tests | Target-integrity check, graph architecture/complexity/call traces, focused pipeline/CLI tests, assembled integration tests | REF-001, REF-002, REF-003 |
| Execution, validation, retry, and concurrency | Reviewed | Internal pipeline runtime contract; runner, chunk planning/validation, concurrent lane engine, typed execution/validation, retry/normalize, synchronization, output suppression, and focused concurrency, retry, rejection, session, typed-checkpoint, debug, manifest, candidate-encoding, and handoff tests | Target-integrity check, graph state-machine/call/complexity review, race tests, fresh tests, shuffled tests | RUN-001, RUN-002, RUN-003, RUN-004 |
| State, checkpoints, debugging, and file safety | Reviewed | State and operations docs plus architecture state/security policy; CLI output, cache-root, checkpoint identity, recomputation, debug allocation, and terminal owners; core file I/O, debug bundle, and source digest/clone helpers; framework checkpoint, chunk-plan, chunk-map, debug, evidence-context implementations and focused tests | Target-integrity check, graph architecture/complexity/call traces, state collaborator race tests, CLI tests | STATE-001, STATE-002, STATE-003 |
| LLM runtime, prompt filesystems, and assets | Reviewed | LLM internal/integration/configuration/operations docs and related ADRs; scheduled client, scheduler, PromptKit adapter/profile inspector, profile/source fingerprints, redaction, debug, asset registry, schema loader, promptfs implementations and focused tests; root assets, fallback profile, all fourteen module manifests and prompt YAML files, shared/model-visible fragments, private response schemas, and representative composed loaders | Target-integrity check, graph architecture/symbol/call review, content scan and exact prompt-order comparison, focused race tests, composed CLI and D&D prompt-cache tests | LLM-001, LLM-002, LLM-003, LLM-004 |
| Generic and Seriatim modules | Reviewed | Internal module guide; Seriatim, JSON output, chunk-map, and evidence-context integration contracts; all implementation/tests under internal/modules/generic/ and internal/modules/seriatim/; framework evidence preparation/output paths and focused tests; maintained CLI example and production output contracts |
Target-integrity check, graph architecture/symbol/call/caller review, direct import map, required module and codec tests, focused composed evidence/output tests | MOD-001, MOD-002, MOD-003 |
| Shared D&D types, codecs, and family mechanics | Revisit | D&D internal/module docs and all D&D integration contracts; root durable types; all ten codec packages and candidate JSON; shared references, ordering, citations, inputs, comparison, diagnostics, registry resolver, entity reconciliation, and assets; typed merger/evidence/default-chain/fallback/family registration; representative module asset declarations and focused tests | Target-integrity check, scoped graph architecture/search/traces, ten-family convention matrix, required codec/shared/register tests | DND-CORE-001 |
| NPC, item, and location registries | Pending | — | — | — |
| NPC, item, and location occurrences | Pending | — | — | — |
| Spells, scene chunking, and scene descriptions | Pending | — | — | — |
| Combat turns and enemy events | Pending | — | — | — |
| Test ownership, comments, and final synthesis | Pending | — | — | — |