637 lines
42 KiB
Markdown
637 lines
42 KiB
Markdown
# Codebase Audit
|
||
|
||
## Audit Metadata
|
||
|
||
- **Production target:** `92e89076a268089e703978fb9d7176200e93344c`
|
||
- **Branch at target:** `main`
|
||
- **Audit date:** 2026-08-08
|
||
- **Go version:** `go1.26.5 linux/amd64`
|
||
- **PromptKit version:** `gitea.maximumdirect.net/eric/promptkit v0.5.0`
|
||
- **Knowledge-graph project:** `notarius-audit-92e8907`
|
||
- **Knowledge-graph target:** branch `main`, head
|
||
`92e89076a268089e703978fb9d7176200e93344c`
|
||
- **Initial worktree:** Clean. There were no pre-existing production or roadmap
|
||
changes to record.
|
||
|
||
The commit above is the production snapshot under audit. Later commits that
|
||
change only roadmap audit documents do not change that production target.
|
||
|
||
## Executive Summary
|
||
|
||
Pending final synthesis. The initial baseline is healthy. The architecture,
|
||
configuration/CLI, pipeline composition, and reference/handoff reviews have
|
||
found two Medium findings and seven Low findings, with no production dependency
|
||
inversion or unsafe typed-erasure boundary.
|
||
|
||
## Finding Index
|
||
|
||
Final cross-area ordering is pending synthesis.
|
||
|
||
| ID | Severity | Category | Title |
|
||
| --- | --- | --- | --- |
|
||
| ARCH-001 | Low | Documentation/Comments | Repair broken ADR cross-references |
|
||
| CFGCLI-001 | Medium | Correctness | Reject additional YAML documents |
|
||
| CFGCLI-002 | Low | Correctness | Reject a blank command-level LLM profile |
|
||
| PIPE-001 | Low | Correctness | Reject normalized module-reference collisions in the resolver |
|
||
| PIPE-002 | Low | Efficiency | Clone construction inputs once per builder boundary |
|
||
| REF-001 | Medium | Efficiency | Bound reference reads before allocating the file |
|
||
| REF-002 | Low | Efficiency | Index accepted outputs once per ordered handoff |
|
||
| REF-003 | Low | Correctness | Include canonical size in generated-reference fingerprints |
|
||
|
||
## Findings
|
||
|
||
### Architecture And Dependency Boundaries
|
||
|
||
### ARCH-001 — Repair broken ADR cross-references
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Documentation/Comments
|
||
- **Evidence:**
|
||
`docs/adr/0012-resolve-opaque-entity-identifiers-deterministically.md:15`
|
||
links ADR-0003 as `0003-strongly-typed-stage-interfaces.md`, and line 17
|
||
links ADR-0009 as
|
||
`0009-prefer-minimal-evidence-grounded-extraction-artifacts.md`. Neither file
|
||
exists. The maintained files are
|
||
`0003-typed-interfaces-with-two-zone-data-model.md` and
|
||
`0009-minimal-evidence-grounded-extraction-artifacts.md`.
|
||
- **Impact:** Readers and documentation tooling cannot follow ADR-0012 to the
|
||
two architectural decisions it explicitly relies on. Runtime behavior is
|
||
unaffected.
|
||
- **Recommendation:** Correct only the two relative link targets in ADR-0012.
|
||
- **Preserve:** Keep the accepted decision text and its intended references to
|
||
ADR-0003 and ADR-0009 unchanged.
|
||
- **Validation:** Run a relative Markdown-link check across `docs/adr/` and
|
||
confirm both targets resolve; verify the change contains no decision-text
|
||
edits.
|
||
- **Grouping:** Independent.
|
||
|
||
### Configuration And CLI Composition
|
||
|
||
### CFGCLI-001 — Reject additional YAML documents
|
||
|
||
- **Severity:** Medium
|
||
- **Category:** Correctness
|
||
- **Evidence:** `internal/core/config/file_config.go:327`–`353`
|
||
constructs a `yaml.Decoder`, enables `KnownFields`, and calls `Decode` only
|
||
once. `yaml.Decoder.Decode` reads the next YAML document, so input such as a
|
||
valid version 4 configuration followed by `---` and another configuration is
|
||
accepted with the second document ignored. The strict file tests in
|
||
`internal/core/config/file_config_contract_test.go` cover malformed values,
|
||
unknown fields, and duplicate normalized identifiers, but not an additional
|
||
document.
|
||
- **Impact:** An operator can append a syntactically valid configuration
|
||
document, receive a successful validation result, and then run with only the
|
||
first document. Settings in the ignored document—including operational or
|
||
pipeline settings—have no effect without a diagnostic, contradicting the
|
||
documented strict single-file model.
|
||
- **Recommendation:** After decoding `FileConfig`, decode once more and require
|
||
`io.EOF`; reject any second document, including an empty or malformed one,
|
||
with a contextual configuration error.
|
||
- **Preserve:** Keep version gating before the full strict decode, unknown-field
|
||
and duplicate-key rejection, and the existing defaults → file → environment
|
||
precedence unchanged.
|
||
- **Validation:** Add focused parser cases for a second valid document, a
|
||
second malformed document, and ordinary trailing whitespace/comments; run
|
||
`go test ./internal/core/config ./internal/cli`.
|
||
- **Grouping:** Independent.
|
||
|
||
### CFGCLI-002 — Reject a blank command-level LLM profile
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Correctness
|
||
- **Evidence:** `internal/cli/run.go:149`–`166` registers `--llm-profile` as a
|
||
plain string flag, while the command's presence-aware empty-value checks
|
||
cover session ID, reasoning effort, output/debug directories, and recompute
|
||
step but not this flag. `runPipelineCommand` passes the resulting string to
|
||
`Config.Resolve`; `internal/framework/pipeline/profile.go:1255`–`1277` trims
|
||
an empty override and treats it as absent. The run contract tests cover a
|
||
valid override and an unknown non-empty profile, but not an explicitly
|
||
supplied blank value.
|
||
- **Impact:** A shell expansion such as `--llm-profile "$PROFILE"` with an
|
||
unset or blank value succeeds by silently using binding, pipeline, or
|
||
PromptKit defaults. The run can therefore use a different model/profile than
|
||
the operator explicitly intended to select.
|
||
- **Recommendation:** Make the CLI flag presence-aware and reject an explicitly
|
||
supplied empty or whitespace-only profile ID as command syntax before config
|
||
loading or physical-state allocation.
|
||
- **Preserve:** Keep a genuinely omitted override optional, trim non-empty IDs,
|
||
retain command → binding → pipeline → PromptKit precedence, and continue to
|
||
apply overrides only to selected LLM-backed bindings and validators.
|
||
- **Validation:** Add command-contract cases for blank and whitespace-only
|
||
values that assert exit status 2 and no state allocation, plus retain the
|
||
valid and unknown-profile run cases; run `go test ./internal/cli`.
|
||
- **Grouping:** Independent.
|
||
|
||
### Pipeline Resolution, Preparation, And Typed Registries
|
||
|
||
### PIPE-001 — Reject normalized module-reference collisions in the resolver
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Correctness
|
||
- **Evidence:** `internal/framework/pipeline/profile.go:1239`–`1252` sends every
|
||
module binding's reference map through `normalizeReferenceMap`. That helper,
|
||
at lines 1361–1377, trims each key but overwrites
|
||
`rawByNormalized[trimmedKey]` without checking whether another raw key already
|
||
produced the same identity. A programmatic binding containing both `slot`
|
||
and ` slot ` therefore retains whichever raw key is visited last by Go's map
|
||
iteration, then silently emits only one resolved binding. The later strict
|
||
reference resolver sees only the collapsed map and cannot diagnose the
|
||
collision. `internal/core/config/validation.go:256`–`266` correctly rejects
|
||
this shape for validated file/config flows, but direct `ResolvePipeline`
|
||
callers do not pass through that owner and there is no focused framework
|
||
regression case.
|
||
- **Impact:** A programmatically assembled profile can resolve successfully to
|
||
different external paths or generated selectors across processes from the
|
||
same ambiguous input. The normal CLI configuration path is protected by
|
||
upstream validation, which limits current production exposure, but the
|
||
resolver's own contract is nondeterministic.
|
||
- **Recommendation:** Make binding reference normalization return an error for
|
||
empty or duplicate trimmed keys before constructing the normalized map, and
|
||
propagate stage/lane context through `resolveBinding` callers. Avoid relying
|
||
on the config layer to make the framework resolver deterministic.
|
||
- **Preserve:** Keep whitespace normalization, exact external-versus-generated
|
||
source validation, sorted resolved bindings, local-over-pipeline precedence,
|
||
and the config layer's earlier contextual diagnostics.
|
||
- **Validation:** Add focused `ResolvePipeline` cases for whitespace-equivalent
|
||
chunk, extract, merge, and normalize reference keys, including different
|
||
source forms, and assert deterministic contextual rejection; run
|
||
`go test ./internal/framework/pipeline ./internal/core/config`.
|
||
- **Grouping:** Independent.
|
||
|
||
### PIPE-002 — Clone construction inputs once per builder boundary
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Efficiency
|
||
- **Evidence:** `Prepare` and `prepareLane` clone binding option maps while
|
||
forming requests (`internal/framework/pipeline/prepare.go:102`–`104` and
|
||
202–206), and `prepareValidatorChain` does the same at lines 258–263.
|
||
Registry/build boundaries then clone the complete request again. Typed
|
||
extractors, mergers, normalizers, and validators add another clone inside
|
||
their registered erased-builder adapters
|
||
(`extractor_registry.go:56`, `merger_registry.go:68`–`74`,
|
||
`normalizer_registry.go:63`–`69`, and `validator_registry.go:101`–`108`),
|
||
after `buildErasedModule` or `buildPreparedValidator` already called
|
||
`cloneBuildRequest` (`prepare.go:272`–`299` and 325–327). Each request clone
|
||
deep-copies materialized reference content as well as options, so typed
|
||
builders receive two reference copies and as many as three option copies;
|
||
untyped stage and validator builders use fewer copies.
|
||
- **Impact:** Every preparation repeats allocation and byte copying for bounded
|
||
external references and nested options, with the highest cost and a
|
||
different ownership path specifically for typed lanes and validators. The
|
||
work is run-construction-time rather than a concurrent operation hot path,
|
||
so the issue is low severity.
|
||
- **Recommendation:** Designate one private construction invocation as the
|
||
ownership boundary and clone the complete `BuildRequest` exactly there.
|
||
Store raw builders or remove the caller-side clone consistently so all stage
|
||
and validator registry variants follow the same single-copy rule.
|
||
- **Preserve:** Builders must continue to receive independently owned options,
|
||
reference maps, slot slices, metadata, and content bytes; preparation must
|
||
retain its own immutable resolved/reference state; nil, key/name, execution
|
||
class, and exact artifact-type checks must remain contextual errors.
|
||
- **Validation:** Extend construction hooks to mutate nested options and
|
||
reference bytes for typed and untyped modules/validators, assert no aliasing
|
||
with resolved or sibling requests, and use allocation/byte-copy observations
|
||
or a focused benchmark to confirm a single defensive copy; run
|
||
`go test ./internal/framework/contracts ./internal/framework/pipeline`.
|
||
- **Grouping:** Independent.
|
||
|
||
### References And Ordered Handoffs
|
||
|
||
### REF-001 — Bound reference reads before allocating the file
|
||
|
||
- **Severity:** Medium
|
||
- **Category:** Efficiency
|
||
- **Evidence:** `internal/framework/pipeline/references.go:88`–`164` implements
|
||
the external-reference materialization boundary. At lines 128–146,
|
||
`materializeReferenceTarget` calls `os.ReadFile(path)` before comparing the
|
||
resulting allocation with `ReferenceSlot.MaxBytes`. The focused
|
||
`TestMaterializeReferencesEnforcesMaxBytes` case uses a nine-byte file and
|
||
verifies the post-read diagnostic, but does not prove that reads are bounded
|
||
by the declared three-byte limit.
|
||
- **Impact:** A mistakenly selected very large file, growing file, device, or
|
||
named pipe can consume memory far beyond the slot's advertised bound before
|
||
the framework rejects it. CLI reference overrides expose the same path, so a
|
||
local operator error can terminate the process instead of producing the
|
||
intended bounded validation failure.
|
||
- **Recommendation:** Open the path and read through a limit of
|
||
`MaxBytes + 1` when a positive maximum is declared, rejecting an extra byte
|
||
before retaining or cloning content. A regular-file size precheck may improve
|
||
diagnostics, but the bounded reader must remain authoritative for changing or
|
||
non-regular inputs. Preserve the existing unbounded behavior only for slots
|
||
that explicitly declare no maximum.
|
||
- **Preserve:** Keep config-relative versus working-directory-relative path
|
||
resolution, UTF-8 and media-type validation, empty-file warnings, canonical
|
||
digest/size/origin metadata, contextual errors without content, and owned
|
||
reference bytes.
|
||
- **Validation:** Add a reader or file fixture that proves no more than
|
||
`MaxBytes + 1` bytes are consumed, including a non-regular or growing-input
|
||
case, while retaining the current UTF-8, media, empty, and ordinary oversize
|
||
diagnostics; run `go test ./internal/framework/pipeline ./internal/cli`.
|
||
- **Grouping:** Independent.
|
||
|
||
### REF-002 — Index accepted outputs once per ordered handoff
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Efficiency
|
||
- **Evidence:** `buildStepReferenceSets` walks every generated binding in the
|
||
receiving step (`internal/framework/pipeline/handoff.go:37`–`89`). For each
|
||
previously unseen producer, `generatedReferenceItem` allocates a `matches`
|
||
slice and scans the complete cumulative `outputs` slice to find that
|
||
step/lane (`handoff.go:104`–`133`). Its cache avoids rescanning when several
|
||
targets fan out from the same producer, but a step consuming `P` distinct
|
||
producers from `O` earlier outputs still performs `P * O` comparisons and up
|
||
to `P` temporary allocations.
|
||
- **Impact:** Ordered pipelines with many distinct generated dependencies pay
|
||
quadratic handoff preparation work before the consumer step can start. The
|
||
current production profiles are small and the scan is outside the lane
|
||
worker hot path, which limits present impact.
|
||
- **Recommendation:** Build one map from normalized `(step ID, lane ID)` to an
|
||
explicit zero/one/many accepted-output state at the start of
|
||
`buildStepReferenceSets`, then let `generatedReferenceItem` perform a direct
|
||
lookup. Keep ambiguity as data in the index so duplicate producer outputs are
|
||
still rejected rather than overwritten.
|
||
- **Preserve:** Retain exact accepted-output cardinality, codec
|
||
decode/re-encode canonicalization, producer lookup, complete schema/media
|
||
validation, per-target byte ownership, deterministic contextual errors, and
|
||
the rule that no consumer lane starts after a failed handoff.
|
||
- **Validation:** Add a many-producer/fanout case that retains missing and
|
||
duplicate rejection, then use a focused benchmark or comparison counter to
|
||
demonstrate one output-index pass plus direct producer lookups; run
|
||
`go test ./internal/framework/pipeline`.
|
||
- **Grouping:** Independent; do not combine with PIPE-002, which concerns
|
||
construction-request copying rather than runtime producer lookup.
|
||
|
||
### REF-003 — Include canonical size in generated-reference fingerprints
|
||
|
||
- **Severity:** Low
|
||
- **Category:** Correctness
|
||
- **Evidence:** The generated `ReferenceItem` records canonical content length
|
||
in `SizeBytes` (`internal/framework/pipeline/handoff.go:154`–`177`), and the
|
||
manifest provenance also retains that value at lines 210–232. However,
|
||
`generatedReferenceFingerprintIdentity` and
|
||
`generatedReferenceDependencies` (`handoff.go:234`–`287`) hash producer,
|
||
kind, complete schema identity, media type, and content digest without the
|
||
canonical size. This differs from the documented resume contract in
|
||
`docs/internal/state.md:49`–`54`. The focused fingerprint test changes only
|
||
canonical content and does not assert size participation.
|
||
- **Impact:** Consumer checkpoint identity does not cover one field that the
|
||
handoff and manifest declare part of canonical reference identity. Current
|
||
construction derives size directly from canonical bytes, so a practical
|
||
stale reuse also requires malformed internal metadata, a digest collision,
|
||
or a future producer-path change; the immediate risk is therefore low.
|
||
- **Recommendation:** Add `SizeBytes` to the private fingerprint identity and
|
||
populate it from the canonical generated item before JSON hashing. Keep the
|
||
existing normalized fingerprint name and all current identity fields.
|
||
- **Preserve:** Do not weaken the content digest, producer provenance,
|
||
artifact kind, complete schema digest/fields, media type, or canonical codec
|
||
checks. Continue to keep content bytes out of checkpoints, manifests, debug
|
||
summaries, and errors.
|
||
- **Validation:** Add a direct dependency-fingerprint case that holds the
|
||
other identity fields constant while changing size metadata, retain the
|
||
canonical-content sensitivity case, and run
|
||
`go test ./internal/framework/pipeline ./internal/modules/integration/...`.
|
||
- **Grouping:** Independent.
|
||
|
||
<!--
|
||
Finding template for later audit stages:
|
||
|
||
### AREA-001 — Concise title
|
||
|
||
- **Severity:** High, Medium, or Low
|
||
- **Category:** Correctness, Efficiency, Duplication, Simplicity, Test Quality,
|
||
or Documentation/Comments
|
||
- **Evidence:** Exact files, symbols, call paths, and observed behavior
|
||
- **Impact:** The concrete risk or cost
|
||
- **Recommendation:** A bounded implementation direction
|
||
- **Preserve:** Invariants and contracts that remediation must retain
|
||
- **Validation:** Focused checks that would demonstrate success
|
||
- **Grouping:** Independent, or the IDs with which this should be implemented
|
||
-->
|
||
|
||
## Intentional Complexity And Duplication To Preserve
|
||
|
||
- `internal/modules/generic/register.Register`,
|
||
`internal/modules/seriatim/register.Register`, and
|
||
`internal/modules/dnd/register.Register` deliberately expose the same small
|
||
registrar shape while retaining family-local registration policy and
|
||
diagnostics. Combining them would move extension ownership out of the domain
|
||
registrars and weaken the composition boundary established by ADR-0004.
|
||
- `internal/modules/dnd/register.registerModules`, `registerEvidence`,
|
||
`registerValidators`, and `registerDefaultChains` use explicit typed
|
||
registration lists. At this architectural pass, that repetition preserves
|
||
artifact Go types, module-specific validator order, and registrar-owned
|
||
production policy. Later D&D stages may evaluate individual shared
|
||
mechanics, but should not replace these lists with a dynamically typed
|
||
registration engine.
|
||
- `internal/framework/pipeline.RegisterArtifactCodec` and
|
||
`exactTypedValue` perform apparently repetitive exact-type checks around
|
||
private erasure. The checks deliberately turn incompatible values into
|
||
errors at each erased boundary rather than permitting a panic or accepting a
|
||
near-matching type, preserving ADR-0003.
|
||
- `internal/cli.runPipelineCommand` is a large linear orchestrator, but its
|
||
ordering is policy: syntax and config rejection precede run identity and
|
||
debug allocation; resolution and profile inspection precede module
|
||
preparation and input parsing; framework success precedes durable output;
|
||
and terminal debug publication precedes the optional JSON receipt. Existing
|
||
helpers isolate reference selection, recomputation, stores, output, result
|
||
encoding, and terminal error precedence. A generic lifecycle abstraction
|
||
would hide physical-state allocation and publication boundaries; bounded
|
||
parsing fixes such as CFGCLI-002 should not reorganize that lifecycle.
|
||
- `internal/core/config.validatePipelineProfiles` explicitly walks pipelines,
|
||
ordered steps, lanes, bindings, and references. Its nested structure mirrors
|
||
the public configuration shape and retains the nearest pipeline/step/lane
|
||
context in errors. Replacing it with a reflection-driven validator would
|
||
weaken those diagnostics and the presence-aware file-model boundary.
|
||
- `internal/cli.selectedReferenceTargets` and `recomputePolicy` perform
|
||
explicit resolved-shape traversals for distinct CLI policies: disambiguating
|
||
reference selectors against selected module capabilities, and computing the
|
||
forward forced/backward reusable checkpoint closure. Keeping these typed
|
||
traversals separate avoids adding command syntax or checkpoint policy to the
|
||
framework resolver.
|
||
- `pipeline.ResolvePipeline` and `resolveArtifactLane` are long, but their
|
||
linear sections retain the authoritative composition order: normalize
|
||
identity, select lanes, prove capabilities and exact artifact variants,
|
||
resolve stage-local references and validator chains, apply effective LLM
|
||
profiles, validate options, and only then compute the digest. Splitting these
|
||
checks into a generic stage engine would erase the different input, chunk,
|
||
typed-lane, validator, and output contracts. PIPE-001 is a bounded
|
||
normalization fix and should not reorganize this sequence.
|
||
- `pipeline.validateGeneratedBindings` has deeply nested traversal because it
|
||
proves a cross-step selector against ordered producer identity, the target's
|
||
declared slot, accepted artifact kinds, registered codec, and accepted media
|
||
types in one pass. Those checks are distinct static composition invariants;
|
||
materialized bytes, runtime handoff construction, and checkpoint hydration
|
||
remain separate runtime owners.
|
||
- The stage, validator, codec, and evidence registries intentionally use
|
||
private typed entries and small stage-specific lookup methods. Their
|
||
repetition preserves compile-time generic types until a narrow erased
|
||
closure, exact artifact-kind variant selection, and stage-specific
|
||
diagnostics. PIPE-002 concerns redundant request copies around those
|
||
closures, not the typed registry split itself.
|
||
- Generated-reference handoff deliberately decodes and re-encodes an accepted
|
||
normalized artifact through the registered producer codec even though fresh
|
||
outputs were already serialized. The same boundary also receives hydrated
|
||
checkpoint artifacts, so canonicalizing once per unique producer verifies
|
||
exact kind, schema, media type, and bytes before fanout. REF-002 removes only
|
||
repeated output discovery; it must not bypass this codec check.
|
||
- Operation paths clone reference sets for each module request while retaining
|
||
a separate pristine set for dependency fingerprints and validators. That
|
||
apparent duplication isolates module mutation across chunks and retries and
|
||
keeps validators on authoritative inputs. PIPE-002 applies only to adjacent
|
||
construction-builder copies and should not remove these operation-time
|
||
ownership boundaries.
|
||
|
||
## Areas Reviewed Without Findings
|
||
|
||
### Architecture And Dependency Boundaries
|
||
|
||
- **Composition root:** `internal/cli.newProductionComponents` constructs the
|
||
complete registry set and asset registry, then invokes only the generic,
|
||
Seriatim, and D&D family registrars. Direct production imports confirm that
|
||
`internal/cli` is the only layer importing those registrar packages.
|
||
- **Dependency direction:** A direct production import map found no core or
|
||
framework package importing `internal/modules`, no module importing
|
||
`internal/cli`, no concrete generic or Seriatim module importing D&D, and no
|
||
module importing the file-backed checkpoint, chunk-plan, debug, file-I/O, or
|
||
debug-bundle implementations. PromptKit is imported directly only by
|
||
`internal/framework/llm`.
|
||
- **Graph cross-layer calls:** The refreshed graph reported one
|
||
framework-to-module edge from `pipeline.Prepare` to a symbol named `request`
|
||
in a D&D validator test. Tracing it showed a confidence `0.06` suffix match
|
||
from the local closure call in `Prepare`; `trace_path` classified the target
|
||
as test-only, and the production import map disproved a dependency. The graph
|
||
reported no module-to-CLI calls.
|
||
- **Assets leaf:** `assets/package.go` imports only `embed` and `io/fs`, embeds
|
||
content, and exposes the read-only `FS() fs.FS` accessor. It contains no
|
||
business logic and has no `internal` or PromptKit dependency.
|
||
- **Fixed pipeline shape:** `pipeline.ResolvePipeline` resolves input and
|
||
chunk once, fixed extract/merge/normalize bindings per artifact lane, and one
|
||
output binding. Ordered steps are barriers around those fixed lanes rather
|
||
than arbitrary graph topology. `pipeline.Prepare`, `Runner.Run`,
|
||
`runPreparedSteps`, and `runLanes` retain that shape through construction and
|
||
execution.
|
||
- **Typed artifact boundary:** Typed registrations retain the exact Go type for
|
||
codecs and lane operations. Private erasure in `RegisterArtifactCodec` and
|
||
`exactTypedValue` verifies exact types and returns contextual errors;
|
||
normalized values cross into output through serialized artifacts.
|
||
- **Physical-state ownership:** The CLI owns root selection, store factories,
|
||
and durable file placement (`chunkPlanStoreForRun`, checkpoint/debug setup,
|
||
and `writeOutputFiles`). The framework receives collaborator interfaces and
|
||
returns logical output files. The generic JSON output module's direct import
|
||
of `internal/framework/chunkmap` validates and republishes the accepted
|
||
serialized chunk-map contract; it neither chooses a physical root nor writes
|
||
files.
|
||
- **Accepted architectural decisions:** ADRs 0001–0005 and 0007–0012 were read
|
||
against the current high-level composition. Apart from ARCH-001, the
|
||
composition root, fixed ordered pipeline, typed boundary, domain packaging,
|
||
canonical chunk-plan policy, separate state surfaces, checkpoint policy,
|
||
evidence rules, workload profile ownership, centralized asset leaf, and
|
||
deterministic entity identity boundary have corresponding current owners.
|
||
|
||
### Configuration And CLI Composition
|
||
|
||
- **End-to-end command path:** `RunWithOptions` normalizes injectable process
|
||
collaborators once and dispatches to `runPipelineCommand`. The run command
|
||
parses and normalizes command input, discovers and loads configuration,
|
||
applies command overrides, builds the effective catalog, resolves reference
|
||
selectors and the pipeline, inspects effective profiles, materializes
|
||
references, constructs runtime/state collaborators, invokes `pipeline.Run`,
|
||
publishes output files, terminalizes debug state, and only then publishes a
|
||
requested JSON receipt.
|
||
- **Precedence and resolution:** `loadConfig` enforces explicit `--config` over
|
||
`NOTARIUS_CONFIG` over the system default, then applies `Default`, file
|
||
configuration, supported environment overrides, and run-only CLI overrides
|
||
in order. `Config.Resolve` recomputes derived worker defaults, validates,
|
||
clones the selected profile, and delegates catalog-dependent composition to
|
||
the framework resolver. Apart from CFGCLI-001 and CFGCLI-002, unknown fields,
|
||
malformed values, normalized-key collisions, unknown command flags, and
|
||
invalid selected modules/options are rejected at their owning boundary.
|
||
- **Profile-source equality:** Validation-time
|
||
`validateExplicitPromptKitProfiles` and runtime
|
||
`buildProductionLLMClient` pass the same profile directory, profile file,
|
||
mapped local backend, and shared fallback asset registry. Effective profile
|
||
collection is sorted, deduplicated, and limited to selected LLM-backed
|
||
modules and validators, so inspection and runtime selection use the resolved
|
||
profile values rather than recomputing inheritance.
|
||
- **Session identity:** `resolvePromptSessionID` uses a versioned SHA-256 value
|
||
over the trimmed resolved input-module key, a separator, and exact raw input
|
||
bytes. It contains no pipeline ID, reference, profile, retry, input path,
|
||
working directory, or run ID; an explicit non-empty session replaces the
|
||
generated value. Run contracts verify the same effective session reaches all
|
||
prompt-facing requests, manifests, debug metadata, and checkpoint identity.
|
||
- **Reference and recomputation controls:** CLI reference selectors are
|
||
resolved only against selected chunk/extract/merge/normalize capabilities
|
||
before the authoritative effective resolution. Recompute policy is derived
|
||
after reference materialization, forces the requested step and transitive
|
||
consumers, and requires reusable checkpoints for non-forced transitive
|
||
producers. Focused contract tests exercise selector ambiguity, lane
|
||
selection, ordered dependency closure, and execution behavior.
|
||
- **Publication and terminal outcomes:** Syntax/config failures before run
|
||
identity allocate no output or debug state. After debug allocation,
|
||
resolution, profile, preparation, input, framework, partial-summary, and
|
||
output failures all pass through `failPipelineCommand`. `terminalize` writes
|
||
a run report once, preserves an existing primary failure over report/error-log
|
||
failures, promotes a success-report failure to primary, and reports other
|
||
persistence failures secondarily. Framework cancellation follows the same
|
||
wrapped primary-error path. Durable outputs are attempted only after runner
|
||
success; a JSON result is encoded before output publication but written to
|
||
stdout only after output and debug terminalization. A receipt-delivery
|
||
failure leaves already published bundles intact and returns failure.
|
||
- **Test ownership:** Configuration tests own strict file/env application,
|
||
structural validation, effective cloning/digests, and redaction. CLI command,
|
||
run, reference, recomputation, session, production, example, result, and state
|
||
contracts assert process-level ordering and side effects rather than merely
|
||
repeating lower-level resolver assertions. The two uncovered command/parser
|
||
cases are recorded as CFGCLI-001 and CFGCLI-002.
|
||
|
||
### Pipeline Resolution, Preparation, And Typed Registries
|
||
|
||
- **Static composition:** `ResolvePipeline` rejects mixed legacy/ordered
|
||
shapes, empty and duplicate normalized step/lane identities, invalid
|
||
invocation filtering, unknown modules, missing capabilities, incompatible
|
||
artifact variants, unsupported lane-level validators, invalid generated
|
||
selectors, and missing output capabilities before producing a resolved
|
||
value. Apart from PIPE-001's direct-call collision, selected lanes and steps
|
||
have stable sorted/order-preserving identities.
|
||
- **Effective policy and identity:** Execution classes come from normalized
|
||
registry specs. Command override → binding → pipeline LLM-profile precedence
|
||
applies only to selected LLM-backed modules and validators; deterministic
|
||
bindings reject explicit profiles. Module and validator option validators
|
||
receive owned maps before `resolvedPipelineDigest` hashes the complete
|
||
effective composition. Digest tests cover map canonicalization, validator
|
||
policy, artifact schema identity, effective profiles, and exclusion of the
|
||
digest field itself.
|
||
- **Typed registry boundary:** Extractor registrations retain one exact Go type;
|
||
merger, normalizer, and typed-validator registrations select an exact
|
||
module/artifact-kind variant; codecs validate complete schema/media identity;
|
||
and evidence projectors must match the active codec type. Every erased
|
||
operation checks the implementation or value type and returns an error rather
|
||
than asserting or panicking. Kind-neutral `Spec` methods are confined to
|
||
catalog inspection, while behavior-sensitive resolution uses exact variant
|
||
lookups.
|
||
- **Construction boundary:** `Prepare` validates the resolved shape and needed
|
||
registries, clones retained bindings, options, validator chains, reference
|
||
targets, schemas, and bytes, then constructs input, chunker, chunk validators,
|
||
every ordered typed lane and local validator chain, output, and the optional
|
||
evidence plan before returning. Implementations and operations remain
|
||
private; public prepared bindings/lanes are separate clones. Focused tests
|
||
verify deterministic construction order, late failure before input parsing,
|
||
nil and identity rejection, generated-selector retention, and independent
|
||
builder reference inputs. PIPE-002 records only the extra adjacent copies.
|
||
- **Checkpoint supplements:** Preparation collects component-provided semantic
|
||
fingerprints only after the complete implementation set exists. Scopes
|
||
include stage, globally unique lane identity, module, and validator position;
|
||
empty or duplicate values fail preparation, results are sorted, and the
|
||
accessor returns a defensive copy. Scheduling limits and diagnostics are not
|
||
included. Resolved composition—including options, reference selectors,
|
||
effective profiles, retries, and validator order—remains owned by the
|
||
resolved digest rather than being redundantly restated as component
|
||
fingerprints.
|
||
- **Registry comparison:** Input, chunker, and output registries consistently
|
||
normalize keys/specs, reject nil validators/builders, validate options on
|
||
owned maps, clone stored specs, sort discovery output, and verify constructed
|
||
identity. Typed stage and validator registries add only the exact-type and
|
||
artifact-variant mechanics their contracts require. Validator-chain lookup
|
||
distinguishes absent, default, explicit replacement, and explicit empty
|
||
chains while returning defensive copies. No dead compatibility path or safe
|
||
consolidation was found beyond the copy reduction in PIPE-002.
|
||
- **Test ownership:** Contract tests cover clone/serialization boundaries;
|
||
registry tests cover invalid registration, sorted/defensive discovery,
|
||
strict options, exact types, schema compatibility, and evidence ownership;
|
||
resolution tests cover heterogeneous variants and effective identity; and
|
||
preparation tests own all-before-parse construction and fingerprint
|
||
collection. The missing module-reference collision case is recorded in
|
||
PIPE-001 rather than as a separate test-only finding.
|
||
|
||
### References And Ordered Handoffs
|
||
|
||
- **External path:** Pipeline defaults are filtered to declared slots; local
|
||
bindings override eligible defaults; CLI overrides apply to one exact final
|
||
target; and unbind removes only an external binding before required-slot
|
||
enforcement. Selected legacy lanes determine eligible targets, while
|
||
explicit ordered steps reject `--only`. Config-relative and CLI-relative
|
||
paths remain distinct, materialized values retain digest/media/size/origin,
|
||
preparation supplies owned construction inputs, and operation requests get
|
||
independent content. REF-001 records the only missing bound in this path.
|
||
- **Generated path:** Pipeline-level selectors are rejected. Ordered local or
|
||
step bindings must name a declared earlier step and selected lane whose exact
|
||
artifact kind, registered codec media type, and target slot constraints are
|
||
compatible. Because all edges point strictly backward, forward references
|
||
and cycles are rejected during resolution. At the step barrier, exactly one
|
||
accepted normalized output is decoded and re-encoded through its codec;
|
||
missing, duplicate, wrong-kind, wrong-schema/media, oversized, rejected, or
|
||
unavailable producer state stops the consumer before execution. REF-002
|
||
concerns only the repeated lookup used to establish that cardinality.
|
||
- **Resume and recomputation:** Ordinary resume progressively compares
|
||
generated dependency fingerprints on extract, merge, and normalize
|
||
checkpoints. Selective recomputation instead forces the selected step and
|
||
transitive consumers while requiring unforced transitive producers to supply
|
||
accepted normalize state without extract/merge files or dependency matches.
|
||
Hydration validates stored provenance and canonical bytes through the active
|
||
codec before publishing an owned normalized output. REF-003 records the
|
||
omitted size field in the ordinary consumer fingerprint.
|
||
- **Provenance and evidence:** External manifests contain paths, digests,
|
||
media, sizes, and binding sources; generated manifests contain bounded
|
||
producer, codec/schema, digest, and size identity without content or a fake
|
||
path. Runtime stage debug envelopes omit reference sets, contract JSON omits
|
||
`ReferenceItem.Content`, and focused assembled-module tests confirm generated
|
||
references ground operations without becoming source evidence.
|
||
- **Complexity ownership:** Static target resolution keeps precedence,
|
||
unbinding, and required-slot policy together for contextual errors;
|
||
`validateGeneratedBindings` owns cross-step static compatibility; and
|
||
`buildStepReferenceSets` owns the runtime barrier. Apart from REF-002's
|
||
repeated full-output scan, their explicit traversals preserve distinct
|
||
invariants more clearly than a generic graph or reflection engine.
|
||
- **Test ownership:** Profile and CLI reference contracts cover defaults,
|
||
local precedence, unbinding, required slots, selected targets, ambiguity,
|
||
and typed producer compatibility. Reference materialization tests cover
|
||
origin, UTF-8, media, size diagnostics, warnings, and provenance; handoff and
|
||
runner tests cover canonical fanout, invalid/missing producers, ordering,
|
||
fingerprints, and accepted checkpoint hydration; assembled integration tests
|
||
cover semantic generated-reference consumers. REF-001 and REF-003 identify
|
||
the two unproved identity/resource details.
|
||
|
||
## Validation Record
|
||
|
||
| Date | Scope | Command or check | Result |
|
||
| --- | --- | --- | --- |
|
||
| 2026-08-08 | Initial worktree | `git status --short` | Pass; no output |
|
||
| 2026-08-08 | Knowledge graph | Full index as `notarius-audit-92e8907` | Pass; 8,322 nodes and 47,887 edges; branch/head matched the production target |
|
||
| 2026-08-08 | Baseline tests | `go test ./...` | Pass |
|
||
| 2026-08-08 | Baseline static analysis | `go vet ./...` | Pass |
|
||
| 2026-08-08 | Baseline build | `go build ./cmd/notarius` | Pass |
|
||
| 2026-08-08 | Baseline whitespace | `git diff --check` | Pass |
|
||
| 2026-08-08 | Production imports | Direct `go list` import-edge audit plus graph call tracing | Pass; no production dependency inversion found |
|
||
| 2026-08-08 | Accepted ADR links | Relative Markdown-link target scan under `docs/adr/` | Two unresolved targets recorded as ARCH-001 |
|
||
| 2026-08-08 | Audit target integrity before configuration/CLI review | `git diff --quiet 92e89076a268089e703978fb9d7176200e93344c..HEAD -- . ':(exclude)docs/roadmap/**'` | Pass; production target unchanged |
|
||
| 2026-08-08 | YAML decoder contract | `go doc gopkg.in/yaml.v3.Decoder.Decode` and `ParseFileConfigYAML` call trace | Decode consumes the next document; no EOF/second-document check, recorded as CFGCLI-001 |
|
||
| 2026-08-08 | Focused configuration and CLI tests | `go test ./internal/core/config ./internal/cli` | Pass |
|
||
| 2026-08-08 | Focused configuration and CLI static analysis | `go vet ./internal/core/config ./internal/cli` | Pass |
|
||
| 2026-08-08 | Audit target integrity before pipeline composition review | `git diff --quiet 92e89076a268089e703978fb9d7176200e93344c..HEAD -- . ':(exclude)docs/roadmap/**'` | Pass; production target unchanged |
|
||
| 2026-08-08 | Pipeline graph review | Architecture, complexity query, exact symbol reads, and call traces for `ResolvePipeline`, binding normalization, typed registries, `Prepare`, and checkpoint fingerprints | Pass; PIPE-001 and PIPE-002 recorded; generated handoff execution deferred to the next area |
|
||
| 2026-08-08 | Focused contracts and pipeline tests | `go test ./internal/framework/contracts ./internal/framework/pipeline` | Pass |
|
||
| 2026-08-08 | Focused contracts and pipeline static analysis | `go vet ./internal/framework/contracts ./internal/framework/pipeline` | Pass |
|
||
| 2026-08-08 | Audit target integrity before references/handoffs review | `git diff --quiet 92e89076a268089e703978fb9d7176200e93344c..HEAD -- . ':(exclude)docs/roadmap/**'` | Pass; production target unchanged |
|
||
| 2026-08-08 | Reference and handoff graph review | Architecture, exact symbol reads, and call traces across external materialization, target resolution, operation cloning, generated codec handoff, consumer fingerprints, ordered execution, and accepted-checkpoint hydration | REF-001, REF-002, and REF-003 recorded; no content/evidence leak found |
|
||
| 2026-08-08 | Focused pipeline and CLI tests | `go test ./internal/framework/pipeline ./internal/cli` | Pass |
|
||
| 2026-08-08 | Assembled integration tests | `go test ./internal/modules/integration/...` | Pass |
|
||
|
||
## Coverage Matrix
|
||
|
||
| Audit area | Status | Packages and documents inspected | Validation run | Finding IDs |
|
||
| --- | --- | --- | --- | --- |
|
||
| Architecture and dependency boundaries | Reviewed | Architecture, documentation, and testing policies; internal overview; accepted ADRs; `internal/cli/catalog.go`; production registrars; root `assets` package; representative pipeline, typed-codec, output, and state-owner symbols | Full baseline, fresh graph, direct import map, cross-layer call traces, ADR link scan | ARCH-001 |
|
||
| Configuration and CLI composition | Reviewed | `docs/config.md`, `docs/cli.md`, `docs/operations.md`, internal configuration/CLI docs; `internal/core/config/`; CLI run, catalog, session, profile, result, and terminal owners; focused config, command, run, reference, recomputation, session, production, example, result, and state tests | Target-integrity check, graph call/data-owner traces, YAML decoder contract, focused tests and vet | CFGCLI-001, CFGCLI-002 |
|
||
| Pipeline resolution, preparation, and typed registries | Reviewed | Internal pipeline/module docs; `internal/framework/contracts/`; pipeline profile, options, module, construction, preparation, fingerprint, stage/validator/chain/codec/evidence registry implementations and focused tests | Target-integrity check, graph architecture/complexity/call traces, focused tests and vet | PIPE-001, PIPE-002 |
|
||
| References and ordered handoffs | Reviewed | Reference and ordered-step sections of configuration, internal pipeline, and state docs; pipeline reference resolution/materialization, preparation ownership, generated handoff, consumer fingerprint, checkpoint hydration, and runner barriers; CLI selector/recomputation owners; focused profile, reference, handoff, checkpoint, recomputation, and assembled integration tests | Target-integrity check, graph architecture/complexity/call traces, focused pipeline/CLI tests, assembled integration tests | REF-001, REF-002, REF-003 |
|
||
| Execution, validation, retry, and concurrency | Pending | — | — | — |
|
||
| State, checkpoints, debugging, and file safety | Pending | — | — | — |
|
||
| LLM runtime, prompt filesystems, and assets | Pending | — | — | — |
|
||
| Generic and Seriatim modules | Pending | — | — | — |
|
||
| Shared D&D types, codecs, and family mechanics | Pending | — | — | — |
|
||
| NPC, item, and location registries | Pending | — | — | — |
|
||
| NPC, item, and location occurrences | Pending | — | — | — |
|
||
| Spells, scene chunking, and scene descriptions | Pending | — | — | — |
|
||
| Combat turns and enemy events | Pending | — | — | — |
|
||
| Test ownership, comments, and final synthesis | Pending | — | — | — |
|