1229 lines
76 KiB
Markdown
1229 lines
76 KiB
Markdown
# Codebase Audit
|
|
|
|
## Sequence Prerequisite
|
|
|
|
Stage 0 had not been executed when this artifact was created: the repository
|
|
contained no `audit.md` at the start of the Stage 1 review. Consequently, the
|
|
reproducible repository baseline, package inventory, validation matrix, graph
|
|
refresh, and initial coverage ledger required by Stage 0 remain outstanding.
|
|
This review did not backfill that out-of-scope work.
|
|
|
|
The Stage 1 review began from commit
|
|
`ebf1602635e108e2a7ac1abd3a3ca24a620104ce` on branch `main`, with a clean
|
|
working tree and Go `go1.26.5 linux/amd64`. These details identify this review
|
|
only; they are not a substitute for the Stage 0 baseline.
|
|
|
|
## Stage 1: Public Values, Conversion, Errors, And Formatting
|
|
|
|
### Scope Reviewed
|
|
|
|
The review covered `doc.go`, `types.go`, `convert.go`, `errors.go`,
|
|
`capacity_error.go`, `formatting.go`, and `prepared_execution.go`, plus the
|
|
directly relevant root tests. Internal domain declarations and callers were
|
|
consulted only to confirm field-complete conversion, ownership, and public
|
|
error mapping. Engine assembly, runtime orchestration, adapter implementation,
|
|
and JSON codec mechanics were not audited.
|
|
|
|
### Accepted Findings
|
|
|
|
#### S01-F01: Run-request formatting tests do not protect input and variable redaction
|
|
|
|
- **Category:** testing
|
|
- **Severity:** medium
|
|
- **Confidence:** high
|
|
- **Status:** accepted
|
|
- **Affected code:** `formatting.go` (`RunRequest.String`,
|
|
`RunRequest.GoString`, and `RunRequest.redactedString`) and `engine_test.go`
|
|
(`TestRunRequestFormattingRedactsDirectAPIKey`)
|
|
- **Contract at issue:** The formatter GoDoc promises that `String` and
|
|
`GoString` omit direct credentials and input and variable contents. The
|
|
documentation and testing policies treat prompt inputs and other private
|
|
content as sensitive and give consequential disclosure behavior a strong
|
|
presumption of durable test protection.
|
|
- **Evidence:** The implementation currently satisfies the contract by
|
|
formatting only request identifiers, collection lengths, presence flags,
|
|
and whether an API key is set. The focused test supplies an inline input but
|
|
asserts only that the API-key sentinel is absent and `APIKeySet:true` is
|
|
present; it supplies no variable values and never checks whether the input
|
|
URI, input body, or variable values appear. The test would remain green if a
|
|
later formatter change appended input or variable contents while continuing
|
|
to omit the API key.
|
|
- **Failure mode:** A logging or diagnostic-formatting refactor could disclose
|
|
prompt input or template-variable content through ordinary `%v`, `%+v`, or
|
|
`%#v` formatting without a contract-test failure.
|
|
- **Recommended direction:** Extend the existing formatting test, rather than
|
|
adding a parallel test, with distinct input URI, input body, and variable
|
|
sentinels and assert that each is absent from all three supported formatting
|
|
forms. Retain the positive structural assertions so the test continues to
|
|
distinguish a useful summary from an empty formatter.
|
|
- **Required verification:** Run the focused request-formatting test and the
|
|
root package tests. Confirm that a deliberate formatter mutation exposing
|
|
any sentinel makes the focused test fail.
|
|
|
|
### Unresolved Observations
|
|
|
|
None. Medium- or low-confidence concerns discovered during this review were
|
|
not promoted to findings.
|
|
|
|
### Coverage Ledger
|
|
|
|
- **Public value declarations and zero values:** Reviewed request, prepared,
|
|
result, artifact, inspection, target, output, validation, rendered-prompt,
|
|
structured-output, generation, and token-usage values. Nil maps, slices,
|
|
pointers, optional values, and zero-value public enum strings cross the
|
|
facade without panics or invented values.
|
|
- **Request conversion:** `toDomainRunRequest` and its helpers preserve every
|
|
public field, copy maps and pointer values, validate and deeply copy nested
|
|
JSON-compatible overrides, and retain direct credentials only in the
|
|
internal request field intended for execution.
|
|
- **Prepared and result conversion:** `fromDomainPreparedRun`,
|
|
`fromDomainRunResult`, and their helpers preserve all public fields while
|
|
copying artifact bytes, validation diagnostics, hashes, rendered messages,
|
|
cache-control pointers, effective extra parameters, and structured-output
|
|
schemas. Internal credential and target-presence fields do not escape.
|
|
- **Inspection and extension conversion:** Profile and prompt inspection
|
|
outputs are independent copies. Generation requests receive copied prompt,
|
|
target, presence, and structured-output values; generation responses contain
|
|
no mutable fields requiring additional copying.
|
|
- **Copy-rule ownership:** Caller-supplied JSON-compatible values enter through
|
|
`internal/jsonvalue` validation and copying. The outward conversion helpers
|
|
copy already-validated domain snapshots without introducing a second
|
|
acceptance policy. No consolidation finding was warranted in this scope.
|
|
- **Public errors:** Not-found identities remain distinct from load failures;
|
|
profile-required and missing-credential errors retain their more specific
|
|
identity together with `ErrInvalidRequest`; collaborator and cancellation
|
|
identities remain discoverable; and typed capacity errors expose only a
|
|
copied backend ID plus `ErrCapacityExceeded` rather than the internal error
|
|
type. Nil and zero `CapacityError` values are safe.
|
|
- **Diagnostic formatting:** `RunRequest` and `GenerateRequest` currently omit
|
|
direct credentials and content from `String`, `GoString`, `%+v`, and `%#v`.
|
|
`PreparedExecution` always formats as an opaque constant, including through
|
|
a copied handle. Prepared and result values intentionally expose rendered or
|
|
generated content as documented in package GoDoc; applications retain
|
|
responsibility for logging those content-bearing values.
|
|
- **Prepared handle values:** Nil and zero handles return zero details and may
|
|
be discarded safely. Details are fresh deep copies and remain stable after
|
|
execution or discard. Copying a handle shares its single-use lifecycle
|
|
without exposing the internal representation.
|
|
- **Test ownership:** Root external-package tests appropriately own public
|
|
snapshots, structured error identity, and opaque-handle behavior. Focused
|
|
internal error-mapping tests cover internal-type containment. The one
|
|
material redaction gap is recorded as S01-F01.
|
|
|
|
### Verification Performed
|
|
|
|
The code knowledge graph was used to discover the scoped symbols, trace their
|
|
callers and callees through the facade and internal domain boundary, and locate
|
|
the focused tests. Important conclusions were confirmed against source.
|
|
|
|
The following focused commands passed:
|
|
|
|
```sh
|
|
go test . -run 'Test(PreparedRunJSONDoesNotExposeSecretOrTargetPresence|RunRequestFormattingRedactsDirectAPIKey|GenerateRequestFormattingRedactsDirectAPIKey|MapPublicErrorPreservesGenerationCancellation|MapPublicErrorTranslatesCapacityError|CapacityExceededSentinelContract|InspectProfileReturnsIndependentTargetMatchingPreparation|InspectPromptReturnsIndependentMetadataMatchingPreparation|PreparedExecutionFreezesSourcesAndReturnsIndependentDetails|PreparedExecutionLifecycleAndEngineBinding|PreparedExecutionDiscardAndFormattingDoNotExposePrivateState|InMemoryProfileExtraParamsAreCopiedAcrossPublicBoundary|ExtraParamsTypedNestedValuesAreCopiedAcrossPublicBoundary|ArtifactReaderReceivesPublicReferenceAndPreparesArtifact|RunPassesPreparedRequestToInjectedLLMClient|PublicErrorsSupportErrorsIs)$'
|
|
go test -race . -run 'Test(PreparedExecutionFreezesSourcesAndReturnsIndependentDetails|PreparedExecutionConcurrentClaimAllowsOneGeneration|PreparedExecutionRunAndDiscardRaceHasOneWinner|PreparedExecutionDiscardAndFormattingDoNotExposePrivateState|MapPublicErrorTranslatesCapacityError|RunRequestFormattingRedactsDirectAPIKey|GenerateRequestFormattingRedactsDirectAPIKey)$' -count=3
|
|
```
|
|
|
|
### Handoff
|
|
|
|
- Execute the missing Stage 0 baseline before relying on this file as a
|
|
complete audit ledger or beginning the next component review.
|
|
- Stage 2 owns public configuration helpers, `json.go`, extension-adapter
|
|
implementation, and adapter-specific mutation and cancellation behavior.
|
|
- Stages 3 and 4 own engine construction and runtime operations respectively;
|
|
this review did not evaluate those paths beyond tracing their use of the
|
|
scoped conversion and error boundary.
|
|
|
|
## Stage 2: Public Configuration And Extension Adapters
|
|
|
|
### Scope Reviewed
|
|
|
|
The review covered `backends.go`, `profiles.go`, `artifact_reader.go`,
|
|
`json.go`, and `llm_adapter.go`, together with directly relevant root tests,
|
|
external-package contract tests, and `internal/profile` validation tests.
|
|
Internal backend, profile, artifact, and LLM declarations were consulted only
|
|
to compare boundary contracts and policy ownership. `NewEngine` option
|
|
assembly, source composition, runtime orchestration, internal profile-source
|
|
behavior, and transport mechanics were not audited.
|
|
|
|
### Accepted Findings
|
|
|
|
#### S02-F01: Out-of-range JSON durations silently overflow during decoding
|
|
|
|
- **Category:** correctness
|
|
- **Severity:** medium
|
|
- **Confidence:** confirmed
|
|
- **Status:** accepted
|
|
- **Affected code:** `json.go` (`RunResult.UnmarshalJSON` and
|
|
`runResultJSON.DurationMS`) and `public_contract_test.go`
|
|
(`TestRunResultJSONUsesMillisecondsAndRoundTrips`)
|
|
- **Contract at issue:** `RunResult` has a stable JSON representation in which
|
|
`duration_ms` is an integer millisecond count. Decoding must not silently
|
|
turn an accepted wire value into unrelated duration metadata.
|
|
- **Evidence:** The wire field accepts the full `int64` range, then decoding
|
|
multiplies that value by `time.Millisecond` without checking whether the
|
|
nanosecond-valued `time.Duration` can represent the result. A focused probe
|
|
decoded `{"duration_ms":9223372036854775807}` with a nil error and produced
|
|
`Duration == -1ms`. The existing round-trip test exercises only `1500ms` and
|
|
does not cover either representable boundaries or overflow.
|
|
- **Failure mode:** Malformed or untrusted persisted JSON can be accepted while
|
|
corrupting a very large positive duration into a negative or otherwise
|
|
wrapped value. Downstream timing displays, comparisons, or metrics then
|
|
consume false data without a decode error.
|
|
- **Recommended direction:** Validate the millisecond value against the range
|
|
that can be safely converted to `time.Duration` before multiplication and
|
|
return a contextual JSON decoding error for values outside that range.
|
|
- **Required verification:** Add boundary cases for the largest safely
|
|
representable positive and negative millisecond values and their first
|
|
out-of-range neighbors, plus the reproduced maximum-`int64` input. Retain
|
|
ordinary and zero-value round-trip coverage.
|
|
|
|
#### S02-F02: In-memory and filesystem profiles duplicate semantic validation
|
|
|
|
- **Category:** duplication
|
|
- **Severity:** medium
|
|
- **Confidence:** high
|
|
- **Status:** accepted
|
|
- **Affected code:** `profiles.go` (`validatePublicProfile`,
|
|
`toDomainProfile`, and the memory profile repository) and
|
|
`internal/profile/filesystem_repository.go` (`validateProfile` and
|
|
`loadProfile`), plus their focused tests
|
|
- **Contract at issue:** Profiles supplied in memory and profiles loaded from a
|
|
filesystem are two sources for the same execution-profile domain value.
|
|
Required identity, backend-or-endpoint selection, model presence, and
|
|
numeric bounds are one semantic acceptance policy and need one owner.
|
|
- **Evidence:** `validatePublicProfile` and `validateProfile` independently
|
|
implement the same seven conditions with the same error text: required ID,
|
|
backend or endpoint, required model, temperature in `[0,2]`, non-negative
|
|
maximum tokens, top-p in `[0,1]`, and non-negative timeout. The root tests do
|
|
not exercise the required-field or scalar-bound cases for in-memory
|
|
profiles, and the filesystem tests do not protect all scalar bounds. This is
|
|
policy duplication rather than mere translation or error wrapping.
|
|
- **Failure mode:** A future constraint or correction can be applied to one
|
|
profile source but not the other, making an otherwise identical profile
|
|
valid or invalid according to where it was stored. Sparse boundary tests
|
|
would not reliably expose the divergence.
|
|
- **Recommended direction:** Give the domain-level profile acceptance rule one
|
|
internal owner that both in-memory and filesystem repositories invoke,
|
|
while leaving source-specific normalization and public error translation at
|
|
their existing boundaries.
|
|
- **Required verification:** Protect the shared validator with a table covering
|
|
every required field and both sides of every numeric bound, then retain a
|
|
small integration check for each source and for the public
|
|
`ErrInvalidConfig` translation.
|
|
|
|
#### S02-F03: The injected LLM client's mutation-ownership contract is untested
|
|
|
|
- **Category:** testing
|
|
- **Severity:** medium
|
|
- **Confidence:** high
|
|
- **Status:** accepted
|
|
- **Affected code:** `llm_adapter.go` (`publicLLMClientAdapter.Generate`),
|
|
`convert.go` (`fromDomainGenerateRequest` and its nested conversions),
|
|
`types.go` (`LLMClient`), and injected-client tests in `engine_test.go`
|
|
- **Contract at issue:** The public `LLMClient` contract explicitly states that
|
|
maps, slices, and pointers in `GenerateRequest` are client-owned copies that
|
|
may be mutated or retained. The adapter is the boundary responsible for
|
|
satisfying that ownership promise.
|
|
- **Evidence:** The adapter currently constructs independent messages,
|
|
cache-control pointers, target parameters, and structured-output schema
|
|
values before invoking the client. Existing fakes retain requests and tests
|
|
inspect field propagation, errors, and cancellation, but no test mutates the
|
|
nested request received by the client and proves that the source domain
|
|
request remains unchanged. A shallow-copy regression would therefore
|
|
preserve all current field-equality assertions.
|
|
- **Failure mode:** A conforming injected client could mutate or asynchronously
|
|
retain nested request data and thereby alter prepared engine state, affect a
|
|
later operation, or introduce a race despite following the documented
|
|
interface contract.
|
|
- **Recommended direction:** Add a focused adapter-boundary ownership test that
|
|
has a client mutate and retain each mutable nested shape, then verifies that
|
|
the domain request and its nested values remain unchanged. Keep engine-level
|
|
tests focused on observable request propagation and error identity.
|
|
- **Required verification:** Exercise prompt messages and cache control, target
|
|
extra parameters, and structured-output schema under the focused test; run
|
|
it with the race detector as well as normally.
|
|
|
|
#### S02-F04: The profile convenience constructor's full mapping is unprotected
|
|
|
|
- **Category:** testing
|
|
- **Severity:** medium
|
|
- **Confidence:** high
|
|
- **Status:** accepted
|
|
- **Affected code:** `profiles.go` (`OpenAICompatibleProfile` and
|
|
`OpenAICompatibleProfileConfig`) and the three
|
|
`TestOpenAICompatibleProfile...` tests in `engine_test.go`
|
|
- **Contract at issue:** The exported convenience constructor promises a
|
|
`Profile` suitable for the general `WithProfiles` path. Its consumer-visible
|
|
behavior is the complete, field-for-field mapping of configuration values,
|
|
followed by the documented deferred validation and copying rules.
|
|
- **Evidence:** The implementation currently maps every configuration field.
|
|
The principal integration test asserts backend ID, model, direct API-key
|
|
behavior, and extra parameters, while the other tests cover deferred nested
|
|
parameter validation and ownership. No test protects endpoint,
|
|
temperature, maximum tokens, top-p, timeout, service tier, or reasoning
|
|
effort as constructor output. Dropping any of those assignments would leave
|
|
the current constructor-specific tests green.
|
|
- **Failure mode:** A maintenance edit can silently discard a supported model
|
|
setting from the convenience path while the equivalent general `Profile`
|
|
configuration continues to work, creating source-dependent behavior for
|
|
consumers.
|
|
- **Recommended direction:** Add one direct, table-like all-field mapping test
|
|
for the constructor and keep only the integration assertions that establish
|
|
its passage through ordinary profile validation and ownership boundaries.
|
|
- **Required verification:** Populate every scalar and string setting with a
|
|
distinct non-zero value, compare the complete returned `Profile`, and retain
|
|
the existing nested-extra-parameter and invalid-parameter integration cases.
|
|
|
|
#### S02-F05: Stable JSON field mappings have multiple manual owners
|
|
|
|
- **Category:** duplication
|
|
- **Severity:** medium
|
|
- **Confidence:** high
|
|
- **Status:** accepted
|
|
- **Affected code:** `json.go` (`PreparedRun.MarshalJSON`, `runResultJSON`,
|
|
`RunResult.MarshalJSON`, and `RunResult.UnmarshalJSON`) and JSON tests in
|
|
`public_contract_test.go`
|
|
- **Contract at issue:** The stable public JSON shape should preserve every
|
|
public field except for intentional timing representation and omission
|
|
rules. The list of ordinary fields is one serialization policy, not a
|
|
separate rule for each encoding direction.
|
|
- **Evidence:** `PreparedRun.MarshalJSON` redeclares and assigns every field in
|
|
an anonymous wire struct. `RunResult` repeats its field list in the public
|
|
type, `runResultJSON`, the marshal literal, and the unmarshal literal. The
|
|
custom handling is needed only for timing fields, but ordinary fields are
|
|
manually synchronized around it. Existing JSON tests cover timing,
|
|
artifact content type, session ID, and backend omission but do not round-trip
|
|
fully populated values. Adding a public field to either value can therefore
|
|
omit it from stable JSON without a compile failure or focused test failure.
|
|
- **Failure mode:** Public Go values and their documented stable JSON form can
|
|
drift, or marshal and unmarshal can become asymmetric, as fields evolve.
|
|
Consumer data may be silently absent after persistence or interchange.
|
|
- **Recommended direction:** Structure the wire representation so ordinary
|
|
fields derive from a single alias or embedded representation and only the
|
|
timing exceptions require explicit mapping. Avoid changing existing JSON
|
|
names or omission behavior while consolidating ownership.
|
|
- **Required verification:** Add fully populated `PreparedRun` and `RunResult`
|
|
JSON contract cases that check required names and omissions and compare all
|
|
fields after round trip, alongside the timing-boundary regression from
|
|
S02-F01.
|
|
|
|
### Unresolved Observations
|
|
|
|
None. Questions belonging to engine assembly or internal component behavior
|
|
were handed to their owning stages rather than promoted from partial traces.
|
|
|
|
### Coverage Ledger
|
|
|
|
- **Backend helpers:** `LocalBackend` is a side-effect-free conventional-value
|
|
constructor. `WithBackend` copies the queue-capacity pointer when the option
|
|
is applied, and existing tests protect normalization, invalid and duplicate
|
|
definitions, nested extra-parameter freezing at construction, lookup copy
|
|
behavior, and engine isolation. Actual registry composition remains Stage 3
|
|
scope and internal registry policy remains Stage 6 scope.
|
|
- **Profile helpers:** `OpenAICompatibleProfile` correctly performs a shallow
|
|
top-level extra-parameter copy and defers deep validation and freezing to the
|
|
general profile path as documented. The full-mapping test gap is S02-F04;
|
|
the duplicated acceptance policy is S02-F02.
|
|
- **Memory profile repository:** Repository construction rejects duplicate IDs
|
|
and invalid nested JSON-compatible values, stores domain copies, and returns
|
|
independent profile copies. Its source-neutral validation rule lacks a
|
|
single owner as recorded in S02-F02.
|
|
- **Artifact reader adapter:** The public and internal reader interfaces each
|
|
contain only `Read`. The adapter passes the caller context and error identity
|
|
through, rejects a nil successful artifact, translates references without
|
|
policy duplication, and copies returned body bytes. Focused and integrated
|
|
tests protect mutation isolation, nil handling, reference translation,
|
|
cancellation identity, and collaborator error identity.
|
|
- **LLM client adapter:** The public and internal client interfaces each
|
|
contain only `Generate`. The adapter forwards the exact context, preserves
|
|
client error identity for the use-case boundary, rejects a nil successful
|
|
response, and translates the scalar response without extra policy. The
|
|
request conversion currently deep-copies mutable data; its missing mutation
|
|
regression protection is S02-F03.
|
|
- **Adapter cancellation and errors:** Existing public tests establish caller
|
|
cancellation identity for generation and artifact loading and preserve
|
|
injected sentinel errors through public wrapping. No adapter adds an
|
|
independent deadline or cancellation mechanism.
|
|
- **Stable JSON:** Intentional timestamp, millisecond-duration, zero-value,
|
|
session, artifact, and backend-identity behavior is partly protected. The
|
|
confirmed overflow is S02-F01 and manual mapping drift is S02-F05.
|
|
- **Filesystem, reader, and client ownership:** Reader and client values are
|
|
stored as narrow injected interfaces and mutable values crossing their
|
|
adapter calls are copied as described above. Filesystem option validation,
|
|
lifetime, and composition reside in `engine.go` and are intentionally handed
|
|
to Stage 3 rather than inferred from this stage's helper review.
|
|
|
|
### Verification Performed
|
|
|
|
The code knowledge graph was used to locate each scoped helper and adapter,
|
|
trace its callers and callees, compare public and internal interface widths,
|
|
and confirm the duplicated profile rule. Source and focused tests were then
|
|
read to verify the graph conclusions.
|
|
|
|
The following focused commands passed:
|
|
|
|
```sh
|
|
go test . -run 'Test(PublicArtifactReaderAdapterCopiesBody|RunSucceedsWithInjectedLLMClient|RunPassesPreparedRequestToInjectedLLMClient|EngineRunPropagatesCallerCancellation|WithArtifactReaderRejectsNilReader|ArtifactReaderReceivesPublicReferenceAndPreparesArtifact|ArtifactReaderFailuresPreserveArtifactLoadErrors|RunAddsLLMGenerateToCollaboratorPublicError|PublicErrorsSupportErrorsIs|OpenAICompatibleProfileRunsThroughNormalProfilePath|OpenAICompatibleProfileDefersExtraParamsValidation|OpenAICompatibleProfileNestedExtraParamsRunThroughWithProfiles|WithProfilesRejectsDuplicateIDs|WithProfilesRejectsInvalidExtraParams|WithProfilesRejectsCyclicExtraParams|LocalBackendConstructsAndRegistersConventionalBackend|WithBackendCopiesQueueCapacity|BackendRegistrationRejectsInvalidAndDuplicateDefinitions|BackendExtraParamsAreDeeplyCopiedAtConstructionAndLookup|PreparedRunJSONOmitsZeroTimingValues|BackendIdentityJSONNamesAndOmission|PreparedRunJSONTimingRoundTrips|RunResultJSONUsesMillisecondsAndRoundTrips)$'
|
|
go test ./internal/profile -run 'Test(FilesystemRepository_GetProfile|FSRepository)$'
|
|
go test -race . -run 'Test(EngineRunPropagatesCallerCancellation|ArtifactReaderFailuresPreserveArtifactLoadErrors|PublicArtifactReaderAdapterCopiesBody|OpenAICompatibleProfileNestedExtraParamsRunThroughWithProfiles)$' -count=3
|
|
```
|
|
|
|
A temporary program outside the repository decoded
|
|
`{"duration_ms":9223372036854775807}` into `RunResult`; `go run` reported
|
|
`error=<nil> duration=-1ms nanoseconds=-1000000`, confirming S02-F01. The
|
|
temporary source was removed and no probe output was added to the repository.
|
|
|
|
### Handoff
|
|
|
|
- The Stage 0 baseline remains absent and was not backfilled during this
|
|
component review.
|
|
- Stage 3 owns `NewEngine` option application and the actual composition,
|
|
validation, and lifetime of configured filesystems, readers, clients,
|
|
profiles, and backends.
|
|
- Stage 6 owns internal backend registry and built-in profile policy. Stage 8
|
|
owns the broader filesystem profile repository review; it should use
|
|
S02-F02 as established evidence rather than repeating the public-side audit.
|
|
- Stage 14 owns transport-specific request construction, deadlines, response
|
|
decoding, and resource handling. This stage assessed only the public
|
|
injection adapter.
|
|
|
|
## Stage 3: Engine Construction, Options, And Source Assembly
|
|
|
|
### Scope Reviewed
|
|
|
|
The review covered the construction and option portions of `engine.go`, the
|
|
construction effect of `WithBackend` in `backends.go`, and directly relevant
|
|
tests in `engine_test.go`, `public_contract_test.go`, and
|
|
`capacity_contract_test.go`. Narrow traces into backend registry snapshots,
|
|
capacity-manager construction, built-in client construction, repositories,
|
|
validators, and `usecase.NewRunner` were used only to confirm the values and
|
|
dependencies assembled by `NewEngine`. Runtime engine methods, repository
|
|
parsing mechanics, validation mechanics, transport behavior, and capacity
|
|
scheduling were not audited.
|
|
|
|
### Accepted Findings
|
|
|
|
#### S03-F01: Single-file source options alter valid caller paths
|
|
|
|
- **Category:** correctness
|
|
- **Severity:** medium
|
|
- **Confidence:** confirmed
|
|
- **Status:** accepted
|
|
- **Affected code:** `engine.go` (`fileSource`, `WithPromptFile`,
|
|
`WithProfileFile`, and `WithSchemaFile`) and
|
|
`TestSourceOptionsRejectInvalidInputs` plus the three single-file success
|
|
tests in `engine_test.go`
|
|
- **Contract at issue:** Each single-file option accepts a path naming an
|
|
existing non-directory file. Filesystem paths are exact caller values;
|
|
leading and trailing whitespace are legal filename characters and the
|
|
option GoDoc does not define normalization.
|
|
- **Evidence:** `fileSource` assigns `strings.TrimSpace(name)` to `cleanName`
|
|
and performs every path operation and `os.Stat` against that altered value.
|
|
A focused probe created an existing file named `prompt.yaml `, confirmed
|
|
that `os.Stat` on the supplied path succeeded, and passed the same value to
|
|
`WithPromptFile`. `NewEngine` returned `ErrInvalidConfig` because it instead
|
|
attempted to stat `prompt.yaml` without the trailing space. All three public
|
|
file options share this helper. Existing tests cover ordinary paths, blank
|
|
paths, one missing path, and one directory path, but no exact-path boundary.
|
|
- **Failure mode:** A consumer cannot configure an otherwise valid prompt,
|
|
profile, or schema file whose name begins or ends with whitespace. The error
|
|
also reports the altered path, obscuring why the supplied existing file was
|
|
rejected.
|
|
- **Recommended direction:** Use trimming only to enforce the chosen blank-
|
|
input rule, then perform path decomposition, validation, error reporting,
|
|
and filesystem access with the original caller-supplied path.
|
|
- **Required verification:** Add a compact shared regression that constructs
|
|
engines through all three single-file options using existing paths with a
|
|
leading or trailing whitespace character. Retain the ordinary missing-file
|
|
and directory rejection cases.
|
|
|
|
#### S03-F02: Construction precedence tests do not isolate documented ordering rules
|
|
|
|
- **Category:** testing
|
|
- **Severity:** medium
|
|
- **Confidence:** high
|
|
- **Status:** accepted
|
|
- **Affected code:** `engine.go` (`Option`, `NewEngine`, and
|
|
`newProfileRepository`) and `TestSourceOptionsRejectInvalidInputs`,
|
|
`TestRepeatedOptionsUseLastValueInEachCategory`,
|
|
`TestInMemoryProfilesOverrideBuiltInsAndProfileSources`, and
|
|
`TestFallbackProfileSourcePrecedence`
|
|
- **Contract at issue:** Option order selects the last valid value within a
|
|
category, but profile lookup has a fixed cross-category order independent of
|
|
argument order: in-memory, ordinary configured, application fallback, then
|
|
built-in. A file or FS ordinary-profile option replaces `Config.ProfileDir`,
|
|
and any invalid option must fail construction even if a later option would
|
|
replace it.
|
|
- **Evidence:** The implementation correctly stores categories separately and
|
|
assembles the fixed profile overlay after applying options. The main
|
|
precedence tests, however, pass fallback, ordinary, and in-memory options in
|
|
the same low-to-high order that a generic order-based overlay would use, so
|
|
they would remain green if argument order accidentally began controlling
|
|
cross-category precedence. No directly relevant test gives
|
|
`Config.ProfileDir` and an ordinary-profile option colliding IDs to protect
|
|
the documented replacement, and invalid-option tests do not place a valid
|
|
replacement after the invalid value. Same-category last-value behavior is
|
|
well covered but does not protect these distinct rules.
|
|
- **Failure mode:** An assembly refactor could make mixed profile-source order
|
|
depend on option order, allow `Config.ProfileDir` to compete with its
|
|
replacement option, or silently discard an earlier invalid option. The
|
|
current tests could still pass while consumers observe different selected
|
|
profiles or construction success.
|
|
- **Recommended direction:** Extend the existing precedence coverage with a
|
|
small set of discriminating cases rather than a combinatorial matrix: reverse
|
|
the cross-category option order, collide `Config.ProfileDir` with its option
|
|
replacement, and place a valid same-category option after an invalid one.
|
|
- **Required verification:** Assert the selected model for the two profile-
|
|
source cases and `errors.Is(err, ErrInvalidConfig)` for the invalid-then-
|
|
valid case. Keep the test at the public construction boundary and avoid
|
|
assertions about private repository nesting.
|
|
|
|
### Unresolved Observations
|
|
|
|
None. Lower-confidence concerns about typed-nil interface values and unusual
|
|
non-regular files were not promoted because the documented Go interface and
|
|
file contracts do not establish stronger behavior.
|
|
|
|
### Coverage Ledger
|
|
|
|
- **Option application:** `NewEngine` applies non-nil options once in argument
|
|
order and stops on the first error. Nil options compose safely in an option
|
|
slice. Same-category prompt, ordinary profile, fallback profile, in-memory
|
|
profile, schema, client, and reader options use last-valid-value semantics;
|
|
backend registrations alone accumulate. The unprotected ordering edges are
|
|
recorded as S03-F02.
|
|
- **Required and default dependencies:** A nonblank configured prompt
|
|
directory or prompt-source option is required. Profiles always end with the
|
|
embedded built-in repository; schema validation defaults to the documented
|
|
directory; the artifact reader, renderer, and model client receive
|
|
application-neutral defaults when not injected. Construction performs no
|
|
provider request and requires no credential.
|
|
- **Prompt and schema source selection:** Prompt and schema FS or file options
|
|
replace their corresponding `Config` directory, retain the injected `fs.FS`
|
|
for lazy access, and validate nil filesystems and blank roots. Single-file
|
|
exact-path handling is defective as recorded in S03-F01. Source contents
|
|
remain lazy and their parsing and containment belong to Stages 7 and 10.
|
|
- **Profile composition:** `newProfileRepository` builds one explicit overlay
|
|
in the documented order: built-in, application fallback, one ordinary
|
|
configured source, then in-memory profiles. Only one ordinary source is
|
|
installed, and an ordinary option suppresses `Config.ProfileDir`. Matching
|
|
malformed higher-precedence definitions stop lookup rather than becoming
|
|
failover. The implementation is clear; S03-F02 concerns discriminating test
|
|
coverage, not current behavior.
|
|
- **Backend and capacity assembly:** All consumer backend additions enter one
|
|
immutable registry with the built-in backend. `NewEngine` takes one capacity-
|
|
policy snapshot, constructs a fresh manager, and wraps either the injected
|
|
or built-in client with that same manager before passing both to the runner.
|
|
Invalid definitions and capacity policies fail as `ErrInvalidConfig`, and
|
|
tests protect additive registrations, deep-copy isolation, limited and
|
|
unlimited behavior, and independence between engines. Registry rules and
|
|
scheduler mechanics remain Stages 6 and 15 scope.
|
|
- **Caller-owned values and collaborators:** Queue-capacity pointers are copied
|
|
when `WithBackend` is created; backend maps and in-memory profile values are
|
|
deeply frozen during construction. Injected filesystems, readers, clients,
|
|
and HTTP transports remain explicit collaborator references. The built-in
|
|
LLM constructor clones the supplied `http.Client` and focused internal tests
|
|
protect non-mutation for positive, zero, and negative timeouts.
|
|
- **Client and validator selection:** `WithLLMClient` prevents construction of
|
|
the built-in client while retaining engine-local capacity wrapping.
|
|
Otherwise `Config.Timeout` and a cloned `Config.HTTPClient` configure the
|
|
built-in client. Schema options construct the matching validator, while an
|
|
empty `SchemaDir` uses the application-neutral default. Transport and
|
|
validation semantics remain Stages 14 and 10 scope.
|
|
- **Failure atomicity and global state:** Every error path returns before an
|
|
`Engine` is published. Construction state is local, registry and capacity
|
|
values are rebuilt for each engine, and there is no process-global mutable
|
|
configuration. File-backed prompt, profile, and schema contents are read
|
|
lazily; malformed or missing source content is classified only when an
|
|
operation selects it.
|
|
- **Assembly clarity and cost:** Construction is a single option pass followed
|
|
by one repository, registry, manager, client, validator, reader, renderer,
|
|
and runner assembly. No relevant repeated I/O, parsing, or copying cost was
|
|
found, and the category flags make replacement and default selection
|
|
explicit without duplicating internal component policy.
|
|
- **Test ownership:** Root external-package tests appropriately protect public
|
|
option validity, source selection, profile precedence, copy isolation,
|
|
default-client configuration, and engine-local backend and capacity
|
|
behavior. Focused internal tests own registry normalization, manager policy,
|
|
and HTTP-client cloning. S03-F02 identifies the material missing distinctions
|
|
rather than recommending duplicate internal choreography tests.
|
|
|
|
### Verification Performed
|
|
|
|
The code knowledge graph was used to find `NewEngine`, every option category,
|
|
source-assembly helpers, and directly relevant tests; trace construction into
|
|
the registry, capacity manager, repositories, validators, model client, and
|
|
runner; and confirm that later runtime mechanics were outside the reviewed
|
|
path. Important ownership and error conclusions were confirmed against source.
|
|
|
|
The following focused commands passed:
|
|
|
|
```sh
|
|
go test . -run 'Test(NewEngineRejectsMissingPromptDir|NewEngineAcceptsMissingProfileDir|SourceOptionsRejectInvalidInputs|PackageOptionsComposeFromSlice|RepeatedOptionsUseLastValueInEachCategory|FallbackProfileSourcePrecedence|FallbackProfileSourcePreservesLazyLoadingAndErrors|BackendOptionsAccumulateAndRegistrationsAreEngineLocal|BackendRegistrationRejectsInvalidAndDuplicateDefinitions|BackendExtraParamsAreDeeplyCopiedAtConstructionAndLookup|BackendCapacityIsIndependentBetweenEngines|WithLLMClientRejectsNilClient|WithArtifactReaderRejectsNilReader|PromptRepositoryReadFailureMapsToPromptLoad|SelectedProfileRepositoryReadFailureMapsToProfileLoad|PrepareWorksWithPromptFSAndRelativeContentFile|PrepareWorksWithPromptFile|PrepareWorksWithProfileFSOverBuiltIns|PrepareWorksWithProfileFileOverBuiltIns|RunStructuredOutputWorksWithSchemaFS|RunStructuredOutputWorksWithSchemaFile|EngineRunLayersTransportAndGenerationTimeouts)$'
|
|
go test ./internal/backend -run 'Test(RegistryIncludesExactOpenRouterDefinition|RegistryNormalizesUniqueAdditionsAndIsolatesMutations|NewRegistryNormalizesCapacityPolicy)$'
|
|
go test ./internal/capacity -run 'Test(NewManagerRejectsInvalidPolicies|ManagerAdmissionIsBoundedAndReleaseIsIdempotent|ManagerAdmissionHonorsContextAndUnlimitedBackends)$'
|
|
go test ./internal/llm -run 'TestNewOpenAICompatibleClientDoesNotMutateSupplied(Nonzero|Zero)TimeoutClient|TestNewOpenAICompatibleClientTreatsSuppliedNegativeTimeoutAsUnset'
|
|
go test -race . -run 'Test(BackendCapacityIsIndependentBetweenEngines|BackendOptionsAccumulateAndRegistrationsAreEngineLocal|EngineSupportsConcurrentPrepareAndRun|RepeatedOptionsUseLastValueInEachCategory)$' -count=3
|
|
```
|
|
|
|
A temporary program outside the repository created an existing
|
|
`prompt.yaml ` file and called `NewEngine` with `WithPromptFile` using that
|
|
exact path. Direct `os.Stat` returned nil, while construction returned
|
|
`ErrInvalidConfig` after reporting the trimmed `prompt.yaml` path, confirming
|
|
S03-F01. The temporary source and generated file were removed.
|
|
|
|
### Handoff
|
|
|
|
- The Stage 0 baseline remains absent and was not backfilled during this
|
|
component review.
|
|
- Stage 4 owns operation entry points and public runtime error/result behavior;
|
|
construction tests were read only through the behavior needed to observe
|
|
assembled dependencies.
|
|
- Stages 6, 7, 8, 10, 14, and 15 own backend policy, prompt sources, profile
|
|
repositories, validators, provider transport, and capacity scheduling
|
|
respectively. This stage established only that `NewEngine` selects and wires
|
|
their boundaries consistently.
|
|
|
|
## Stage 4: Engine Operations And Root Contract Coverage
|
|
|
|
### Scope Reviewed
|
|
|
|
The review covered the public operation portion of `engine.go`
|
|
(`InspectPrompt`, `InspectProfile`, `Prepare`, `PrepareExecution`, `Run`, and
|
|
`RunPrepared`) and the directly relevant root tests in `engine_test.go`,
|
|
`public_contract_test.go`, `prepared_execution_contract_test.go`, and
|
|
`errors_internal_test.go`. `prepared_execution.go` and conversion helpers were
|
|
consulted only to confirm the operation boundary established in Stage 1.
|
|
Internal runner tests were consulted only to identify test ownership; runner,
|
|
transport, validation, capacity, and repository mechanics were treated as
|
|
black boxes.
|
|
|
|
### Accepted Findings
|
|
|
|
#### S04-F01: Ordinary-run cancellation identity is protected only below the public boundary
|
|
|
|
- **Category:** testing
|
|
- **Severity:** medium
|
|
- **Confidence:** high
|
|
- **Status:** accepted
|
|
- **Affected code:** `engine.go` (`Engine.Run`), `errors.go`
|
|
(`mapPublicError`), `engine_test.go`
|
|
(`TestEngineRunPropagatesCallerCancellation`),
|
|
`errors_internal_test.go`
|
|
(`TestMapPublicErrorPreservesGenerationCancellation`), and
|
|
`internal/usecase/runner_test.go`
|
|
(`TestRunnerRunCancellationPreservesGenerationCategory`)
|
|
- **Contract at issue:** `Engine.Run` passes the caller's context through the
|
|
execution boundary and preserves the active collaborator's cancellation
|
|
identity while adding the public operation category. Cancellation and
|
|
injected dependency failures are consequential public behaviors that should
|
|
be asserted through the consumer-visible boundary.
|
|
- **Evidence:** `Engine.Run` currently passes `ctx` unchanged to the runner and
|
|
maps its returned error without dropping wrapped identities. The external-
|
|
package cancellation test drives a real request context through the built-in
|
|
HTTP transport but asserts only `errors.Is(err, ErrLLMGenerate)` after
|
|
cancellation. The `context.Canceled` identity is asserted separately only
|
|
against the unexported `mapPublicError` helper and internal runner. A search
|
|
of the root operation tests found no other ordinary-run assertion for that
|
|
identity. Prepared execution does assert both identities at the public
|
|
boundary, but it exercises a different entry point.
|
|
- **Failure mode:** A facade or ordinary-run composition change could replace
|
|
the caller context, stop wrapping the collaborator cancellation, or discard
|
|
it during public error mapping. The internal tests and existing public test
|
|
could all remain green while consumers lose the ability to distinguish
|
|
caller cancellation with `errors.Is(err, context.Canceled)`.
|
|
- **Recommended direction:** Extend the existing external-package
|
|
`TestEngineRunPropagatesCallerCancellation` assertion to require both
|
|
`ErrLLMGenerate` and `context.Canceled`. Retain the focused internal tests
|
|
only for the distinct internal translation and runner responsibilities they
|
|
protect; do not add a parallel end-to-end cancellation test.
|
|
- **Required verification:** Run the focused public cancellation test normally
|
|
and with the race detector. Confirm that deliberately removing caller-context
|
|
propagation or cancellation wrapping at the facade boundary makes that test
|
|
fail.
|
|
|
|
### Unresolved Observations
|
|
|
|
None. The absence of package-level operation convenience functions was
|
|
confirmed and is not a consistency defect; the reviewed public API exposes
|
|
these operations only as `Engine` methods.
|
|
|
|
### Coverage Ledger
|
|
|
|
- **Facade shape and request translation:** All six methods reject a nil or
|
|
uninitialized engine before delegation. `Prepare`, `PrepareExecution`, and
|
|
`Run` use the same field-complete, defensive request conversion and classify
|
|
conversion failures as `ErrInvalidRequest`; inspections pass their scalar
|
|
selectors with the documented prompt/profile normalization behavior.
|
|
`RunPrepared` unwraps only the opaque handle reference. Each successful
|
|
facade method delegates once and converts the returned domain snapshot.
|
|
- **Context propagation:** Every method passes the supplied context directly
|
|
to its matching runner operation. Public tests protect cancellation before
|
|
inspection source work, active ordinary generation, and prepared generation,
|
|
and prove that a completed preparation is independent of later cancellation
|
|
of its preparation context. The missing consumer-boundary assertion for the
|
|
ordinary-run cancellation identity is recorded as S04-F01.
|
|
- **Inspection operations:** Prompt inspection loads declared metadata and
|
|
referenced content without profile, artifact, schema, capacity, or provider
|
|
work. Profile inspection resolves the effective target and credential state
|
|
without prompt or generation work. External-package tests protect nil and
|
|
blank inputs, not-found versus load identities, cancellation, point-in-time
|
|
behavior, agreement with preparation, and deep ownership of returned nested
|
|
values.
|
|
- **Preparation and ordinary execution:** `Prepare` returns a caller-owned,
|
|
credential-redacted prepared snapshot and performs no model generation.
|
|
`Run` returns a caller-owned result after one execution path; content
|
|
validation failure remains a successful result, while operational failures
|
|
return no partial result. Root tests protect representative translation,
|
|
prepared/generated metadata agreement, injected artifact and LLM behavior,
|
|
validation-result semantics, and the documented public error categories.
|
|
- **Prepared execution boundary:** `PrepareExecution` publishes one opaque,
|
|
engine-bound handle whose details are independent copies of frozen
|
|
preparation state. `RunPrepared` preserves owner binding, atomic single-use
|
|
claim behavior, independent execution context, no-result-on-error semantics,
|
|
collaborator identities, credential revalidation, capacity rejection, and
|
|
execution-only timing. External-package tests also protect concurrent claim
|
|
and run/discard behavior. The internal claim, admission, validation, and
|
|
release mechanisms remain assigned to later component stages.
|
|
- **Public error mapping:** Every runner error is routed through one facade
|
|
mapping point. Ordinary public identities and injected collaborator errors
|
|
remain discoverable with `errors.Is`; capacity failures become public
|
|
`CapacityError` values without leaking the internal type; successful paths
|
|
do not invent errors. Internal mapping tests appropriately own internal-type
|
|
containment, while public operation tests own consumer-visible categories
|
|
and collaborator identities except for S04-F01.
|
|
- **Ownership:** Request conversion freezes caller maps, slices, pointers, and
|
|
JSON-compatible values before internal use. Inspection, prepared, details,
|
|
and result conversions return fresh mutable values. The prepared-execution
|
|
contract tests demonstrate that later caller and source mutations do not
|
|
change frozen execution and that mutating one returned snapshot does not
|
|
change engine-owned state.
|
|
- **Convenience-function consistency:** The graph and source search found no
|
|
package-level `Prepare`, `PrepareExecution`, `Run`, `RunPrepared`,
|
|
`InspectPrompt`, or `InspectProfile` functions. There is therefore no second
|
|
operation surface whose translation, errors, or ownership can drift from
|
|
the methods.
|
|
- **Test ownership and duplication:** Root external-package tests protect the
|
|
exported facade and representative assembled workflows. Focused internal
|
|
tests own runner coordination and public-error translation mechanics. Some
|
|
lifecycle and error categories necessarily appear at both levels, but the
|
|
assertions address different stable boundaries; no removable semantic
|
|
duplication was found. S04-F01 is the one important identity currently
|
|
asserted only below the applicable public operation boundary.
|
|
- **Clarity and cost:** The operation facade is a uniform sequence of guard,
|
|
translation where needed, one delegation, public error mapping, and outward
|
|
conversion. No duplicated orchestration policy, repeated I/O, avoidable
|
|
copying, or operation-layer complexity was found. Costs inside preparation,
|
|
generation, validation, transport, and capacity remain assigned to their
|
|
owning later stages.
|
|
|
|
### Verification Performed
|
|
|
|
The code knowledge graph was used to find every public engine operation,
|
|
confirm their runner and conversion edges, inventory directly relevant root
|
|
tests, locate the internal cancellation assertions, and verify that no
|
|
package-level operation convenience functions exist. Important context,
|
|
error, ownership, and no-partial-result conclusions were confirmed against
|
|
source.
|
|
|
|
The following focused commands passed:
|
|
|
|
```sh
|
|
go test . -run 'Test(PrepareWorksWithFrameworkContractCorpus|RunSucceedsWithInjectedLLMClient|RunPassesPreparedRequestToInjectedLLMClient|EngineRunPropagatesCallerCancellation|ArtifactReaderFailuresPreserveArtifactLoadErrors|RunAddsLLMGenerateToCollaboratorPublicError|PrepareWithoutProfileMatchesSpecificPublicError|RunValidationFailureReturnsResult|PublicErrorsSupportErrorsIs|InspectProfileResolvesCredentialStatesWithoutPromptOrGeneration|InspectProfilePreservesPublicErrorIdentities|InspectProfileReturnsIndependentTargetMatchingPreparation|InspectPromptReturnsDeclaredMetadataWithoutExecutionWork|InspectPromptPreservesPublicErrorIdentities|InspectPromptReturnsIndependentMetadataMatchingPreparation|PreparedExecutionFreezesSourcesAndReturnsIndependentDetails|PreparedExecutionLifecycleAndEngineBinding|PreparedExecutionConcurrentClaimAllowsOneGeneration|PreparedExecutionRunAndDiscardRaceHasOneWinner|PreparedExecutionDiscardAndFormattingDoNotExposePrivateState|PreparedExecutionCredentialCapacityAndTimingBoundaries)$'
|
|
go test -race . -run 'Test(EngineRunPropagatesCallerCancellation|InspectProfilePreservesPublicErrorIdentities|InspectPromptPreservesPublicErrorIdentities|PreparedExecutionFreezesSourcesAndReturnsIndependentDetails|PreparedExecutionLifecycleAndEngineBinding|PreparedExecutionConcurrentClaimAllowsOneGeneration|PreparedExecutionRunAndDiscardRaceHasOneWinner)$' -count=3
|
|
go test ./internal/usecase -run 'TestRunnerRunCancellationPreservesGenerationCategory'
|
|
go test . -run 'Test(MapPublicErrorPreservesGenerationCancellation|MapPublicErrorTranslatesCapacityError)$'
|
|
```
|
|
|
|
### Handoff
|
|
|
|
- The Stage 0 baseline remains absent and was not backfilled during this
|
|
operation-boundary review.
|
|
- Stages 7, 8, 10, 11, 13, 14, and 15 own prompt loading, profile loading,
|
|
validation, runner preparation/execution, prepared-handle lifecycle,
|
|
provider transport, and capacity mechanics respectively. Those stages
|
|
should treat the facade behavior recorded here as their outward contract
|
|
rather than repeat this public-boundary audit.
|
|
|
|
## Stage 5: Internal Domain And JSON-Compatible Values
|
|
|
|
### Scope Reviewed
|
|
|
|
The review covered every source and test file in `internal/domain` and
|
|
`internal/jsonvalue`. Narrow traces into the root conversion boundary,
|
|
backend registry, prompt renderer, provider request builder, and prepared-run
|
|
cloning were used only to confirm the assumptions those callers make about
|
|
session normalization, copied JSON-compatible values, credential redaction,
|
|
and frozen schemas. Their broader validation, execution, transport, and
|
|
registry behavior was not audited.
|
|
|
|
### Accepted Findings
|
|
|
|
#### S05-F01: Session normalization accepts values that JSON encoding changes
|
|
|
|
- **Category:** correctness
|
|
- **Severity:** medium
|
|
- **Confidence:** confirmed
|
|
- **Status:** accepted
|
|
- **Affected code:** `internal/domain/session.go` (`NormalizeSessionID`),
|
|
`internal/domain/session_test.go` (`TestNormalizeSessionID`), and callers in
|
|
`internal/usecase`, `internal/prompt`, and `internal/llm`
|
|
- **Contract at issue:** A normalized session ID is opaque consumer metadata
|
|
whose Unicode-code-point length is bounded and whose value is carried into
|
|
prepared metadata, rendered-prompt hashing, collaborator requests, and the
|
|
provider's JSON `session_id`. Normalization must not approve one byte string
|
|
while downstream encoding transmits a different identifier.
|
|
- **Evidence:** `NormalizeSessionID` trims the string and counts runes but never
|
|
checks `utf8.ValidString`. Go strings may contain invalid UTF-8, and
|
|
`utf8.RuneCountInString` counts malformed bytes as error runes rather than
|
|
rejecting them. A temporary probe passed `x\xffy`: normalization returned
|
|
the original invalid string with no error, while `encoding/json` emitted
|
|
`"x\ufffdy"`. Existing tests cover Unicode whitespace and the rune-count
|
|
boundary but no malformed encoding. The runner hashes the normalized string
|
|
before the provider request is JSON-encoded, so the exposed hash can also
|
|
describe a different session value than the provider observes.
|
|
- **Failure mode:** A direct or rendered session ID containing malformed UTF-8
|
|
is accepted, retained, and hashed in one form but silently replaced with
|
|
Unicode replacement characters on the wire. Provider correlation and local
|
|
prepared/result metadata can therefore disagree for an accepted request.
|
|
- **Recommended direction:** Make valid UTF-8 part of the shared normalization
|
|
rule and reject malformed values before trimming/counting succeeds. Keep the
|
|
error contextual but independent of provider-transport implementation.
|
|
- **Required verification:** Add malformed UTF-8 cases before, within, and
|
|
after otherwise valid content to the domain table, then retain focused
|
|
caller checks that direct-request failures map to `ErrInvalidRequest` and
|
|
rendered-template failures map to the renderer category without provider
|
|
work.
|
|
|
|
#### S05-F02: Numeric acceptance depends on the caller's Go representation
|
|
|
|
- **Category:** contract-documentation consistency
|
|
- **Severity:** medium
|
|
- **Confidence:** confirmed
|
|
- **Status:** accepted
|
|
- **Affected code:** `internal/jsonvalue/jsonvalue.go` (`copyValue` and
|
|
`maxSafeJSONInteger`), `internal/jsonvalue/jsonvalue_test.go`
|
|
(`TestCopyMapRejectsInvalidValues` and
|
|
`TestCopyMapValidatesJSONNumberSyntaxAndRange`), and the finite-number
|
|
contracts in `types.go` and `backends.go`
|
|
- **Contract at issue:** Public extra-parameter contracts accept finite
|
|
JSON-compatible numbers, and the shared copier promises to preserve
|
|
compatible concrete numeric types. Equivalent numeric values should not
|
|
become valid or invalid solely because the caller chose an integer,
|
|
floating-point, or `json.Number` representation unless that distinction is
|
|
an explicit contract.
|
|
- **Evidence:** Signed and unsigned integers outside `[-(2^53-1), 2^53-1]`
|
|
are rejected, but finite floats receive no corresponding safe-integer check
|
|
and `json.Number` is checked only for JSON syntax and finite `float64`
|
|
range. A temporary probe submitted the exact value `9007199254740992` as
|
|
`int64`, `float64`, and `json.Number`: only the `int64` was rejected. Go's
|
|
JSON encoder can emit the integer spelling without losing it. Existing tests
|
|
deliberately require rejection for the integer form while treating no
|
|
cross-type boundary as policy, and the public GoDoc says only that numbers
|
|
must be finite.
|
|
- **Failure mode:** Semantically equivalent backend, profile, or request extra
|
|
parameters have different construction or request outcomes based on an
|
|
incidental Go type. Consumers decoding into `json.Number` can bypass the
|
|
integer limit that consumers using `int64` encounter, so the current limit
|
|
neither implements the public finite-number contract nor a uniform safe-
|
|
integer policy.
|
|
- **Recommended direction:** Establish one numeric acceptance rule in
|
|
`internal/jsonvalue` and apply it consistently to every supported concrete
|
|
representation. The current public contract points toward accepting all
|
|
finite JSON-encodable numeric values; if a narrower interoperability limit
|
|
is intentionally retained, make it an explicit public contract and enforce
|
|
it for integral floats and `json.Number` as well.
|
|
- **Required verification:** Add a representation matrix at the largest
|
|
accepted and first rejected positive and negative integer boundaries for
|
|
signed integers, unsigned integers, integral floats, and `json.Number`, plus
|
|
finite fractional/exponent and non-finite cases. Retain concrete-type
|
|
assertions for accepted values and a public-boundary error-mapping check.
|
|
|
|
#### S05-F03: JSON-value copying has no nesting or work bound
|
|
|
|
- **Category:** correctness
|
|
- **Severity:** high
|
|
- **Confidence:** high
|
|
- **Status:** accepted
|
|
- **Affected code:** `internal/jsonvalue/jsonvalue.go` (`Copy`, `CopyMap`,
|
|
`copyValue`, `copyMapValue`, and `copySequenceValue`) and
|
|
`internal/jsonvalue/jsonvalue_test.go`
|
|
- **Contract at issue:** Consumer-controlled extra-parameter and schema values
|
|
must fail as ordinary validation errors when their structure is unsafe to
|
|
process. Cycle rejection alone does not bound recursive stack use or the
|
|
amount of copying performed for an acyclic value.
|
|
- **Evidence:** Each pointer, map, slice, or array level recursively calls
|
|
`copyValue`, with no depth, visited-node, or copied-node budget. The `seen`
|
|
map tracks only the active recursion path and therefore detects cycles but
|
|
imposes no size bound. A temporary probe built an acyclic chain 20,000 maps
|
|
deep; `CopyMap` accepted and copied it. Sufficiently deeper caller-created
|
|
values can continue growing the goroutine stack until Go's fatal stack
|
|
limit. Shared acyclic subgraphs are also recopied once for every path rather
|
|
than counted or memoized, so a compact caller value can induce much larger
|
|
work. Existing tests cover direct map and slice cycles only.
|
|
- **Failure mode:** A deeply nested configuration or request can consume
|
|
disproportionate CPU, allocations, and stack and can eventually terminate
|
|
the process instead of returning `ErrInvalidConfig` or `ErrInvalidRequest`.
|
|
The same generic path is used during engine construction, request
|
|
conversion, backend lookup, and prepared schema/detail copying.
|
|
- **Recommended direction:** Give the shared copier an explicit, defensible
|
|
traversal budget that bounds nesting and total copied work, returning a
|
|
path-aware validation error when exceeded. Consider preserving already-
|
|
copied acyclic aliases or otherwise account for repeated subgraphs so the
|
|
bound covers expansion as well as source-node count. Keep the policy in this
|
|
package rather than adding different limits at each caller.
|
|
- **Required verification:** Add just-below, at-limit, and first-over-limit
|
|
cases for alternating map/slice/array nesting and for a shared acyclic
|
|
subgraph that expands through multiple paths. Confirm public configuration
|
|
and request callers translate the bounded failure without panic or provider
|
|
work. Exercise the focused tests under constrained stack/memory settings if
|
|
practical without making the default suite environment-dependent.
|
|
|
|
#### S05-F04: The generic copier's supported shape contract is only partially tested
|
|
|
|
- **Category:** testing
|
|
- **Severity:** medium
|
|
- **Confidence:** high
|
|
- **Status:** accepted
|
|
- **Affected code:** `internal/jsonvalue/jsonvalue.go` and all four tests in
|
|
`internal/jsonvalue/jsonvalue_test.go`
|
|
- **Contract at issue:** `Copy` and `CopyMap` are the single shared validator
|
|
and ownership boundary for maps, slices, arrays, scalar and numeric concrete
|
|
types, schemas with empty object keys, extra-parameter maps without empty
|
|
keys, cycles, unsupported values, and JSON null versus empty containers. The
|
|
reflection branches that implement those distinctions need compact tests at
|
|
this package boundary.
|
|
- **Evidence:** Successful-copy tests cover one `map[string]int`, one
|
|
`[]string`, `int64`, `json.Number`, and an empty schema-object key. Rejection
|
|
tests cover non-string and empty map keys, one unsupported channel, map and
|
|
slice cycles, non-finite floats, two out-of-range integers, and malformed
|
|
`json.Number`. No test exercises the distinct array allocation path, named
|
|
scalar/map/slice/array types, signed and unsigned width preservation,
|
|
ordinary floats, pointer indirection, nil interface/typed map/typed slice
|
|
collapse to JSON null, or nil-versus-empty map and slice preservation. The
|
|
temporary probe confirmed typed nil maps and slices become nil while empty
|
|
containers remain allocated, but this intended JSON distinction has no
|
|
durable owner.
|
|
- **Failure mode:** A reflection refactor can silently change accepted types,
|
|
return a different concrete type, alias an array or nested collection, or
|
|
collapse an empty container to null without any focused test failing. Every
|
|
engine configuration, request override, and prepared schema/detail path
|
|
relies on this machinery.
|
|
- **Recommended direction:** Expand the package table by behavior branch, not
|
|
by every type permutation: representative named and unnamed scalars, maps,
|
|
slices, arrays, pointer/interface indirection, nil and empty containers, and
|
|
mixed nested structures. Assert both deep mutation isolation and concrete
|
|
type where preservation is promised. Keep higher-level tests to a small
|
|
integration sample rather than repeating this matrix.
|
|
- **Required verification:** Run the focused package tests and representative
|
|
backend, profile, request, and prepared-schema ownership tests. Confirm
|
|
deliberate shallow-copy, array-allocation, type-conversion, and nil/empty
|
|
regressions each fail at the shared package boundary.
|
|
|
|
#### S05-F05: Internal prepared-run JSON tests exercise an unused serialization boundary
|
|
|
|
- **Category:** testing
|
|
- **Severity:** low
|
|
- **Confidence:** high
|
|
- **Status:** accepted
|
|
- **Affected code:** JSON tags on `internal/domain.PreparedRun` and the three
|
|
tests in `internal/domain/prepared_run_test.go`
|
|
- **Contract at issue:** The root `promptkit.PreparedRun` owns the stable
|
|
consumer JSON representation. Internal domain values should be tested for
|
|
invariants their internal consumers use, not as a parallel serialization
|
|
contract with no production caller.
|
|
- **Evidence:** Production paths construct and clone `domain.PreparedRun`, then
|
|
convert it to the root public value before consumer serialization. Source
|
|
and graph searches found direct `json.Marshal` calls on the internal type
|
|
only in `prepared_run_test.go`. The secret and session assertions overlap
|
|
runner/public contract coverage, while the cache-control assertion is made
|
|
only against the internal type and would remain green if root conversion
|
|
dropped that field. The secret test also constructs a domain prepared value
|
|
containing an API key even though the type's invariant says such values must
|
|
never contain resolved credentials; it proves only that a dormant JSON tag
|
|
hides the deliberately invalid state.
|
|
- **Failure mode:** Maintainers pay for and may preserve internal JSON tags and
|
|
tests that do not protect the supported facade, while a regression in the
|
|
public prepared conversion can escape the only cache-control serialization
|
|
assertion. Legitimate internal representation refactors can require test
|
|
edits without changing any consumer behavior.
|
|
- **Recommended direction:** Test credential absence at the producer and clone
|
|
boundaries that own the domain invariant, and keep stable JSON assertions on
|
|
the root public type. Move or replace the cache-control case at that public
|
|
boundary if the compatibility risk warrants it; remove the parallel
|
|
internal serialization expectations and tags unless an actual internal
|
|
serialization consumer exists.
|
|
- **Required verification:** Confirm prepared construction and cloning never
|
|
retain direct credentials, and assert session omission plus message
|
|
cache-control behavior through root `PreparedRun` JSON. A source search
|
|
should show no remaining production dependency before removing internal
|
|
tags or tests.
|
|
|
|
### Unresolved Observations
|
|
|
|
None. Pointer values are currently dereferenced into JSON tree values and
|
|
typed nil pointers, maps, and slices collapse to JSON null. Those behaviors are
|
|
consistent with JSON encoding, but S05-F04 records the need to give the
|
|
supported nil and indirection distinctions durable package-level coverage.
|
|
|
|
### Coverage Ledger
|
|
|
|
- **Domain role and invalid states:** `internal/domain` is a dependency-neutral
|
|
vocabulary of carrier types, enums, presence bits, and credential-redacted
|
|
result shapes, not a collection of independently valid aggregate
|
|
constructors. Prompt/profile/output/target validity remains with the
|
|
parsers, registries, runner, and validators that have the necessary context.
|
|
Zero enum values and partially populated carrier structs are therefore
|
|
representable by design; callers validate before effectful use.
|
|
- **Session normalization:** One shared function trims Unicode whitespace,
|
|
omits blank values, and measures the 256-character limit in Unicode code
|
|
points. Direct request, rendered template, and transport callers all use it,
|
|
avoiding divergent length rules. Invalid UTF-8 is the uncovered invariant
|
|
defect recorded as S05-F01.
|
|
- **Credential containment:** Domain request and execution-target values can
|
|
temporarily carry a direct API key for execution, with JSON/YAML tags that
|
|
omit it. Preparation explicitly clears the credential before constructing a
|
|
`PreparedRun`, execution adds it only to the transient generation target,
|
|
and results clear it again. Focused use-case tests protect these producer
|
|
invariants; dormant internal serialization tests are addressed by S05-F05.
|
|
- **Prepared and schema immutability:** Prepared execution takes separate deep
|
|
snapshots for executable state and durable details. Cloning copies target
|
|
extra parameters through `jsonvalue.CopyMap`, input hashes, rendered
|
|
messages and cache-control pointers, structured-output wrappers, and schema
|
|
trees through `jsonvalue.Copy`. `Details` produces another fresh snapshot.
|
|
Root contract tests demonstrate that source/request/detail mutations do not
|
|
alter execution or later details. Lifecycle mechanics remain Stage 13 scope.
|
|
- **Generic ownership:** `internal/jsonvalue` is the single owner for recursive
|
|
validation and copying of arbitrary JSON-shaped trees. Root profiles and
|
|
request overrides, backend construction/lookups, and prepared schema/detail
|
|
cloning all call this package rather than maintaining separate recursive
|
|
copiers. Shallow map copies in the runner operate only on already validated,
|
|
engine-owned snapshots while applying precedence; no competing acceptance
|
|
policy was found.
|
|
- **Supported values:** The implementation accepts nil, booleans, strings,
|
|
finite floating-point values, bounded integer values, valid finite
|
|
`json.Number`, string-keyed maps, slices, arrays, interfaces, and pointer
|
|
indirection. It preserves assignable concrete scalar and collection types,
|
|
otherwise produces canonical `map[string]any` or `[]any` trees. S05-F02
|
|
records the inconsistent integer policy, and S05-F04 records missing branch
|
|
protection.
|
|
- **Nil and empty distinctions:** A nil interface, pointer, map, or slice
|
|
becomes JSON null; non-nil empty maps and slices remain non-nil empty
|
|
containers. A nil top-level extra-parameter map remains nil. `Copy` permits
|
|
empty object keys for JSON Schemas, while `CopyMap` rejects empty keys at
|
|
every depth for provider extra parameters. These are coherent separate entry
|
|
contracts rather than duplicated machinery.
|
|
- **Cycles and unsupported values:** Active-path identity tracking rejects
|
|
pointer, map, and slice cycles, including nested cycles; maps with non-string
|
|
keys, channels, functions, structs, complex values, unsafe pointers,
|
|
malformed numbers, and non-finite floats are rejected with a structural
|
|
path. The lack of an acyclic traversal bound is S05-F03.
|
|
- **Copy cost:** Ordinary accepted trees are visited and allocated once per
|
|
occurrence, with sorted map keys making the first reported invalid path
|
|
deterministic. Boundary copies occur when caller ownership changes, backend
|
|
lookups publish snapshots, prepared execution separates payload and details,
|
|
and details are returned. No unnecessary duplicate validation owner was
|
|
found, but shared acyclic subgraphs can be expanded repeatedly as recorded
|
|
in S05-F03.
|
|
- **Test ownership:** Domain session tests own the shared normalization rule;
|
|
use-case tests own credential-free prepared construction and frozen
|
|
execution; root tests own public snapshots and JSON. JSON-value package tests
|
|
correctly use the narrow shared boundary but do not yet discriminate all
|
|
supported reflection branches (S05-F04). Internal prepared JSON tests are on
|
|
a non-production boundary (S05-F05).
|
|
|
|
### Verification Performed
|
|
|
|
The code knowledge graph was used to inventory every declaration and test in
|
|
both packages, trace all callers of `NormalizeSessionID`, `Copy`, and
|
|
`CopyMap`, and confirm prepared-run/schema clone and serialization ownership.
|
|
Important behavior was confirmed against complete package source and tests.
|
|
|
|
The following focused commands passed:
|
|
|
|
```sh
|
|
go test ./internal/domain ./internal/jsonvalue
|
|
go test ./internal/usecase -run 'Test(RunnerDirectSessionResolution|HashRenderedPromptIncludesSessionIDWhenPresent|RunnerPrepareExecutionCompletesWithoutAdmissionOrGeneration|RunnerRunPreparedKeepsDirectCredentialOutOfMetadata|ExecutionProfileToTargetPopulatesAllFieldsAndCopiesExtraParams)$'
|
|
go test . -run 'Test(PreparedExecutionFreezesSourcesAndReturnsIndependentDetails|ExtraParamsTypedNestedValuesAreCopiedAcrossPublicBoundary|BackendExtraParamsAreDeeplyCopiedAtConstructionAndLookup|RunRejectsInvalidExtraParams)$'
|
|
go test -race ./internal/domain ./internal/jsonvalue -count=3
|
|
go test -race . -run 'Test(PreparedExecutionFreezesSourcesAndReturnsIndependentDetails|ExtraParamsTypedNestedValuesAreCopiedAcrossPublicBoundary|BackendExtraParamsAreDeeplyCopiedAtConstructionAndLookup)$' -count=3
|
|
```
|
|
|
|
A temporary program under the repository imported the two internal packages
|
|
and confirmed S05-F01, S05-F02, and the 20,000-level acceptance evidence for
|
|
S05-F03. It also recorded the current nil-versus-empty behavior for the
|
|
coverage ledger. The temporary source and directory were removed before the
|
|
audit artifact was edited.
|
|
|
|
### Handoff
|
|
|
|
- The Stage 0 baseline remains absent and was not backfilled during this
|
|
shared-value review.
|
|
- Stage 6 should use the numeric and traversal findings when reviewing backend
|
|
extra parameters without repeating the generic copier audit. Stages 10 and
|
|
13 own validation-plan construction and prepared-handle lifecycle; they
|
|
should treat the schema-copy and credential-redaction behavior recorded here
|
|
as established boundary evidence.
|
|
- Stage 11 owns runner precedence and execution coordination. The shallow
|
|
runner map copies were consulted only to confirm that validated nested values
|
|
already have a single owner; their broader merge behavior remains out of
|
|
scope here.
|
|
|
|
## Stage 6: Backend Registry, Defaults, And Built-In Profiles
|
|
|
|
### Scope Reviewed
|
|
|
|
The review covered every source, test, and embedded YAML file in
|
|
`internal/backend`, `internal/defaults`, and `internal/profile/builtin`.
|
|
Narrow traces into `WithBackend`, `NewEngine`, execution-target resolution,
|
|
and the LLM-owned reserved request-field rule were used only to confirm public
|
|
translation, registry assembly, default consumption, and rule ownership.
|
|
Capacity admission and scheduling mechanics and outbound HTTP request
|
|
construction were not audited.
|
|
|
|
### Accepted Findings
|
|
|
|
#### S06-F01: The fixed model-request timeout is writable process-global state
|
|
|
|
- **Category:** clarity
|
|
- **Severity:** low
|
|
- **Confidence:** high
|
|
- **Status:** accepted
|
|
- **Affected code:** `internal/defaults/defaults.go`
|
|
(`LLMRequestTimeoutDefault`) and its reads in
|
|
`internal/llm/openai_compatible_client.go`
|
|
(`NewOpenAICompatibleClient` and `OpenAICompatibleClient.Generate`)
|
|
- **Contract at issue:** Framework defaults are fixed, application-neutral
|
|
policy. The architecture requires explicit dependencies rather than hidden
|
|
process-global state, and engine instances must not acquire behavior from a
|
|
writable package variable.
|
|
- **Evidence:** `LLMRequestTimeoutDefault` is declared as an exported package
|
|
`var`, although `10 * time.Minute` is a constant expression and every other
|
|
scalar in the defaults package is a constant. The model client reads this
|
|
binding when it constructs a default or cloned HTTP client and again when a
|
|
zero-valued internal client needs an HTTP client. A complete repository
|
|
search found no writer, setter, or documented mutability contract; the
|
|
binding is currently writable state solely because of its declaration.
|
|
- **Failure mode:** A later internal package or test can reassign the timeout
|
|
and silently change clients constructed afterward. A concurrent write can
|
|
also race with engine construction or the zero-value fallback, making a
|
|
nominally immutable framework default engine-order-dependent. Assigning a
|
|
non-positive duration would remove the intended whole-request cap.
|
|
- **Recommended direction:** Represent the timeout as a constant, preserving
|
|
its value and existing transport semantics. Do not add a setter or a test
|
|
that mutates the default; compile-time immutability is the stronger and
|
|
cheaper invariant.
|
|
- **Required verification:** Run the focused model-client construction and
|
|
deadline tests plus the race-enabled package suite. Confirm that all timeout
|
|
consumers still compile and that no assignment relied on the former
|
|
writable binding.
|
|
|
|
### Unresolved Observations
|
|
|
|
None. The OpenAI-compatible completion path is stored with other framework
|
|
constants but is consumed only by the model client; its placement does not
|
|
introduce mutable state or a competing rule. Actual URL and header construction
|
|
remains Stage 14 scope.
|
|
|
|
### Coverage Ledger
|
|
|
|
- **Registry construction and collisions:** `NewRegistry` builds one private
|
|
map from the built-in OpenRouter definition followed by consumer additions.
|
|
IDs are trimmed once, remain case-sensitive, and are checked after
|
|
normalization for both built-in and consumer collisions. Construction is
|
|
failure-atomic and publishes no partially populated registry. The public
|
|
option merely translates fields and copies the queue-capacity pointer;
|
|
validation has one owner in the registry.
|
|
- **Lookup and immutable snapshots:** The registry exposes no mutation or
|
|
enumeration API. `GetBackend` returns a fresh deep copy of extra parameters,
|
|
and `CapacityPolicies` creates a fresh map of scalar policy values. Nil
|
|
receivers return a not-found error or an empty policy map rather than
|
|
panicking. Focused tests mutate caller inputs, returned nested maps, and
|
|
returned policy maps and demonstrate isolation across lookups.
|
|
- **Endpoint and credential metadata:** Backend endpoints are trimmed and must
|
|
be absolute HTTP or HTTPS URLs with a host and without credentials, query
|
|
text, or fragments. Credential environment names are optional, trimmed, and
|
|
restricted to the documented portable identifier form. Existing package and
|
|
root integration tests cover rejected endpoint classes, invalid environment
|
|
names, trimmed values, and ordinary HTTP and HTTPS endpoints. Backend values
|
|
expose no arbitrary header map, so there is no registry-owned header state
|
|
to validate or copy; authorization and content-type construction belong to
|
|
Stage 14.
|
|
- **Parameter validation:** Empty and reserved top-level parameter keys are
|
|
rejected before registration. The registry consumes
|
|
`llm.IsReservedOpenAIChatRequestField`, while the model client owns and tests
|
|
the complete reserved-field list against its actual top-level payload. This
|
|
preserves dependency direction and gives the registry one representative
|
|
integration case rather than duplicating the transport's list. Recursive
|
|
validation and copying remain owned by `internal/jsonvalue`; S05-F02 and
|
|
S05-F03 already record its numeric inconsistency and missing traversal bound
|
|
and were not repeated here.
|
|
- **Capacity policy normalization:** Negative limits and queue capacities,
|
|
queues on unlimited backends, and overflowing total capacities are rejected.
|
|
A positive limit with no explicit queue receives the documented capacity,
|
|
while explicit zero is retained and unlimited backends produce no policy.
|
|
Registry policy extraction is correct; permit acquisition, fairness,
|
|
cancellation, and runtime bounds remain Stage 15 scope.
|
|
- **Default ownership:** The default execution target contains only the
|
|
documented 600-second framework baseline; all optional provider controls
|
|
remain unspecified. Schema, artifact-name, media-type, timeout, and
|
|
OpenAI-compatible path constants are application-neutral library values,
|
|
while OpenRouter capacity and connection defaults correctly remain with the
|
|
backend registry. The only mutable-default concern is S06-F01.
|
|
- **Built-in backend and profile consistency:** All 24 embedded profiles load
|
|
through the ordinary repository, have unique nonblank IDs, select the exact
|
|
`openrouter` registry ID, and omit endpoint, API-key environment, and raw
|
|
API-key fields. Their IDs and model values match the canonical catalog in
|
|
`docs/formats.md`. Consumer profiles may intentionally override matching
|
|
built-in profile IDs through repository precedence, whereas consumer
|
|
backends may not replace the reserved built-in backend ID.
|
|
- **Test ownership and cost:** Backend package tests own registry validation,
|
|
exact operational OpenRouter policy, copies, lookup errors, and capacity
|
|
snapshots. Built-in repository tests own embedded-catalog validity and
|
|
backend linkage. LLM tests own the reserved request-field list, and root
|
|
tests retain only representative public assembly and copy behavior. No
|
|
material redundant validation matrix or missing registry boundary test was
|
|
found.
|
|
|
|
### Verification Performed
|
|
|
|
The code knowledge graph was used to inventory the three scoped packages,
|
|
trace registry and built-in repository assembly, find all consumers of the
|
|
defaults, and confirm that the reserved request-field function has exactly the
|
|
registry normalizer and provider payload builder as callers. Important
|
|
conclusions were confirmed against complete source, tests, embedded profiles,
|
|
and canonical documentation.
|
|
|
|
The following focused commands passed:
|
|
|
|
```sh
|
|
go test -cover ./internal/backend ./internal/defaults ./internal/profile/builtin
|
|
go test ./internal/llm -run 'TestOpenAICompatibleClientRejectsInvalidExtraParamsBeforeProviderCall|TestNewOpenAICompatibleClientDoesNotMutateSupplied(Nonzero|Zero)TimeoutClient|TestNewOpenAICompatibleClientTreatsSuppliedNegativeTimeoutAsUnset'
|
|
go test ./internal/usecase -run 'Test(ResolveExecutionTargetUsesBackendProfileAndRequestPrecedence|ResolveExecutionTargetDefaultsAndProfileZeros|RunnerInspectProfileResolvesProfileAndBackendOnce)'
|
|
go test . -run 'Test(BackendOptionsAccumulateAndRegistrationsAreEngineLocal|BackendRegistrationRejectsInvalidAndDuplicateDefinitions|BackendExtraParamsAreDeeplyCopiedAtConstructionAndLookup|WithBackendCopiesQueueCapacity|PrepareUsesBuiltInProfileWithoutProfileDir|CustomProfileOverridesBuiltInProfile|RunUsesResolvedBackendWithBuiltInLLMClient|EngineExecutionSettingPrecedence)$'
|
|
go test -race ./internal/backend ./internal/profile/builtin -count=3
|
|
go vet ./internal/backend ./internal/defaults ./internal/profile/builtin
|
|
```
|
|
|
|
The coverage diagnostic reported 94.0% statement coverage for
|
|
`internal/backend`, 100.0% for `internal/profile/builtin`, and no direct test
|
|
coverage for `internal/defaults`. Coverage alone was not treated as a finding:
|
|
the defaults are exercised through the consuming use-case, model-client,
|
|
artifact, validator, and root contract tests, and a separate test of constant
|
|
declarations would add no behavioral protection.
|
|
|
|
### Handoff
|
|
|
|
- The Stage 0 baseline remains absent and was not backfilled during this
|
|
registry and defaults review.
|
|
- Stage 8 owns general filesystem profile parsing, validation, discovery, and
|
|
overlay mechanics. This stage established only that the embedded catalog
|
|
supplies valid ordinary profiles tied to the built-in backend.
|
|
- Stage 14 owns completion-URL composition, authentication and content-type
|
|
headers, payload merging and serialization, deadline behavior, response
|
|
handling, and transport resources. It should treat the shared reserved-field
|
|
ownership recorded here as established.
|
|
- Stage 15 owns admission, permit scheduling, queue behavior, fairness, and
|
|
cancellation. It should treat the registry's normalized immutable capacity
|
|
snapshot as established input.
|