256 lines
13 KiB
Markdown
256 lines
13 KiB
Markdown
# Prompt-Owned Schema Review Findings
|
||
|
||
Reviewed commit `e9a12d4` against all eight stages of
|
||
[implementation.md](implementation.md) and the accepted
|
||
[feature roadmap](prompt-owned-schemas.md).
|
||
|
||
## 1. P2 — Inactive Runtime Schema Values Break Snapshot Ownership
|
||
|
||
Status: complete. Relevant stages: 1 and 6.
|
||
|
||
Resolution: inactive runtime inline schemas are omitted from execution-owned
|
||
contracts, without validating them. Public regressions cover struct/map, cyclic,
|
||
and unencodable values across inactive modes, preserved legacy path metadata,
|
||
and independent prepared details. Configured inspection and active schemas retain
|
||
their existing ownership behavior.
|
||
|
||
Locations:
|
||
|
||
- `internal/usecase/runner.go:688`: `resolveOutputContract` retains the caller's
|
||
raw `Schema` value outside JSON Schema mode.
|
||
- `internal/jsonvalue/trusted.go:20`: `cloneTrustedReflect` recursively copies
|
||
maps, slices, pointers, and arrays, but returns struct values unchanged.
|
||
- `convert.go:339`: public output-contract projections rely on that copier.
|
||
|
||
### Evidence And Impact
|
||
|
||
A public-API reproduction supplies this JSON-encodable Go value in a runtime
|
||
replacement using `ValidationNone`:
|
||
|
||
```go
|
||
type schemaValue struct {
|
||
Properties map[string]any `json:"properties"`
|
||
}
|
||
original := schemaValue{Properties: map[string]any{"title": "original"}}
|
||
```
|
||
|
||
After successful `PrepareExecution`, mutating
|
||
`handle.Details().OutputContract.Schema.(schemaValue).Properties["title"]`
|
||
changes both `original.Properties` and the value returned by a later `Details()`.
|
||
The reproduction printed `details mutation` for both values. A caller changing
|
||
its original map after preparation likewise changes retained details.
|
||
|
||
This violates the existing caller-owned `Details`/frozen-snapshot contract.
|
||
The schema is inactive, so this reproduction does not change output validation;
|
||
it still exposes shared mutable state and can introduce races when callers rely
|
||
on the documented ownership boundary. Active runtime schemas take the JSON
|
||
encoding/decoding path and do not exhibit this particular problem.
|
||
|
||
### Required Outcome And Refactoring Opportunity
|
||
|
||
Ensure accepted inactive runtime schema inputs cannot create aliases between
|
||
caller storage, returned details/notifications, and retained state. Preserve the
|
||
accepted rule that inactive schema values are ignored: do not fix this by adding
|
||
schema-validation or JSON-encodability requirements to non-schema modes.
|
||
|
||
The simplest approach worth considering is to omit the new inactive runtime
|
||
`Schema` value from execution-owned snapshots. Keep configured-resource
|
||
inspection's declared schema metadata and the established inactive `SchemaPath`
|
||
behavior separate. Alternatively, if inactive inline values are retained in
|
||
prepared projections, their copying must actually satisfy the ownership contract
|
||
for accepted inputs, including JSON-encodable structs with mutable fields.
|
||
Avoid expanding this into a general arbitrary-Go-object cloning API.
|
||
|
||
The new `TestInactiveSchemaDoesNotInterpretCyclicValue` also asserts preservation
|
||
of a cyclic map's graph topology. Ignoring inactive schema values was requested;
|
||
preserving arbitrary inactive object graphs was not. That incidental assertion
|
||
should not prevent the simpler solution. Keep a test for successful handling of
|
||
inactive cyclic/unencodable values, without imposing graph-preservation behavior
|
||
unless deliberately adopted.
|
||
|
||
Add a focused public regression for the struct/map case: mutations of caller
|
||
input and returned details must not change retained details. If inactive schema
|
||
values are omitted, assert omission and successful preparation instead. Retain
|
||
the existing active-schema ownership coverage.
|
||
|
||
## 2. P2 — Keep Lightweight Prompt Selection Independent Of Inline Schemas
|
||
|
||
Status: complete. Identified by the follow-up integration audit. Relevant stages:
|
||
1, 3, and 5.
|
||
|
||
Resolution: removed the output contract from the internal lightweight selection
|
||
type and its fixture/projection paths. Selection no longer visits schema trees;
|
||
the full resource and execution views retain their schema data. Focused selection,
|
||
requirements, smoke, and schema tests pass.
|
||
|
||
Locations:
|
||
|
||
- `internal/configuration/prompts.go:147`: the internal requirement-selection
|
||
snapshot includes the complete output contract.
|
||
- `internal/configuration/prompts.go:351`: `ownPromptInspection` now calls
|
||
`ownOutputContract`, which deep-copies the inline schema.
|
||
- `internal/usecase/requirements_inspection.go:38`: dependency inspection obtains
|
||
this snapshot to select the prompt and its default profile.
|
||
- `internal/usecase/smoke_selection.go:41`: prompt-selected diagnostics use the
|
||
same metadata lookup, independently of their private diagnostic schema.
|
||
|
||
### Evidence And Impact
|
||
|
||
The production path is `resolvePromptInspection` → `GetPromptInspection` →
|
||
`ownPromptInspection` → `ownOutputContract` → `CloneTrusted(schema)`.
|
||
`InspectRequirements` consumes only `PromptID` and `DefaultProfileID` from that
|
||
selection; its input/template analysis separately uses `GetRequirementSnapshot`.
|
||
Smoke target selection likewise consumes only prompt identity/default profile.
|
||
Neither caller reads the selected prompt's output contract.
|
||
|
||
Before this feature, copying that output contract copied scalar fields. With an
|
||
inline schema of S nodes/container entries, every such lookup now adds O(S) time
|
||
and container allocations, in addition to its previous input-metadata copy.
|
||
A large schema annotation is enough to impose this cost without changing any
|
||
template requirement or diagnostic selection. File-backed schemas do not impose
|
||
an equivalent document-copy cost on these paths, creating an avoidable difference
|
||
between otherwise equivalent schema forms. No timing measurements are needed to
|
||
establish the extra traversal.
|
||
|
||
### Required Outcome
|
||
|
||
Keep internal identity/default-profile selection separate from output-schema
|
||
payloads. Narrow the selection projection or omit unused output-contract data
|
||
there, rather than copying a full schema to satisfy ownership of a value that
|
||
neither caller needs. Preserve full, detached schema metadata for the public
|
||
`InspectPrompt` resource view and for actual execution preparation. Do not
|
||
introduce a schema-size limit or weaken snapshot ownership as a workaround.
|
||
|
||
Retain existing requirements and smoke-selection behavioral tests; verify both
|
||
paths still select the same prompt/profile and that application schema files
|
||
remain unopened. Prefer a narrow internal value boundary over a benchmark gate
|
||
or tests coupled to incidental copier call counts.
|
||
|
||
## 3. P3 — Scope The Existing Request-Value Limit Documentation
|
||
|
||
Status: complete. Identified by the follow-up integration audit. Relevant stages:
|
||
1, 6, and 8.
|
||
|
||
Resolution: RunRequest GoDoc now scopes bounded copying to Execution.ExtraParams
|
||
and explicitly distinguishes inline schema documents. No runtime limits changed.
|
||
|
||
`types.go:82–86` still says generically that excessively deep or large
|
||
JSON-shaped request values are rejected. The new `OutputContract.Schema` route
|
||
deliberately uses standard JSON encoding/decoding and trusted document copying,
|
||
without the provider-extra copier's depth/node bounds. The format reference
|
||
already describes that exemption under Schemas.
|
||
|
||
An offline public-API check successfully prepared a runtime inline schema with
|
||
an `examples` array containing 100,001 null entries. That exceeds the bounded
|
||
copier's 100,000-node budget but is correctly accepted by the schema path.
|
||
The code follows the accepted design; the older broad GoDoc no longer accurately
|
||
describes all JSON-shaped request values after this addition.
|
||
|
||
Update the `RunRequest` GoDoc to scope the existing limits to the request fields
|
||
that actually use them, and distinguish inline schema documents. Preserve the
|
||
general ownership statements. Do not add bounds to schemas to make the old
|
||
wording true. This is a documentation correction, not a request for new limit
|
||
options, schema guarantees, or additional runtime validation.
|
||
|
||
## 4. P3 — Allocate Trusted Slice Copies By Length, Not Spare Capacity
|
||
|
||
Status: open; efficiency improvement for consideration. Identified by the
|
||
algorithmic audit. Relevant stages: 1 and 6.
|
||
|
||
Location: `internal/jsonvalue/trusted.go:50` in `cloneTrustedReflect`.
|
||
|
||
### Complexity And Evidence
|
||
|
||
The revised copier allocates slices with:
|
||
|
||
```go
|
||
reflect.MakeSlice(value.Type(), value.Len(), value.Cap())
|
||
```
|
||
|
||
It then copies only `value.Len()` elements. For a slice of length N and capacity
|
||
C, the backing allocation reserves O(C) element slots although the value being
|
||
detached contains only N elements. C can be arbitrarily larger than N, so this
|
||
allocation is not bounded by the logical slice contents. Repeated prepared-detail
|
||
or notification projections can repeat that excess allocation.
|
||
|
||
Before this sprint, the `[]any` fast path used `make([]any, len(v))` and the
|
||
`[]string`/`[]byte` paths copied populated elements without reproducing arbitrary
|
||
spare capacity. Removing those paths routes them through the capacity-preserving
|
||
reflection branch. In particular, currently accepted inactive runtime schema
|
||
values can carry a caller-allocated slice directly into this path.
|
||
|
||
A small public-API structural check prepared an inactive `Schema` containing a
|
||
`[]any` with length 1 and capacity 32. Its detached `Details()` copy also had
|
||
length 1 and capacity 32, confirming that unused capacity is allocated again.
|
||
This was a value-shape check, not a timing, profiling, or memory-usage benchmark.
|
||
|
||
### Refactoring Opportunity
|
||
|
||
Allocate detached slice storage from the populated length, e.g.
|
||
`reflect.MakeSlice(value.Type(), value.Len(), value.Len())`, and retain the
|
||
existing element-copy behavior. This reduces backing-storage space to O(N),
|
||
while element traversal remains O(N). Preserve concrete slice types, contents,
|
||
nil-versus-empty behavior, and independence from caller storage. Spare capacity
|
||
is not a requested public contract; do not add one.
|
||
|
||
This is distinct from finding 1: the example's elements are already detached
|
||
correctly, but too much backing storage is allocated. Omitting inactive schema
|
||
values may remove this particular public entry path; restoring/simplifying the
|
||
shared copier as part of that fix can also address the allocation regression.
|
||
Do not introduce schema limits or a general object-graph facility to solve it.
|
||
|
||
Use focused internal slice-copy coverage for small populated/empty slices with
|
||
large spare capacity, alongside existing ownership and nil/empty tests. No new
|
||
benchmark gate or public exact-capacity guarantee is needed.
|
||
|
||
## Review Coverage And Validation
|
||
|
||
The initial implementation review covered configuration/root capture, effective
|
||
source context, selection/clearing, shared compilation, request replacement,
|
||
provider/repair integration, smoke isolation, mixed collections, and documentation
|
||
migration without identifying another actionable finding in that pass.
|
||
|
||
On the reviewed commit, using Go 1.27.4 with `GOTOOLCHAIN=local` and `GOWORK=off`:
|
||
|
||
- `go test -race ./...` passed, including the maintained offline examples.
|
||
- `go vet ./...` passed.
|
||
- The separate public-API reproduction above confirmed the ownership bug not
|
||
covered by the passing suite.
|
||
|
||
This review changes no implementation code.
|
||
|
||
### Follow-Up Integration Audit
|
||
|
||
Reviewed the same implementation code at `3bc16bf` (the preceding commit added
|
||
only this findings document). All eight stages were revisited with particular
|
||
attention to their existing callers and data projections. Findings 2 and 3 are
|
||
additional to the still-open ownership finding, not duplicates of it.
|
||
|
||
An additional offline public-API diagnostic exercised detached `OnPrepared` and
|
||
injected-client schemas, a repaired file-backed run, an inline replacement during
|
||
`Coordinate`, a prepared run after source removal, and a later ordinary
|
||
preparation failure. Those interactions passed. It also confirmed acceptance of
|
||
the large inline annotation described in finding 3. The earlier race-suite/vet
|
||
results apply to the unchanged implementation; no live provider calls or
|
||
performance measurements were used.
|
||
|
||
### Algorithmic Audit
|
||
|
||
Reviewed the unchanged implementation at `4120d78` against the eight stages.
|
||
Finding 4 is additional; finding 2 already records the avoidable schema-sized
|
||
lookup traversal and is not duplicated here.
|
||
|
||
Root capture uses a shared metadata snapshot and indexed root selection rather
|
||
than rescanning configuration per prompt. New schema selection/binding has no
|
||
growing nested scan. File reading grows its buffer amortized-linearly, and
|
||
decoded documents take linear copy passes through the selected ownership paths.
|
||
Prepared execution and repair reuse the captured plan instead of multiplying
|
||
schema reads/compilation by the number of attempts. Fixed test/example fixtures
|
||
add no unbounded algorithmic workload.
|
||
|
||
The expressly selected JSON conversion, existing schema-location/portability
|
||
checks and compiler behavior, and existing hash algorithms were not reopened.
|
||
No additional ordinary correctness issue was found beyond the recorded findings.
|
||
No real-world performance data, timing measurements, or profiling were collected;
|
||
the earlier successful suite/vet results apply to unchanged implementation code.
|