Files
promptkit/docs/roadmap/remediation.md

13 KiB
Raw Blame History

Prompt-Owned Schema Review Findings

Reviewed commit e9a12d4 against all eight stages of implementation.md and the accepted feature roadmap.

1. P2 — Inactive Runtime Schema Values Break Snapshot Ownership

Status: complete. Relevant stages: 1 and 6.

Resolution: inactive runtime inline schemas are omitted from execution-owned contracts, without validating them. Public regressions cover struct/map, cyclic, and unencodable values across inactive modes, preserved legacy path metadata, and independent prepared details. Configured inspection and active schemas retain their existing ownership behavior.

Locations:

  • internal/usecase/runner.go:688: resolveOutputContract retains the caller's raw Schema value outside JSON Schema mode.
  • internal/jsonvalue/trusted.go:20: cloneTrustedReflect recursively copies maps, slices, pointers, and arrays, but returns struct values unchanged.
  • convert.go:339: public output-contract projections rely on that copier.

Evidence And Impact

A public-API reproduction supplies this JSON-encodable Go value in a runtime replacement using ValidationNone:

type schemaValue struct {
    Properties map[string]any `json:"properties"`
}
original := schemaValue{Properties: map[string]any{"title": "original"}}

After successful PrepareExecution, mutating handle.Details().OutputContract.Schema.(schemaValue).Properties["title"] changes both original.Properties and the value returned by a later Details(). The reproduction printed details mutation for both values. A caller changing its original map after preparation likewise changes retained details.

This violates the existing caller-owned Details/frozen-snapshot contract. The schema is inactive, so this reproduction does not change output validation; it still exposes shared mutable state and can introduce races when callers rely on the documented ownership boundary. Active runtime schemas take the JSON encoding/decoding path and do not exhibit this particular problem.

Required Outcome And Refactoring Opportunity

Ensure accepted inactive runtime schema inputs cannot create aliases between caller storage, returned details/notifications, and retained state. Preserve the accepted rule that inactive schema values are ignored: do not fix this by adding schema-validation or JSON-encodability requirements to non-schema modes.

The simplest approach worth considering is to omit the new inactive runtime Schema value from execution-owned snapshots. Keep configured-resource inspection's declared schema metadata and the established inactive SchemaPath behavior separate. Alternatively, if inactive inline values are retained in prepared projections, their copying must actually satisfy the ownership contract for accepted inputs, including JSON-encodable structs with mutable fields. Avoid expanding this into a general arbitrary-Go-object cloning API.

The new TestInactiveSchemaDoesNotInterpretCyclicValue also asserts preservation of a cyclic map's graph topology. Ignoring inactive schema values was requested; preserving arbitrary inactive object graphs was not. That incidental assertion should not prevent the simpler solution. Keep a test for successful handling of inactive cyclic/unencodable values, without imposing graph-preservation behavior unless deliberately adopted.

Add a focused public regression for the struct/map case: mutations of caller input and returned details must not change retained details. If inactive schema values are omitted, assert omission and successful preparation instead. Retain the existing active-schema ownership coverage.

2. P2 — Keep Lightweight Prompt Selection Independent Of Inline Schemas

Status: complete. Identified by the follow-up integration audit. Relevant stages: 1, 3, and 5.

Resolution: removed the output contract from the internal lightweight selection type and its fixture/projection paths. Selection no longer visits schema trees; the full resource and execution views retain their schema data. Focused selection, requirements, smoke, and schema tests pass.

Locations:

  • internal/configuration/prompts.go:147: the internal requirement-selection snapshot includes the complete output contract.
  • internal/configuration/prompts.go:351: ownPromptInspection now calls ownOutputContract, which deep-copies the inline schema.
  • internal/usecase/requirements_inspection.go:38: dependency inspection obtains this snapshot to select the prompt and its default profile.
  • internal/usecase/smoke_selection.go:41: prompt-selected diagnostics use the same metadata lookup, independently of their private diagnostic schema.

Evidence And Impact

The production path is resolvePromptInspection → GetPromptInspection → ownPromptInspection → ownOutputContract → CloneTrusted(schema). InspectRequirements consumes only PromptID and DefaultProfileID from that selection; its input/template analysis separately uses GetRequirementSnapshot. Smoke target selection likewise consumes only prompt identity/default profile. Neither caller reads the selected prompt's output contract.

Before this feature, copying that output contract copied scalar fields. With an inline schema of S nodes/container entries, every such lookup now adds O(S) time and container allocations, in addition to its previous input-metadata copy. A large schema annotation is enough to impose this cost without changing any template requirement or diagnostic selection. File-backed schemas do not impose an equivalent document-copy cost on these paths, creating an avoidable difference between otherwise equivalent schema forms. No timing measurements are needed to establish the extra traversal.

Required Outcome

Keep internal identity/default-profile selection separate from output-schema payloads. Narrow the selection projection or omit unused output-contract data there, rather than copying a full schema to satisfy ownership of a value that neither caller needs. Preserve full, detached schema metadata for the public InspectPrompt resource view and for actual execution preparation. Do not introduce a schema-size limit or weaken snapshot ownership as a workaround.

Retain existing requirements and smoke-selection behavioral tests; verify both paths still select the same prompt/profile and that application schema files remain unopened. Prefer a narrow internal value boundary over a benchmark gate or tests coupled to incidental copier call counts.

3. P3 — Scope The Existing Request-Value Limit Documentation

Status: open. Identified by the follow-up integration audit. Relevant stages: 1, 6, and 8.

types.go:82–86 still says generically that excessively deep or large JSON-shaped request values are rejected. The new OutputContract.Schema route deliberately uses standard JSON encoding/decoding and trusted document copying, without the provider-extra copier's depth/node bounds. The format reference already describes that exemption under Schemas.

An offline public-API check successfully prepared a runtime inline schema with an examples array containing 100,001 null entries. That exceeds the bounded copier's 100,000-node budget but is correctly accepted by the schema path. The code follows the accepted design; the older broad GoDoc no longer accurately describes all JSON-shaped request values after this addition.

Update the RunRequest GoDoc to scope the existing limits to the request fields that actually use them, and distinguish inline schema documents. Preserve the general ownership statements. Do not add bounds to schemas to make the old wording true. This is a documentation correction, not a request for new limit options, schema guarantees, or additional runtime validation.

4. P3 — Allocate Trusted Slice Copies By Length, Not Spare Capacity

Status: open; efficiency improvement for consideration. Identified by the algorithmic audit. Relevant stages: 1 and 6.

Location: internal/jsonvalue/trusted.go:50 in cloneTrustedReflect.

Complexity And Evidence

The revised copier allocates slices with:

reflect.MakeSlice(value.Type(), value.Len(), value.Cap())

It then copies only value.Len() elements. For a slice of length N and capacity C, the backing allocation reserves O(C) element slots although the value being detached contains only N elements. C can be arbitrarily larger than N, so this allocation is not bounded by the logical slice contents. Repeated prepared-detail or notification projections can repeat that excess allocation.

Before this sprint, the []any fast path used make([]any, len(v)) and the []string/[]byte paths copied populated elements without reproducing arbitrary spare capacity. Removing those paths routes them through the capacity-preserving reflection branch. In particular, currently accepted inactive runtime schema values can carry a caller-allocated slice directly into this path.

A small public-API structural check prepared an inactive Schema containing a []any with length 1 and capacity 32. Its detached Details() copy also had length 1 and capacity 32, confirming that unused capacity is allocated again. This was a value-shape check, not a timing, profiling, or memory-usage benchmark.

Refactoring Opportunity

Allocate detached slice storage from the populated length, e.g. reflect.MakeSlice(value.Type(), value.Len(), value.Len()), and retain the existing element-copy behavior. This reduces backing-storage space to O(N), while element traversal remains O(N). Preserve concrete slice types, contents, nil-versus-empty behavior, and independence from caller storage. Spare capacity is not a requested public contract; do not add one.

This is distinct from finding 1: the example's elements are already detached correctly, but too much backing storage is allocated. Omitting inactive schema values may remove this particular public entry path; restoring/simplifying the shared copier as part of that fix can also address the allocation regression. Do not introduce schema limits or a general object-graph facility to solve it.

Use focused internal slice-copy coverage for small populated/empty slices with large spare capacity, alongside existing ownership and nil/empty tests. No new benchmark gate or public exact-capacity guarantee is needed.

Review Coverage And Validation

The initial implementation review covered configuration/root capture, effective source context, selection/clearing, shared compilation, request replacement, provider/repair integration, smoke isolation, mixed collections, and documentation migration without identifying another actionable finding in that pass.

On the reviewed commit, using Go 1.27.4 with GOTOOLCHAIN=local and GOWORK=off:

  • go test -race ./... passed, including the maintained offline examples.
  • go vet ./... passed.
  • The separate public-API reproduction above confirmed the ownership bug not covered by the passing suite.

This review changes no implementation code.

Follow-Up Integration Audit

Reviewed the same implementation code at 3bc16bf (the preceding commit added only this findings document). All eight stages were revisited with particular attention to their existing callers and data projections. Findings 2 and 3 are additional to the still-open ownership finding, not duplicates of it.

An additional offline public-API diagnostic exercised detached OnPrepared and injected-client schemas, a repaired file-backed run, an inline replacement during Coordinate, a prepared run after source removal, and a later ordinary preparation failure. Those interactions passed. It also confirmed acceptance of the large inline annotation described in finding 3. The earlier race-suite/vet results apply to the unchanged implementation; no live provider calls or performance measurements were used.

Algorithmic Audit

Reviewed the unchanged implementation at 4120d78 against the eight stages. Finding 4 is additional; finding 2 already records the avoidable schema-sized lookup traversal and is not duplicated here.

Root capture uses a shared metadata snapshot and indexed root selection rather than rescanning configuration per prompt. New schema selection/binding has no growing nested scan. File reading grows its buffer amortized-linearly, and decoded documents take linear copy passes through the selected ownership paths. Prepared execution and repair reuse the captured plan instead of multiplying schema reads/compilation by the number of attempts. Fixed test/example fixtures add no unbounded algorithmic workload.

The expressly selected JSON conversion, existing schema-location/portability checks and compiler behavior, and existing hash algorithms were not reopened. No additional ordinary correctness issue was found beyond the recorded findings. No real-world performance data, timing measurements, or profiling were collected; the earlier successful suite/vet results apply to unchanged implementation code.