diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md deleted file mode 100644 index 88a6512..0000000 --- a/docs/roadmap/implementation.md +++ /dev/null @@ -1,414 +0,0 @@ -# Prompt-Owned Schemas Implementation Plan - -## Objective And Authority - -Implement [prompt-owned-schemas.md](prompt-owned-schemas.md). That feature -roadmap owns the accepted selection, composition, scope, ownership, and migration -decisions. The [consumer request](prompt-owned-schemas-request.md) is background, -not authority for application-specific behavior. Respect -[ADR-0001](../adr/0001-configuration-boundary.md): interpret only ConfigKit's final -effective configuration and its effective resource context. - -The baseline is commit `adfccb5`, with ConfigKit v1.3.0 and the prepared -message-content hash. This is one end-to-end sprint, divided into sequential -single-prompt stages for a Luna coding agent. Intermediate stages may fail to -compile. Do not add compatibility shims or duplicate implementations merely to -make an intermediate commit coherent. No stage requires human approval, external -publication, or a dependency upgrade. Commit completed units under repository -policy; commits are not review gates. - -Start with [development.md](../development.md) and the applicable -[agent](../policy/agents.md), [architecture](../policy/architecture.md), -[testing](../policy/testing.md), and [documentation](../policy/documentation.md) -policies. Read the [format reference](../formats.md), -[source/validation internals](../internal/sources.md), and existing GoDoc before -changing their contracts. Reuse existing file-path, schema compilation, -cancellation, and ownership machinery where applicable. - -## Concrete Design For This Sprint - -### Public Values And Configuration - -- Add `Schema any` with `json:"schema,omitempty"` to public `OutputContract`. - Nil means no inline selection; a nonnil empty object and `false` are selections. - Preserve `SchemaPath string` and its existing JSON field. Do not add runtime - `SchemaFile`, filesystem handles, or a standalone schema-loading API. -- Add `Schema any` and `SchemaFile *string` to `PromptOutputConfig`, retaining - `SchemaPath *string`. `PromptSource` emits `schema` only for a nonnull inline - value and emits each nonnil reference pointer. As with existing typed helpers, - raw ConfigKit mappings express explicit null and composition controls. -- Add `Schema any` and `SchemaFile *string` to `PromptOutputInspection`, with - `schema` and `schema_file` JSON/YAML names and `omitempty`. Preserve its existing - path presence. Expose configured values/references, not filesystem capabilities - or file contents. Generic configuration inspection already exposes declared - application and collection roots; no additional public root-inspection API is - needed. -- Decode optional `schema_root` strings at the application root and collection - entries. Absent/null collection roots select the application root; absent/null - application roots mean no root. A nonnull blank root is invalid. An explicitly - supplied `"."` is valid when accompanied by usable source context. -- In JSON Schema mode, exactly one nonnull effective `schema`, `schema_file`, or - `schema_path` is required. Reject blank file/path selections and conflicts. - Outside that mode, do not resolve, compile, or impose new selection checks on - unused schema fields. Retain ordinary ConfigKit decoding semantics. - -Use native ConfigKit values for captured configuration. For active runtime -inline values, use standard JSON encoding and the existing schema JSON decoder -to obtain an operation-owned document; do not introduce a custom numeric bridge -or the bounded provider-extra copier. JSON null, including typed nil values -encoded as null, means no selection. Empty objects and booleans must survive. -Unsupported/cyclic Go values that cannot be encoded fail as invalid request -values, not panics. JSON-compatible but invalid schema documents fail compilation -at preparation through the existing validation-error boundary. Preserve existing -file JSON decoding and its numeric behavior; promise no new cross-input numeric -equivalence. Clone captured, already-supported trees using the existing trusted -document ownership path, without provider-extra depth/node limits. - -### Schema-Root Helper Migration - -Replace the out-of-band root settings with one mapping helper: - -```go -func SchemaRootSource(label, root string) configkit.Source -``` - -It returns `configkit.Mapping(label, map[string]any{"schema_root": root})`, -without opening files or validating raw contributions. Callers place its input -at their chosen native ConfigKit layer and provide a filesystem context, just as -they do for `PromptSource`. For example, for an `fs.FS`: - -```go -filesystem := configkit.NewFilesystem("application", schemaFS) -rootInput := configkit.Input{ - Source: promptkit.SchemaRootSource("application schemas", "schemas"), - Filesystem: &configkit.FilesystemContext{ - FS: &filesystem, Root: ".", BaseDir: ".", - }, -} -// Supply rootInput in configkit.Underlay{Inputs: []configkit.Input{rootInput}}, -// or use &rootInput as Main when no other main input is needed. -``` - -Remove `Config.SchemaDir`, `WithSchemaFS`, and `WithSchemaFile`, and the internal -option setters/flags used solely by those APIs. This is an intentional pre-1.0 -API migration, not a deprecation period. Do not silently translate options into -a highest-priority layer or preserve last-option-wins behavior. Callers needing -collection-specific roots use ordinary collection mappings with context; the -helper need not grow collection-specific parameters. - -Migration: a directory or filesystem becomes a context-backed root mapping; -a former single-file source becomes a `schema_file` on a configured prompt or a -root mapping plus its basename in `schema_path`. For an unrelated per-request -file, the caller reads/decodes it and supplies `Schema` inline. Host-directory -examples use an explicit `os.DirFS`/ConfigKit filesystem context. Do not infer a -working-directory root. This preserves the requested directory/filesystem -capabilities without preserving a second source-selection system. - -### Internal Resource And Preparation Model - -Separate serializable selections from resource capabilities: - -- Extend domain output selections with inline `Schema` and the configured - `SchemaFile` reference string. The latter is internal, not a new request field. -- Capture an optional applicable root and optional source-relative file binding - with the prompt definition. Use small source-neutral domain values containing - an `fs.FS` and resolved filesystem path. Exclude those capabilities from JSON - and definition hashing. Keep the root separate from the output contract so it - remains available to a runtime `SchemaPath` replacement. -- Configuration binding resolves a field's effective filesystem/base directory - using the current effective-origin lookup and `SourceContext`, as `readContent` - does. Extract binding from reading so schema capture does not call `Open`, - `Stat`, or compile a schema to establish file existence. Apply existing - `filecatalog` path containment rules. Missing context/invalid reference metadata - is a configuration error; missing or malformed file data is a preparation error. -- Add internal `validate.Input` containing `Contract domain.OutputContract` and - optional `File` binding. Change `ValidationPreparer.PrepareValidation` to accept - this input. Non-schema modes need no file. JSON Schema mode selects the inline - document or the one bound file and uses one common compilation pipeline. -- After complete request-contract replacement, the runner selects the binding: - configured `schema_file` uses its captured binding only without a replacement; - `SchemaPath` binds under the selected prompt's applicable root; inline values - need no file. A replacement never borrows the configured inline/file selection. - A missing root for a runtime path is an invalid request. Existing public error - translation remains authoritative. - -Internal names may follow local conventions, but implement these responsibilities -once. Do not carry ConfigKit objects into execution, reread configuration, query -history during execution, or serialize filesystem objects. The selected root is -not a list to search. Root selection happens before opening the requested file. - -## Stage 1 — Add Schema Value Shapes And Ownership - -Status: complete. - -Read `types.go`, `prompts.go`, `resource_inspection.go`, `convert.go`, domain -output contracts, and the existing schema-document ownership/decoder helpers. - -1. Add the public/configuration/inspection fields selected above, corresponding - internal fields, and the small resource-binding and validation-input values. - Keep selection data separate from the capabilities used to load it. -2. Extend `PromptSource` and output conversions. Preserve false and empty-object - presence; do not use truthiness or map length to decide selection. -3. Establish one shared output-selection check for the effective active mode. - Separate selection validity from schema compilation. Preserve format and - repair-attempt rules. For typed nil runtime inline values, use the standard - JSON null result before counting selections. -4. Add operation-owned inline decoding using the existing JSON document path. - Reuse trusted cloning after capture/decoding. Avoid an encode/decode round trip - at each getter or conversion; ordinary detached getters clone supported trees. -5. Update shallow output-contract/definition copies that now contain mutable - schema values. Do not change schema ownership by accidentally routing these - values through the provider-extra copier. - -Tests: extend focused domain selection and typed-source tests for nil/null, -`false`, `true`, `{}`, conflicting selections, blank references, and inactive -mode behavior. Add a focused runtime inline encoding failure case. Do not run -the full suite while downstream signatures are intentionally incomplete. - -Completion: representations and ownership rules are implemented, with no public -runtime filesystem API or truthiness-based loss of schemas. - -## Stage 2 — Capture Roots Through Native Configuration Inputs - -Status: complete. - -Work in `internal/configuration/load.go`, its normalization/capture helpers, -the new root source helper, `engine.go`, and internal engine construction. - -1. Add optional roots to decoded application/collection records. Validate only - final effective root values and obtain each root's effective source context. - Build application and collection root bindings once during construction, - using the existing shared metadata snapshot rather than scanning the full - resolved tree per prompt. -2. Capture the applicable root with each prompt under the accepted collection - scope rule. A declared collection root wins even if a subsequently selected - file is missing there. Null clearing restores application-root selection. -3. Implement `SchemaRootSource` and remove the legacy public and internal - schema-source configuration paths described above, including the implicit - default. Remove now-unused schema-specific defaults/imports, not unrelated - artifact-source machinery. -4. Wire the engine toward a source-neutral validator receiving operation inputs, - rather than constructing a validator tied to one engine-wide root. Update - affected configuration/engine tests and fixtures enough to express the new - root inputs; later stages finish preparation integration. - -Tests: root/collection capture with an explicit mapping context, root override -context from another source, null fallback, blank/missing context diagnostics, -and native layer precedence for the mapping helper. Use a counting filesystem -to demonstrate construction does not inspect schema file existence. - -Completion: root selection and context come solely from the final ConfigKit -configuration. No option-specific precedence or implicit root remains. - -## Stage 3 — Capture Prompt-Owned Schema Selections And References - -Status: complete. - -Work in `internal/configuration/prompts.go`, resource projections, normalization, -and prompt-definition snapshots/hashing. - -1. Extend output capture for all three fields. Validate exactly one effective - active selection, using ConfigKit's existing presence/null data when needed. - Do not reject overwritten contributions or reconstruct native composition. -2. Extract a reusable effective-field resource binder from `readContent`. - Preserve message-file read/close behavior; schema-file capture uses binding - only. Inherited references use their contributing field's context, not the - child's source. Root and file binding use existing path/context rules. -3. For configured active `schema_path`, require an applicable root and validate - path metadata without opening the file. Bind active `schema_file` from that - field's context. Inline selections need neither binding nor root. -4. Extend `PromptOutputResource`, `InspectPrompt`, and public projections to - include selected values/references without opening files. Keep detached - schema trees in public results and preserve existing inspection presence rules. -5. Include configured inline values and file-reference strings in the existing - definition-hash input; exclude resource capabilities. Use the existing hash - algorithm, not a new digest API or file-content read. Root context and schema - file bytes are not newly promised hash inputs. Leave both prepared-message - hash algorithms and their semantics unchanged. - -Tests: imports/inheritance and overrides preserve effective field context; -composition can merge inline mappings and explicitly clear alternatives; -same basenames in different filesystems stay distinct; inactive schema fields -do not trigger resource binding. Inspect a prompt with a missing schema file -successfully, then leave actual file failure for preparation. - -Completion: final selections, source bindings, inspection, and definition -snapshots agree without eager schema I/O or a second composition pass. - -## Stage 4 — Unify Inline And File Schema Preparation - -Status: complete. - -Work in `internal/validate/validator.go` and the current validator implementation. - -1. Adapt the preparer to `validate.Input`. Replace duplicated engine-root-specific - preparation routes with a source-neutral validator. Migrate internal validator - tests to bound files/inline documents; retain useful path/decoder tests at - their owning helper rather than keeping obsolete constructors just for tests. -2. Load a selected file once through the bound filesystem and existing - context-checked chunk reader. Preserve read/close error handling and the - single-complete-JSON-document decoder. Inline values use their captured or - operation-owned document without a file read. -3. Feed both routes through the existing dialect, self-contained-resource, - schema-location, and root compilation checks. Retain supported IDs, anchors, - internal references, filename independence, and external-resource rejection. - Keep existing canonical escaped retrieval URLs for file sources; use a stable - private absolute retrieval URL for inline documents, without new public IDs. -4. Store the selected document and compiled root in the existing prepared plan. - `SchemaDocument()` returning `false` is a document, not missing metadata. - Use the same value for local checks and provider constraints. -5. Preserve synchronous cancellation checks around reads, decoding, compilation, - and validation. No background dependency workers or new cache is needed. - -Tests: object and boolean schemas work by both routes; equivalent documents -give equivalent validation; invalid schema documents/dialects, external refs, -valid internal refs, exact file JSON numbers, and cancellation retain their -existing classifications. Reuse focused tests rather than duplicating the entire -schema suite for every route. - -Completion: one source-neutral preparation/compilation path produces the existing -operation-local plan with no extra resource reads for `$ref` resolution. - -## Stage 5 — Bind Effective Contracts In Every Execution Path - -Status: complete. - -Work in `internal/usecase/runner.go`, prepared execution, engine wiring, and -`smoke_inference_case.go`. - -1. Resolve the complete effective output contract first, then its file binding. - Retain surrounding root scope for runtime `SchemaPath`; discard configured - `schema_file` binding and configured inline value on a request replacement. - Store the selected input in preparation state and pass it to the preparer. -2. Keep malformed selection/root requirements within existing invalid-config - construction or invalid-request runtime boundaries. Read/decode/compile - failures remain operational validation errors before generation. Include - selected prompt/reference context while preserving error identities and - cancellation precedence. -3. Keep `Prepare`, `ValidateRequest`, `Run`, and `PrepareExecution` on the shared - preparation path. `RunPrepared` retains/reuses the plan and needs no schema - binding, file read, or compilation at execution time. Preserve callback, - operation-budget, admission, and continuation behavior. -4. Preserve prompt-ID-based structured-output naming and the existing provider - submission path. Repair reuses the same plan/document; it does not reselect - schema forms or roots. Do not invent provider-support assertions. -5. Migrate private smoke-schema setup to an explicit bound private resource and - the same preparer, independent of application schema roots. Update internal - fake preparers to the new signature; preserve the intentional nil-validator - behavior of focused use-case tests. - -Tests: runtime inline/path replacement, no borrowed configured selection, missing -runtime root, and collection-root selection during replacement. Existing -ordinary/prepared, validation-only, continuation, and smoke tests should compile -and exercise the common pipeline. Verify preparation failures do not call a model. - -Completion: all execution entry points select and retain exactly the intended -schema plan, without an out-of-band engine schema root. - -## Stage 6 — Verify Public Projections And Ownership - -Status: complete. - -Finish public facade conversion and serialization changes in `convert.go`, -`types.go`, `resource_inspection.go`, `json.go`, and prepared notification paths. - -1. Extend existing JSON/YAML contract tests for inline `false`, `{}`, absence, - inspection references, and backward decoding of values omitting new fields. - The prepared JSON projection already inherits public struct fields; avoid a - second serialization DTO. Preserve ordinary manual-value JSON behavior rather - than introducing schema compilation on JSON decode. -2. Test that mutating typed input after its source snapshot, inspection results, - `Prepare` results, `Details`, `OnPrepared`, or injected-client schema metadata - cannot alter captured configuration or retained execution. Extend existing - ownership fixtures rather than duplicating every handle lifecycle test. -3. Keep configured file-reference inspection separate from the schema document - shown in prepared `StructuredOutput`. Public runtime `OutputContract` has no - `SchemaFile`; do not fabricate an inline selection or path merely to fill that - projection. The prepared structured-output document is the selected content. - `ValidationResult.SchemaPath` keeps its explicit-root path meaning and is - empty for the new inline/source-relative forms; no synthetic sentinel path or - new result field is needed. -4. Check every remaining shallow output-contract copy now containing `Schema`. - Trusted internal sharing stays immutable; public copies detach mutable trees. - Do not marshal filesystem bindings, or recompile schemas in inspection/getters. - -Tests: focused typed-source, resource-inspection, prepared/notification, and JSON -contract packages. Retain schema-document tests that exceed provider-extra copy -limits; new inline handling must not route trusted documents through those limits. - -Completion: all exposed values are faithful and caller-owned where the existing -API requires it, with no resource capability leakage or extra schema work. - -## Stage 7 — Verify Mixed Collections And Captured Lifecycles - -Status: complete. - -Add focused external-package integration coverage using real ConfigKit -composition and injected provider/transport boundaries. Prefer a small fixture -with distinct risks over a Cartesian product of settings. - -1. Build one engine with independent collection roots, application-root fallback, - source-relative schemas, and inline schemas together. Mix embedded-like and - operator filesystem contexts, with identical schema basenames. Replace one - prompt/schema selection while unrelated prompts remain usable. -2. Cover inherited source context and overridden context through actual imports - or layers, including a root inherited from a different source than its prompt. - Exercise explicit null when switching forms and ConfigKit's native whole-value - replacement control. No custom overlay behavior belongs in expected results. -3. Confirm missing files under the chosen root fail without another root search. - Missing/malformed schema files permit engine construction but fail preparation; - missing selection/context/root metadata fails at its configured boundary. -4. Capture a prepared execution, then modify/remove its schema source. The handle - executes and repairs against its captured plan; a later ordinary preparation - observes the changed source. Count reads to protect capture behavior, not - incidental compiler-internal call counts. -5. Confirm equivalent forms submit the selected document to an offline recording - client and validate generated results with those same constraints. Include a - boolean-schema case. Preserve transport error behavior rather than testing - assumptions about live provider support. - -Completion: the mixed Scriptorium/Notarius layout and prepared-plan lifecycle are -covered end to end without application-managed mounts or live infrastructure. - -## Stage 8 — Migrate Documentation And Validate The Assembled Sprint - -Status: complete. - -1. Update GoDoc and [formats.md](../formats.md) for selection/null/composition, - source context, root scope, inactive modes, preparation failures, and removed - implicit root. Update the [consumer guide](../consumers/pkg-promptkit.md) with - runtime inline replacement and context-backed root/helper migration examples. - Keep exact API declarations in GoDoc and configuration rules in the format - reference; link rather than repeat them everywhere. -2. Update [sources.md](../internal/sources.md), - [runner.md](../internal/runner.md), and the internal overview where responsibilities - changed. Adapt maintained examples/fixtures referencing removed schema options - to inline, source-relative, or explicit context-backed roots as appropriate. - Keep one representative maintained mixed-source example rather than adding an - application asset assembly tutorial. -3. Make the canonical migration documentation cover positional public struct - literals, removed schema options/field, and the absence of an implicit root. - Release preparation will select the version and summarize these changes in - its release note under the release policy. This sprint selects no release - number and does not rewrite published notes or require a publication check. -4. Audit the complete diff against the feature roadmap. Remove old schema-source - wiring, obsolete helper-only tests/defaults, and stale current-state references. - Historical published notes and the original request are not current API owners. - Do not remove unrelated path/artifact machinery or change dependency versions - or the Go consumer minimum for this feature. -5. Follow the development guide's newest locally installed Go selection with - `GOTOOLCHAIN=local` and `GOWORK=off`. Format changed Go files; run - `go test -race ./...`, `go vet ./...`, and `git diff --check`. The complete suite - includes maintained examples; no repeated example gate or benchmark run is - required. Check changed local documentation links. Fix failures using focused - reruns and validate the final assembled changes proportionately. - -Completion: all stages' behavior and tests are implemented, documentation describes -the new configuration/API migration, and assembled validation succeeds. Do not tag, -push, publish dependencies, or require external consumer updates to finish this -sprint. Retire completed roadmaps during normal release preparation. - -## Open Questions - -None. The three product decisions are accepted in the feature roadmap; this plan -selects the remaining API, helper migration, and preparation plumbing details. diff --git a/docs/roadmap/next-release-considerations.md b/docs/roadmap/next-release-considerations.md index 1bf80e2..33bb82a 100644 --- a/docs/roadmap/next-release-considerations.md +++ b/docs/roadmap/next-release-considerations.md @@ -2,8 +2,8 @@ Status: candidates for discussion, not accepted implementation requirements. Review these items before the next release; inclusion in this list does not -commit them to that release. Schema support implementation continues under the -[prompt-owned schemas roadmap](prompt-owned-schemas.md). +commit them to that release. Implemented schema support is documented in the +[format reference](../formats.md#schemas). ## 1. Enable One Structured-Output Repair Attempt By Default @@ -32,7 +32,7 @@ Before accepting, settle: - Whether to make a clean breaking migration or support a transition period. - How collection imports, bundled defaults, typed configuration helpers, and inspection use the new location through normal ConfigKit composition. -- Migration guidance and coordination with the schema work currently underway. +- Migration guidance accounting for the implemented schema-source changes. ## Further Candidates diff --git a/docs/roadmap/prompt-owned-schemas-request.md b/docs/roadmap/prompt-owned-schemas-request.md deleted file mode 100644 index 615a7c3..0000000 --- a/docs/roadmap/prompt-owned-schemas-request.md +++ /dev/null @@ -1,166 +0,0 @@ -# Feature Request: Prompt-Owned Inline And Source-Relative Schemas - -**Status:** Requested by the Notarius consumer; proposed behavior, not an -implemented API or an approved implementation plan. - -## Requested Capability - -Allow a prompt's structured-output schema to be supplied directly inline in its -output definition or loaded from a file relative to the source that defines the -schema reference. Use the same effective resource-context model already used by -`content_file`, including imports, inheritance, embedded filesystems, and operator -sources. - -The guiding user observation is that a structured-output schema is part of a -prompt definition. Users generally expect it to behave like any other part of -that definition: they can provide it inline or place it in a separate file loaded -relative to the prompt where it is defined. Requiring a separately assembled, -engine-wide schema filesystem makes that ordinary mental model unnecessarily -hard to express. - -## Consumer Problem - -Notarius is moving to native PromptKit declarations, with embedded defaults and -higher-priority operator prompt sources. It wants to remove its asset registries, -synthetic mounts, file copying, flattened schema namespace, and separate schema -identity/digest machinery. - -PromptKit already composes prompt definitions and resolves message files using -their source contexts. Schema paths currently resolve against a separately -selected schema root. Replacing an embedded prompt with an operator prompt can -therefore require changing the schema source for the whole engine, assembling a -custom filesystem, or copying unrelated default schemas into the replacement -root. An inline schema cannot currently express the simpler alternative either. - -The desired arrangement is straightforward: - -```text -embedded/ - spell-extraction.yml - spell-response.json -operator/ - custom-extraction.yml - custom-response.json -``` - -Each prompt references its nearby schema, or embeds that schema directly. -Replacing one prompt leaves unchanged prompts and their schemas usable. PromptKit -owns this reusable resource resolution; Notarius supplies prompt content and -retains domain interpretation and artifact validation. - -## Why The Current Behavior Appears To Exist - -The original public engine facade exposed separate prompt, profile, and schema -sources (see repository history at `e4899fb`). The current -[`WithSchemaFS` option](../../engine.go) still selects one schema source, and -[internal engine wiring](../../internal/engine/engine.go) constructs a validator -bound to that source. - -This suggests an inheritance from the earlier architecture, rather than a -fundamental requirement that every prompt share one schema root. That is an -inference from the implementation and history, not a documented statement of -the original author's intent. A central root remains useful when an application -selects output contracts programmatically; the request is to support ordinary -prompt-owned schemas as well. - -The newer configuration model already supplies the relevant foundation. -[Configuration ownership ADR-0001](../adr/0001-configuration-boundary.md) includes -effective resource context in the ConfigKit handoff, and the -[`content_file` contract](../formats.md#messages-and-templates) resolves each -effective reference using its own source context. Schema references should have -an equivalent native path through this model. - -## Proposed Consumer Experience - -These output fragments illustrate the two requested forms. `schema` and -`schema_file` are suggested names only; exact syntax and public Go API design -remain upstream decisions. They do not describe currently supported fields. - -Inline: - -```yaml -output: - format: json - validation_mode: json_schema - schema: - type: object - properties: - summary: - type: string - required: [summary] - additionalProperties: false -``` - -File-backed: - -```yaml -output: - format: json - validation_mode: json_schema - schema_file: ./summary-response.json -``` - -Both forms should feed the same native preparation, structural validation, -provider structured-output, and repair behavior. No application schema loader -or registry should be necessary to use them. - -## Resource And Composition Semantics - -- A file reference uses the source context of the effective schema-reference - field, rather than a process working directory or the final child prompt's - location regardless of origin. -- An inherited reference retains its originating context. An overridden reference - uses the overriding source's context. Imports and collection composition follow - the same native rules as other resource-bearing fields. -- Embedded sources and operator files can coexist within one engine. Existing - filesystem-root rules apply; this does not request a second path-security model. -- Inline schema values participate in native configuration composition. Define - presence, replacement/clearing, and selection between inline and file-backed - forms explicitly, using ConfigKit's effective values rather than reconstructing - configuration history in the runner. -- Retain an explicit schema-source route for programmatic request contracts and - existing root-based consumers. Define how it interacts with prompt-owned forms; - avoid ambiguous multi-root searching or silently reinterpreting existing paths. -- Preserve preparation-time loading for schema files and the prepared execution's - captured validation plan. `RunPrepared` should execute with the schema captured - during preparation rather than reopening the source. - -This provides composition through schema references. It does not require a -filename-based schema overlay system: an operator replacement explicitly selects -its schema, while unchanged prompts continue using theirs. - -## Self-Contained Schemas Remain A Separate Concern - -The self-contained-schema decision has a concrete rationale. Local validation -could resolve external files that were absent from the single schema object sent -to the provider. Repository history at `c542b81` records the decision to remove -that mismatch without introducing schema bundling. - -That rationale does not require a global schema root. PromptKit can load a -self-contained schema beside its prompt, or accept it inline, and submit the same -complete object. Preserve the current [self-contained schema semantics](../formats.md#schemas), -including supported in-document references. Loading the root schema from a -prompt-relative file does not imply loading external JSON Schema `$ref` resources. -Schema bundling and network resource loading are outside this request. - -## Suggested Verification And Documentation - -- Equivalent inline and file-backed schemas produce equivalent local validation - and provider constraints. -- Embedded and operator prompts with different schema files work in one engine, - including files with the same basename in different source contexts. -- Replacing one prompt does not require copying or replacing unrelated schemas. -- Imported/inherited references preserve their defining context; overridden - references resolve against the new context independently of working directory. -- Inline/file selection and clearing follow the documented native composition - rules, including deliberate errors for unresolved or ambiguous selections. -- Existing explicit-root and request-level contract behavior has a deliberate, - documented compatibility path. -- Missing or invalid schema files produce useful errors identifying the relevant - prompt/reference. Self-contained-schema behavior remains consistent for both - new forms, and prepared executions retain their captured schema. - -Document the selected syntax, source-context behavior, request-level interaction, -and migration in the format reference, GoDoc, and consumer examples when -implemented. The request does not select a release number or require a new -application-owned asset assembly layer as an interim design. diff --git a/docs/roadmap/prompt-owned-schemas.md b/docs/roadmap/prompt-owned-schemas.md deleted file mode 100644 index 70cafd2..0000000 --- a/docs/roadmap/prompt-owned-schemas.md +++ /dev/null @@ -1,348 +0,0 @@ -# Prompt-Owned Inline And Source-Relative Schemas - -Status: accepted feature roadmap. All three decisions are settled. Examples -describe target, not implemented, syntax. The companion -[implementation plan](implementation.md) owns staged execution and concrete API -and migration details. -This document authorizes no code changes and selects no release number. - -## Purpose And Boundary - -Let prompt collections carry their own structured-output schemas, inline or in -nearby files, without requiring applications to assemble an engine-wide schema -filesystem. An operator can replace one embedded prompt and select its schema -while unrelated prompts continue using their own resources. - -[Notarius's request](prompt-owned-schemas-request.md) supplies the consumer need: -remove application asset registries, synthetic mounts, copying, and a flattened -schema namespace. PromptKit owns reusable resource resolution and structural -validation; Notarius retains content interpretation, domain validation, and -artifact handling. - -Scriptorium represents the primary workflow: one engine loads multiple collections, -some with their own schema roots and others with source-relative or inline schemas. -Supporting these layouts together is a first-class requirement. Runtime schema -substitution is uncommon; modest caller-side work is acceptable for that case. - -Follow the [ConfigKit handoff boundary](../adr/0001-configuration-boundary.md). -ConfigKit owns imports, inheritance, composition, and effective field context. -PromptKit interprets the final result. Neither the configuration adapter nor the -runner reconstructs source history to decide which schema should win. - -## Current Baseline - -The current checkout includes ConfigKit v1.3.0 and the changes described in the -[v0.24.0 release notes](../releases/v0.24.0.md). Schema support still has these -boundaries: - -- [Output contracts](../../types.go) select schemas through `SchemaPath`. - [Configuration capture](../../internal/configuration/prompts.go) copies that - string into the domain contract; no inline or source-relative schema selection - is captured. -- Schema source options configure a root used by the engine's validator, with - an implicit default directory of `"."`. The - [schema format contract](../formats.md#schemas) defines existing path meaning, - dialect, self-contained resources, and operational-error behavior. -- [Message files](../formats.md#messages-and-templates) already resolve using the - effective field's defining source context. Message content is read at engine - construction, while schema files are read during preparation. -- [Schema preparation](../internal/sources.md#schemas-and-output-validation) - creates an operation-local plan containing the schema document and compiled - validator. Local validation, structural repair, and provider structured-output - metadata use that plan. `RunPrepared` reuses its captured plan without reopening - resources. -- A request-level output contract replaces the entire configured contract. Schema - path is currently required for JSON Schema mode and ignored by other modes. - -The change needs new selection and resource plumbing, not another schema compiler -or application-managed overlay system. - -## Target End State - -### Schema Forms - -The accepted configuration forms are: - -| Field | Meaning | -| --- | --- | -| `schema` | An inline schema value in the prompt's output definition. | -| `schema_file` | A schema file resolved using that effective field's source context. | -| `schema_path` | A reference within the applicable explicitly configured schema root. | - -Inline and source-relative forms are the new capability. The explicit-root route -remains available, with optional collection-specific roots and an application-wide -default. Decisions 2 and 3 below record selection and API semantics. There is no -multi-root fallback search or implicit working-directory root. - -Illustrative inline output fragment: - -```yaml -output: - format: json - validation_mode: json_schema - schema: - type: object - properties: - summary: {type: string} - required: [summary] - additionalProperties: false -``` - -The same output can instead select `schema_file: ./summary-response.json`, whose -JSON document expresses that schema. Equivalence means equivalent schema values, -validation, and submitted constraints, not identical YAML/JSON source bytes or -an assurance that every provider supports every locally valid schema. - -### Optional Schema Roots - -Accept an optional `schema_root` at the configuration root and within each -`collections` entry. A collection can manage schemas centrally without requiring -other collections to adopt that layout. Inline and source-relative forms remain -available whether or not a root is configured. - -Illustrative configuration: - -```yaml -schema_root: ./application-schemas -collections: - weatherreporter: - schema_root: ./schemas - prompts: - - id: summarize.alerts - messages: - - role: user - content: Summarize the supplied alerts. - output: - format: json - validation_mode: json_schema - schema_path: alerts.schema.json -``` - -ConfigKit owns composition, layering, inheritance, and effective source context -for these fields, just as for other configuration. PromptKit consumes the final -values without another override or merge pass. Resolve each declared root against -the defining field's effective filesystem and base directory; an overridden root -uses the overriding source's context. This supports embedded and host filesystems -without assuming that a root is a host path. - -After composition, use the selected prompt's collection root when present; -otherwise use the application-wide root. Root-level prompts use the application -root. This is a scope-selection rule, not an exception to ConfigKit composition. -Select one root before opening the schema: a missing file under that root is an -error, not a reason to search another root or the prompt's directory. - -No root is needed for `schema` or `schema_file`. Selecting `schema_path` without -an applicable root is a configuration problem. Remove the implicit `"."` root -as an intentional breaking change; callers relying on it must explicitly select -a root or migrate their schema selection. - -Retain the ability to supply explicit schema directories and filesystems through -Go configuration helpers, but fit those helpers into ConfigKit's normal source -composition. Do not give Go options a special precedence rule or retain a second -runtime override pass. The companion implementation plan specifies helper -adaptation and migration; unchanged legacy option behavior is not a requirement -where it conflicts with this decision. - -### Effective Source Context And Composition - -Resolve `schema_file` using the effective field's defining filesystem and base -directory, as for `content_file`. Inherited references retain their origin; -overridden references use the overriding source. A child prompt's location does -not replace an inherited field's context. Imports and named collections follow -the same native rules. - -Embedded defaults and operator sources can coexist, including different files -with the same basename. Replacing a prompt explicitly selects its schema; it -need not replace the engine-wide root or copy unrelated default schemas. - -Use existing rooted-filesystem and usable-context rules. Do not add a new path -security model, automatic mounts, or schema search paths. A source-relative -reference without a usable context needs a diagnostic identifying the field and -prompt, rather than an inferred working directory. - -Inline values participate in native ConfigKit composition. In JSON Schema mode, -require exactly one nonnull selection among `schema`, `schema_file`, and -`schema_path` after composition. Null clears a selection; `{}` and `false` are -schema values. Blank file references are invalid. Ignore schema selections outside -JSON Schema mode, preserving the existing behavior for paths. Switching forms may -require explicitly clearing an inherited alternative; no field takes precedence -over another. Whole-schema replacement uses ConfigKit's existing controls. -No PromptKit pass repairs or replays configuration composition. A source-neutral -runtime representation may retain the filesystem capability and resolved resource -reference needed for preparation; runtime components need not depend on ConfigKit -objects or inspect configuration history. - -### Preparation And Captured Execution - -At construction, capture the effective inline value or the file reference and -its resource context. Validate selection and context metadata at the configuration -boundary. Keep file reading and schema compilation at preparation time; do not -read all schema files at construction merely to copy message-loading behavior. - -Every selected form feeds the existing validation-plan pipeline. Inline values -need no file read; file forms load the selected root document for the operation. -Compile and retain that document for local validation and provider submission, -then reuse the plan for any structural repair. Preserve existing cancellation, -error classification, dialect, and self-contained-schema semantics. - -`PrepareExecution` retains the prepared schema and validator. `RunPrepared` does -not reopen the schema even if its source changes or disappears. Later ordinary -runs may read the changed file during their own preparation. Inline configuration -remains part of the engine's captured configuration. This does not introduce -schema watching, hot reload, or cross-operation caching requirements. - -Missing, malformed, or unresolvable selected schema files fail preparation through -the existing operational validation-error boundary, with useful prompt/reference -context. Invalid generated content remains an ordinary failed validation result. -Invalid configuration selection or missing source context is a configuration -problem. Avoid promising diagnostics for nonexistent partial validation plans. - -### Public API And Inspection - -Keep complete request-level output-contract replacement, with no additional -configuration merge. A replacement must not retain the prompt's schema selection -or the source context of its `schema_file` field implicitly. Support inline values -and `SchemaPath` in runtime contracts. Runtime `SchemaPath` uses the selected -prompt's applicable collection/application root under Decision 1: replacing the -output contract does not erase the surrounding collection configuration. - -Keep `schema_file` in configuration, where ConfigKit supplies effective source -context, including for typed configuration sources. Do not add a runtime -filesystem-reference API in this work. A caller needing an unrelated schema file -can read and decode it, then supply its value inline. This prioritizes configured -collection workflows without requiring extra runtime source machinery. - -Extend typed prompt configuration helpers, public output contracts, conversions, -and inspection consistently. Inline values and exposed details must follow -existing ownership patterns so caller mutation cannot alter captured configuration -or a prepared execution. Configuration inspection describes the resolved inline -value or reference without opening schema files. Prepared details continue showing -the schema document actually selected for the operation. - -Accept inline schema values through native configuration/public value mechanisms, -including meaningful empty objects and boolean schemas supported by the existing -schema pipeline. Do not treat `false` as absence or impose an object-only rule -merely for a convenient Go representation. Use existing JSON-compatible handling -rather than introduce a numeric-conversion adapter or precision guarantees beyond -those of the selected input path. File-backed JSON decoding retains its existing -behavior. The companion implementation plan selects concrete public Go types. - -Preserve existing structured-output naming, validation/repair semantics, and hash -responsibilities. Account for new configured fields in existing projections and -hash inputs where appropriate, without promising that a prompt or message hash -identifies runtime schema file contents. No new schema registry, public schema-ID -system, or digest API is required by this request. - -### Self-Contained Resources - -Root schema location and JSON Schema reference resolution remain separate. -Loading a root beside its prompt does not enable external `$ref` files or network -loading. Preserve supported in-document IDs, anchors, and references, including -the existing independence from the retrieval filename. Use the same schema value -for local validation and provider constraints. Schema bundling, remote loading, -and new provider-specific dialects are outside this work. - -## Validation Targets - -Use the consumer request's scenarios with real configuration composition and -controlled local/embedded filesystems. Keep provider checks offline through the -existing injected-client or transport seams. - -- Equivalent inline, source-relative, and explicit-root documents produce the - same local constraints and provider schema value through ordinary preparation. -- Embedded/operator prompts coexist with identical schema basenames; replacing - one prompt leaves unrelated schema selections usable. -- One engine loads collections using independent roots, source-relative files, - and inline schemas together, without caller-managed mounts or file copying. -- Imports and inheritance retain effective field context; overrides change it. - Resolution is independent of process working directory and observes existing - root/context rules. -- Collection and application roots follow native ConfigKit composition and - effective field context, including roots supplied through Go helpers. Collection - scope wins when configured; otherwise application scope applies. Missing files - never trigger a search of another root. Inline/source-relative schemas require - no root, and `schema_path` without an applicable root fails clearly. -- Native inline composition, null clearing, conflicting forms, empty objects, - boolean schemas, and malformed selections follow the chosen contract. -- Request replacement does not borrow the prompt's schema selection or - `schema_file` context. Runtime `SchemaPath` uses the applicable configured - collection/application root; inline replacements require no root. Explicit - schema source helpers follow native composition and documented migration, - without a special Go-option precedence rule. -- Missing/invalid files fail at preparation, not eager engine construction; - prepared execution survives later source modification/removal. Inline input - and returned-detail mutation do not alter retained execution state. -- Existing dialect and external-reference rejection remain consistent across - forms; supported in-document references still work. Source-relative loading - introduces no extra `$ref` resource reads. -- Inspection remains generically renderable and respects its configured-resource - boundary. Prepared details and provider submission reflect the selected plan. -- Representative validation/repair, continuation, cancellation, and smoke paths - continue using the common plan without duplicating schema loading or compilation - beyond existing self-contained checks. - -## Accepted Decisions - -### 1. How Should New File References Coexist With `schema_path`? - -**Accepted:** add `schema` and `schema_file` while preserving `schema_path` for -an explicitly selected root. Roots are optional and configurable per collection -and application, as specified in [Optional Schema Roots](#optional-schema-roots). -Use ConfigKit's standard composition and effective source contexts, with no -special Go-option precedence. Remove the implicit `"."` root. The three forms -support different collection layouts without requiring application asset assembly -or fallback searches; migration must cover the deliberate default-root change -and any necessary adaptation of Go helpers. - -**Alternative considered:** replace the selection with one tagged schema-source value -covering inline, relative-file, and explicit-root forms. This makes alternatives -structurally explicit and may simplify future API representation. It requires -configuration and Go migration, or a compatibility translation layer. Separate -fields were selected to keep each reference form's meaning direct; the accepted -root changes do not reinterpret `schema_path` as a source-relative file. - -### 2. How Should Selection, Composition, And Clearing Work? - -**Accepted:** in JSON Schema mode, require exactly one nonnull effective -selection among the three forms. Null clears a selection; `{}` and `false` remain -schema values, not absence. Blank file/path selections are invalid rather than -fallback instructions. Ignore schema selections outside JSON Schema mode, -preserving existing inactive-mode behavior for paths without introducing -unrelated restrictions on unused fields. - -Inline mappings compose according to ConfigKit. Switching forms may require -explicitly clearing the inherited alternative, for example `schema_file: null` -when supplying `schema`. Whole-value replacement uses ConfigKit's composition -controls; PromptKit does not invent its own replacement pass. This is predictable -from final configuration and aligns with the existing message-content selection -pattern, though it requires an explicit clear when switching inherited forms. - -**Alternative considered:** define fixed precedence, such as inline over source-relative -file over legacy path. This also works on final effective values and makes a -higher-priority form convenient without clearing another field. It can silently -leave stale alternatives that reappear when the preferred form is cleared, and -makes the effective configuration less obvious to operators. Explicit conflict -errors were selected to avoid a PromptKit-specific precedence rule. - -### 3. How Much Source Selection Should Programmatic Requests Expose? - -**Accepted:** support inline values and `SchemaPath` in request -contracts; keep source-relative `schema_file` tied to configuration inputs with -resource context. Typed configuration sources can supply that context through -ConfigKit just as they do for message files. Request replacement is complete and -never borrows the prompt's `schema_file` context implicitly. Runtime `SchemaPath` -uses the applicable configured root under Decision 1. Collection root selection -is surrounding configuration, not inheritance from the replaced output contract. - -Mixed collection layouts are the common case and must work together in one -engine. Runtime substitution can use inline values or the applicable configured -root. For an unrelated runtime file, the caller reads and decodes the schema and -supplies the inline value. The small application burden for this uncommon case -does not justify a new runtime filesystem-reference API in this roadmap. - -**Alternative deferred:** let runtime contracts supply an explicit filesystem -and file reference. This could help per-job or tenant-specific schemas and -runtime-selected bundles by reusing PromptKit's reading, decoding, diagnostics, -and preparation lifecycle. It adds a public resource type and ownership/lifetime -semantics. Revisit only for a concrete consumer need; configured collections do -not require it. Neither approach needs implicit working-directory lookup. diff --git a/docs/roadmap/remediation.md b/docs/roadmap/remediation.md deleted file mode 100644 index 4e9fcaf..0000000 --- a/docs/roadmap/remediation.md +++ /dev/null @@ -1,267 +0,0 @@ -# Prompt-Owned Schema Review Findings - -Reviewed commit `e9a12d4` against all eight stages of -[implementation.md](implementation.md) and the accepted -[feature roadmap](prompt-owned-schemas.md). - -## 1. P2 — Inactive Runtime Schema Values Break Snapshot Ownership - -Status: complete. Relevant stages: 1 and 6. - -Resolution: inactive runtime inline schemas are omitted from execution-owned -contracts, without validating them. Public regressions cover struct/map, cyclic, -and unencodable values across inactive modes, preserved legacy path metadata, -and independent prepared details. Configured inspection and active schemas retain -their existing ownership behavior. - -Locations: - -- `internal/usecase/runner.go:688`: `resolveOutputContract` retains the caller's - raw `Schema` value outside JSON Schema mode. -- `internal/jsonvalue/trusted.go:20`: `cloneTrustedReflect` recursively copies - maps, slices, pointers, and arrays, but returns struct values unchanged. -- `convert.go:339`: public output-contract projections rely on that copier. - -### Evidence And Impact - -A public-API reproduction supplies this JSON-encodable Go value in a runtime -replacement using `ValidationNone`: - -```go -type schemaValue struct { - Properties map[string]any `json:"properties"` -} -original := schemaValue{Properties: map[string]any{"title": "original"}} -``` - -After successful `PrepareExecution`, mutating -`handle.Details().OutputContract.Schema.(schemaValue).Properties["title"]` -changes both `original.Properties` and the value returned by a later `Details()`. -The reproduction printed `details mutation` for both values. A caller changing -its original map after preparation likewise changes retained details. - -This violates the existing caller-owned `Details`/frozen-snapshot contract. -The schema is inactive, so this reproduction does not change output validation; -it still exposes shared mutable state and can introduce races when callers rely -on the documented ownership boundary. Active runtime schemas take the JSON -encoding/decoding path and do not exhibit this particular problem. - -### Required Outcome And Refactoring Opportunity - -Ensure accepted inactive runtime schema inputs cannot create aliases between -caller storage, returned details/notifications, and retained state. Preserve the -accepted rule that inactive schema values are ignored: do not fix this by adding -schema-validation or JSON-encodability requirements to non-schema modes. - -The simplest approach worth considering is to omit the new inactive runtime -`Schema` value from execution-owned snapshots. Keep configured-resource -inspection's declared schema metadata and the established inactive `SchemaPath` -behavior separate. Alternatively, if inactive inline values are retained in -prepared projections, their copying must actually satisfy the ownership contract -for accepted inputs, including JSON-encodable structs with mutable fields. -Avoid expanding this into a general arbitrary-Go-object cloning API. - -The new `TestInactiveSchemaDoesNotInterpretCyclicValue` also asserts preservation -of a cyclic map's graph topology. Ignoring inactive schema values was requested; -preserving arbitrary inactive object graphs was not. That incidental assertion -should not prevent the simpler solution. Keep a test for successful handling of -inactive cyclic/unencodable values, without imposing graph-preservation behavior -unless deliberately adopted. - -Add a focused public regression for the struct/map case: mutations of caller -input and returned details must not change retained details. If inactive schema -values are omitted, assert omission and successful preparation instead. Retain -the existing active-schema ownership coverage. - -## 2. P2 — Keep Lightweight Prompt Selection Independent Of Inline Schemas - -Status: complete. Identified by the follow-up integration audit. Relevant stages: -1, 3, and 5. - -Resolution: removed the output contract from the internal lightweight selection -type and its fixture/projection paths. Selection no longer visits schema trees; -the full resource and execution views retain their schema data. Focused selection, -requirements, smoke, and schema tests pass. - -Locations: - -- `internal/configuration/prompts.go:147`: the internal requirement-selection - snapshot includes the complete output contract. -- `internal/configuration/prompts.go:351`: `ownPromptInspection` now calls - `ownOutputContract`, which deep-copies the inline schema. -- `internal/usecase/requirements_inspection.go:38`: dependency inspection obtains - this snapshot to select the prompt and its default profile. -- `internal/usecase/smoke_selection.go:41`: prompt-selected diagnostics use the - same metadata lookup, independently of their private diagnostic schema. - -### Evidence And Impact - -The production path is `resolvePromptInspection` → `GetPromptInspection` → -`ownPromptInspection` → `ownOutputContract` → `CloneTrusted(schema)`. -`InspectRequirements` consumes only `PromptID` and `DefaultProfileID` from that -selection; its input/template analysis separately uses `GetRequirementSnapshot`. -Smoke target selection likewise consumes only prompt identity/default profile. -Neither caller reads the selected prompt's output contract. - -Before this feature, copying that output contract copied scalar fields. With an -inline schema of S nodes/container entries, every such lookup now adds O(S) time -and container allocations, in addition to its previous input-metadata copy. -A large schema annotation is enough to impose this cost without changing any -template requirement or diagnostic selection. File-backed schemas do not impose -an equivalent document-copy cost on these paths, creating an avoidable difference -between otherwise equivalent schema forms. No timing measurements are needed to -establish the extra traversal. - -### Required Outcome - -Keep internal identity/default-profile selection separate from output-schema -payloads. Narrow the selection projection or omit unused output-contract data -there, rather than copying a full schema to satisfy ownership of a value that -neither caller needs. Preserve full, detached schema metadata for the public -`InspectPrompt` resource view and for actual execution preparation. Do not -introduce a schema-size limit or weaken snapshot ownership as a workaround. - -Retain existing requirements and smoke-selection behavioral tests; verify both -paths still select the same prompt/profile and that application schema files -remain unopened. Prefer a narrow internal value boundary over a benchmark gate -or tests coupled to incidental copier call counts. - -## 3. P3 — Scope The Existing Request-Value Limit Documentation - -Status: complete. Identified by the follow-up integration audit. Relevant stages: -1, 6, and 8. - -Resolution: RunRequest GoDoc now scopes bounded copying to Execution.ExtraParams -and explicitly distinguishes inline schema documents. No runtime limits changed. - -`types.go:82–86` still says generically that excessively deep or large -JSON-shaped request values are rejected. The new `OutputContract.Schema` route -deliberately uses standard JSON encoding/decoding and trusted document copying, -without the provider-extra copier's depth/node bounds. The format reference -already describes that exemption under Schemas. - -An offline public-API check successfully prepared a runtime inline schema with -an `examples` array containing 100,001 null entries. That exceeds the bounded -copier's 100,000-node budget but is correctly accepted by the schema path. -The code follows the accepted design; the older broad GoDoc no longer accurately -describes all JSON-shaped request values after this addition. - -Update the `RunRequest` GoDoc to scope the existing limits to the request fields -that actually use them, and distinguish inline schema documents. Preserve the -general ownership statements. Do not add bounds to schemas to make the old -wording true. This is a documentation correction, not a request for new limit -options, schema guarantees, or additional runtime validation. - -## 4. P3 — Allocate Trusted Slice Copies By Length, Not Spare Capacity - -Status: complete. Identified by the algorithmic audit. Relevant stages: 1 and 6. - -Resolution: trusted slice copies allocate backing storage from the populated -length. Internal regressions cover interface, string, byte, named, and nonnil -empty slices with spare capacity; existing nil/empty and ownership tests remain. - -Location: `internal/jsonvalue/trusted.go:50` in `cloneTrustedReflect`. - -### Complexity And Evidence - -The revised copier allocates slices with: - -```go -reflect.MakeSlice(value.Type(), value.Len(), value.Cap()) -``` - -It then copies only `value.Len()` elements. For a slice of length N and capacity -C, the backing allocation reserves O(C) element slots although the value being -detached contains only N elements. C can be arbitrarily larger than N, so this -allocation is not bounded by the logical slice contents. Repeated prepared-detail -or notification projections can repeat that excess allocation. - -Before this sprint, the `[]any` fast path used `make([]any, len(v))` and the -`[]string`/`[]byte` paths copied populated elements without reproducing arbitrary -spare capacity. Removing those paths routes them through the capacity-preserving -reflection branch. In particular, currently accepted inactive runtime schema -values can carry a caller-allocated slice directly into this path. - -A small public-API structural check prepared an inactive `Schema` containing a -`[]any` with length 1 and capacity 32. Its detached `Details()` copy also had -length 1 and capacity 32, confirming that unused capacity is allocated again. -This was a value-shape check, not a timing, profiling, or memory-usage benchmark. - -### Refactoring Opportunity - -Allocate detached slice storage from the populated length, e.g. -`reflect.MakeSlice(value.Type(), value.Len(), value.Len())`, and retain the -existing element-copy behavior. This reduces backing-storage space to O(N), -while element traversal remains O(N). Preserve concrete slice types, contents, -nil-versus-empty behavior, and independence from caller storage. Spare capacity -is not a requested public contract; do not add one. - -This is distinct from finding 1: the example's elements are already detached -correctly, but too much backing storage is allocated. Omitting inactive schema -values may remove this particular public entry path; restoring/simplifying the -shared copier as part of that fix can also address the allocation regression. -Do not introduce schema limits or a general object-graph facility to solve it. - -Use focused internal slice-copy coverage for small populated/empty slices with -large spare capacity, alongside existing ownership and nil/empty tests. No new -benchmark gate or public exact-capacity guarantee is needed. - -## Review Coverage And Validation - -The initial implementation review covered configuration/root capture, effective -source context, selection/clearing, shared compilation, request replacement, -provider/repair integration, smoke isolation, mixed collections, and documentation -migration without identifying another actionable finding in that pass. - -On the reviewed commit, using Go 1.27.4 with `GOTOOLCHAIN=local` and `GOWORK=off`: - -- `go test -race ./...` passed, including the maintained offline examples. -- `go vet ./...` passed. -- The separate public-API reproduction above confirmed the ownership bug not - covered by the passing suite. - -The review itself changed no implementation code. Completed remediations are -recorded under the findings above. - -### Follow-Up Integration Audit - -Reviewed the same implementation code at `3bc16bf` (the preceding commit added -only this findings document). All eight stages were revisited with particular -attention to their existing callers and data projections. Findings 2 and 3 are -additional to the still-open ownership finding, not duplicates of it. - -An additional offline public-API diagnostic exercised detached `OnPrepared` and -injected-client schemas, a repaired file-backed run, an inline replacement during -`Coordinate`, a prepared run after source removal, and a later ordinary -preparation failure. Those interactions passed. It also confirmed acceptance of -the large inline annotation described in finding 3. The earlier race-suite/vet -results apply to the unchanged implementation; no live provider calls or -performance measurements were used. - -### Algorithmic Audit - -Reviewed the unchanged implementation at `4120d78` against the eight stages. -Finding 4 is additional; finding 2 already records the avoidable schema-sized -lookup traversal and is not duplicated here. - -Root capture uses a shared metadata snapshot and indexed root selection rather -than rescanning configuration per prompt. New schema selection/binding has no -growing nested scan. File reading grows its buffer amortized-linearly, and -decoded documents take linear copy passes through the selected ownership paths. -Prepared execution and repair reuse the captured plan instead of multiplying -schema reads/compilation by the number of attempts. Fixed test/example fixtures -add no unbounded algorithmic workload. - -The expressly selected JSON conversion, existing schema-location/portability -checks and compiler behavior, and existing hash algorithms were not reopened. -No additional ordinary correctness issue was found beyond the recorded findings. -No real-world performance data, timing measurements, or profiling were collected; -the earlier successful suite/vet results apply to unchanged implementation code. - -## Remediation Validation - -All four findings are complete. Focused ownership, schema, requirements, -selection, and smoke tests passed during the corresponding fixes. The assembled -remediations passed `go test -race ./...` (including maintained examples), -`go vet ./...`, and `git diff --check`, using Go 1.27.4 with `GOTOOLCHAIN=local` -and `GOWORK=off`.