From 0603a4f0dd0ea01e708ec580e005166c5b77ca79 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Fri, 20 Nov 2026 13:10:42 +0000 Subject: [PATCH] Record collection schema roots and native composition --- docs/roadmap/prompt-owned-schemas.md | 116 +++++++++++++++++++++------ 1 file changed, 91 insertions(+), 25 deletions(-) diff --git a/docs/roadmap/prompt-owned-schemas.md b/docs/roadmap/prompt-owned-schemas.md index 105dd32..108636c 100644 --- a/docs/roadmap/prompt-owned-schemas.md +++ b/docs/roadmap/prompt-owned-schemas.md @@ -1,7 +1,7 @@ # Prompt-Owned Inline And Source-Relative Schemas -Status: proposed target and implementation roadmap. The decisions at the end -remain open; examples use recommended field names, not implemented syntax. +Status: proposed target and implementation roadmap. Decision 1 is accepted; +Decisions 2 and 3 remain open. Examples describe target, not implemented, syntax. This document authorizes no code changes and selects no release number. ## Purpose And Boundary @@ -32,7 +32,8 @@ boundaries: [Configuration capture](../../internal/configuration/prompts.go) copies that string into the domain contract; no inline or source-relative schema selection is captured. -- Schema source options configure a root used by the engine's validator. The +- Schema source options configure a root used by the engine's validator, with + an implicit default directory of `"."`. The [schema format contract](../formats.md#schemas) defines existing path meaning, dialect, self-contained resources, and operational-error behavior. - [Message files](../formats.md#messages-and-templates) already resolve using the @@ -53,18 +54,18 @@ or application-managed overlay system. ### Schema Forms -The recommended configuration forms are: +The accepted configuration forms are: | Field | Meaning | | --- | --- | | `schema` | An inline schema value in the prompt's output definition. | | `schema_file` | A schema file resolved using that effective field's source context. | -| `schema_path` | The existing reference in the explicitly configured schema root. | +| `schema_path` | A reference within the applicable explicitly configured schema root. | Inline and source-relative forms are the new capability. The explicit-root route -remains available for existing consumers and programmatic contracts. Decisions -1–3 below settle selection and API details; no multi-root fallback search or -implicit working-directory interpretation is proposed. +remains available, with optional collection-specific roots and an application-wide +default. Decisions 2 and 3 below settle selection and API details. There is no +multi-root fallback search or implicit working-directory root. Illustrative inline output fragment: @@ -85,6 +86,56 @@ JSON document expresses that schema. Equivalence means equivalent schema values, validation, and submitted constraints, not identical YAML/JSON source bytes or an assurance that every provider supports every locally valid schema. +### Optional Schema Roots + +Accept an optional `schema_root` at the configuration root and within each +`collections` entry. A collection can manage schemas centrally without requiring +other collections to adopt that layout. Inline and source-relative forms remain +available whether or not a root is configured. + +Illustrative configuration: + +```yaml +schema_root: ./application-schemas +collections: + weatherreporter: + schema_root: ./schemas + prompts: + - id: summarize.alerts + messages: + - role: user + content: Summarize the supplied alerts. + output: + format: json + validation_mode: json_schema + schema_path: alerts.schema.json +``` + +ConfigKit owns composition, layering, inheritance, and effective source context +for these fields, just as for other configuration. PromptKit consumes the final +values without another override or merge pass. Resolve each declared root against +the defining field's effective filesystem and base directory; an overridden root +uses the overriding source's context. This supports embedded and host filesystems +without assuming that a root is a host path. + +After composition, use the selected prompt's collection root when present; +otherwise use the application-wide root. Root-level prompts use the application +root. This is a scope-selection rule, not an exception to ConfigKit composition. +Select one root before opening the schema: a missing file under that root is an +error, not a reason to search another root or the prompt's directory. + +No root is needed for `schema` or `schema_file`. Selecting `schema_path` without +an applicable root is a configuration problem. Remove the implicit `"."` root +as an intentional breaking change; callers relying on it must explicitly select +a root or migrate their schema selection. + +Retain the ability to supply explicit schema directories and filesystems through +Go configuration helpers, but fit those helpers into ConfigKit's normal source +composition. Do not give Go options a special precedence rule or retain a second +runtime override pass. API planning must spell out the helper adaptation and any +migration required; unchanged legacy option behavior is not a requirement where +it conflicts with this decision. + ### Effective Source Context And Composition Resolve `schema_file` using the effective field's defining filesystem and base @@ -180,11 +231,13 @@ and new provider-specific dialects are outside this work. 2. **Capture schema ownership.** Extend configuration decoding, normalization, domain selection, and inspection. Share the existing effective-field resource resolution mechanics where useful, but separate resource binding from file - reading so schema bytes remain preparation-time data. + reading so schema bytes remain preparation-time data. Capture optional roots + and their effective contexts, then apply the accepted scope-selection rule. 3. **Unify schema preparation.** Extend the validator's input route to consume the selected inline value or file source. Retain one compilation/plan path for - local checks, provider metadata, and repairs. Keep existing explicit-root - source options operational; avoid parallel validators with divergent semantics. + local checks, provider metadata, and repairs. Adapt explicit-root source helpers + to native ConfigKit composition and remove the implicit root; avoid parallel + validators with divergent semantics. 4. **Complete public integration.** Update output-contract conversion, typed helpers, prompt inspection, preparation details, and ordinary/prepared runs. Check request replacement, continuation, and smoke-test paths that use output @@ -193,8 +246,8 @@ and new provider-specific dialects are outside this work. 5. **Verify and document.** Add focused offline coverage below; update the format reference, GoDoc, consumer examples, and internal source/validation documentation when implementation lands. Show embedded and operator collections coexisting, - switching forms, and retaining the legacy explicit-root route. Describe any - Go struct/API compatibility considerations without selecting a release here. + switching forms, and configuring optional roots. Document migration from the + implicit root and any Go helper/API changes without selecting a release here. Internal refactoring is permitted; these steps specify behavior and ownership, not a permanent package layout. No interim application asset assembly layer is @@ -213,10 +266,16 @@ existing injected-client or transport seams. - Imports and inheritance retain effective field context; overrides change it. Resolution is independent of process working directory and observes existing root/context rules. +- Collection and application roots follow native ConfigKit composition and + effective field context, including roots supplied through Go helpers. Collection + scope wins when configured; otherwise application scope applies. Missing files + never trigger a search of another root. Inline/source-relative schemas require + no root, and `schema_path` without an applicable root fails clearly. - Native inline composition, null clearing, conflicting forms, empty objects, boolean schemas, and malformed selections follow the chosen contract. -- Request replacement does not borrow the prompt's schema or context. Legacy - schema source options retain their documented meaning. +- Request replacement does not borrow the prompt's schema or context. Explicit + schema source helpers follow the accepted composition model and documented + migration, without a special Go-option precedence rule. - Missing/invalid files fail at preparation, not eager engine construction; prepared execution survives later source modification/removal. Inline input and returned-detail mutation do not alter retained execution state. @@ -229,25 +288,30 @@ existing injected-client or transport seams. continue using the common plan without duplicating schema loading or compilation beyond existing self-contained checks. -## Open Decisions +## Decisions And Remaining Questions ### 1. How Should New File References Coexist With `schema_path`? -**Recommendation:** add `schema` and `schema_file` while preserving `schema_path` -for the explicit schema root. Each name has one meaning, so installed collections -can adopt nearby schema files without changing existing consumers. Retain the -current schema-source options. This costs an additional field but gives an -unambiguous migration path with no fallback search. +**Accepted:** add `schema` and `schema_file` while preserving `schema_path` for +an explicitly selected root. Roots are optional and configurable per collection +and application, as specified in [Optional Schema Roots](#optional-schema-roots). +Use ConfigKit's standard composition and effective source contexts, with no +special Go-option precedence. Remove the implicit `"."` root. The three forms +support different collection layouts without requiring application asset assembly +or fallback searches; migration must cover the deliberate default-root change +and any necessary adaptation of Go helpers. -**Alternative:** replace the selection with one tagged schema-source value +**Alternative considered:** replace the selection with one tagged schema-source value covering inline, relative-file, and explicit-root forms. This makes alternatives structurally explicit and may simplify future API representation. It requires -configuration and Go migration, or a compatibility translation layer, for a need -that can be met additively. It is viable if the cleaner single selector justifies -that migration; silently reinterpreting existing `schema_path` is not recommended. +configuration and Go migration, or a compatibility translation layer. Separate +fields were selected to keep each reference form's meaning direct; the accepted +root changes do not reinterpret `schema_path` as a source-relative file. ### 2. How Should Selection, Composition, And Clearing Work? +**Status: open.** + **Recommendation:** in JSON Schema mode, require exactly one nonnull effective selection among the three forms. Null clears a selection; `{}` and `false` remain schema values, not absence. Blank file/path selections are invalid rather than @@ -271,6 +335,8 @@ errors are preferable unless consumers demonstrate substantial value in fallback ### 3. How Much Source Selection Should Programmatic Requests Expose? +**Status: open.** + **Recommendation:** support inline values and existing `SchemaPath` in request contracts; keep source-relative `schema_file` tied to configuration inputs with resource context. Typed configuration sources can supply that context through