From e775a3c116c728795a8cad52a496726bc2b1fc5a Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Fri, 20 Nov 2026 14:41:43 +0000 Subject: [PATCH] Plan prompt-owned schema support --- docs/roadmap/implementation.md | 398 +++++++++++++++++++ docs/roadmap/prompt-owned-schemas-request.md | 166 ++++++++ docs/roadmap/prompt-owned-schemas.md | 52 +-- 3 files changed, 572 insertions(+), 44 deletions(-) create mode 100644 docs/roadmap/implementation.md create mode 100644 docs/roadmap/prompt-owned-schemas-request.md diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md new file mode 100644 index 0000000..b0c329e --- /dev/null +++ b/docs/roadmap/implementation.md @@ -0,0 +1,398 @@ +# Prompt-Owned Schemas Implementation Plan + +## Objective And Authority + +Implement [prompt-owned-schemas.md](prompt-owned-schemas.md). That feature +roadmap owns the accepted selection, composition, scope, ownership, and migration +decisions. The [consumer request](prompt-owned-schemas-request.md) is background, +not authority for application-specific behavior. Respect +[ADR-0001](../adr/0001-configuration-boundary.md): interpret only ConfigKit's final +effective configuration and its effective resource context. + +The baseline is commit `adfccb5`, with ConfigKit v1.3.0 and the prepared +message-content hash. This is one end-to-end sprint, divided into sequential +single-prompt stages for a Luna coding agent. Intermediate stages may fail to +compile. Do not add compatibility shims or duplicate implementations merely to +make an intermediate commit coherent. No stage requires human approval, external +publication, or a dependency upgrade. Commit completed units under repository +policy; commits are not review gates. + +Start with [development.md](../development.md) and the applicable +[agent](../policy/agents.md), [architecture](../policy/architecture.md), +[testing](../policy/testing.md), and [documentation](../policy/documentation.md) +policies. Read the [format reference](../formats.md), +[source/validation internals](../internal/sources.md), and existing GoDoc before +changing their contracts. Reuse existing file-path, schema compilation, +cancellation, and ownership machinery where applicable. + +## Concrete Design For This Sprint + +### Public Values And Configuration + +- Add `Schema any` with `json:"schema,omitempty"` to public `OutputContract`. + Nil means no inline selection; a nonnil empty object and `false` are selections. + Preserve `SchemaPath string` and its existing JSON field. Do not add runtime + `SchemaFile`, filesystem handles, or a standalone schema-loading API. +- Add `Schema any` and `SchemaFile *string` to `PromptOutputConfig`, retaining + `SchemaPath *string`. `PromptSource` emits `schema` only for a nonnull inline + value and emits each nonnil reference pointer. As with existing typed helpers, + raw ConfigKit mappings express explicit null and composition controls. +- Add `Schema any` and `SchemaFile *string` to `PromptOutputInspection`, with + `schema` and `schema_file` JSON/YAML names and `omitempty`. Preserve its existing + path presence. Expose configured values/references, not filesystem capabilities + or file contents. Generic configuration inspection already exposes declared + application and collection roots; no additional public root-inspection API is + needed. +- Decode optional `schema_root` strings at the application root and collection + entries. Absent/null collection roots select the application root; absent/null + application roots mean no root. A nonnull blank root is invalid. An explicitly + supplied `"."` is valid when accompanied by usable source context. +- In JSON Schema mode, exactly one nonnull effective `schema`, `schema_file`, or + `schema_path` is required. Reject blank file/path selections and conflicts. + Outside that mode, do not resolve, compile, or impose new selection checks on + unused schema fields. Retain ordinary ConfigKit decoding semantics. + +Use native ConfigKit values for captured configuration. For active runtime +inline values, use standard JSON encoding and the existing schema JSON decoder +to obtain an operation-owned document; do not introduce a custom numeric bridge +or the bounded provider-extra copier. JSON null, including typed nil values +encoded as null, means no selection. Empty objects and booleans must survive. +Unsupported/cyclic Go values that cannot be encoded fail as invalid request +values, not panics. JSON-compatible but invalid schema documents fail compilation +at preparation through the existing validation-error boundary. Preserve existing +file JSON decoding and its numeric behavior; promise no new cross-input numeric +equivalence. Clone captured, already-supported trees using the existing trusted +document ownership path, without provider-extra depth/node limits. + +### Schema-Root Helper Migration + +Replace the out-of-band root settings with one mapping helper: + +```go +func SchemaRootSource(label, root string) configkit.Source +``` + +It returns `configkit.Mapping(label, map[string]any{"schema_root": root})`, +without opening files or validating raw contributions. Callers place its input +at their chosen native ConfigKit layer and provide a filesystem context, just as +they do for `PromptSource`. For example, for an `fs.FS`: + +```go +filesystem := configkit.NewFilesystem("application", schemaFS) +rootInput := configkit.Input{ + Source: promptkit.SchemaRootSource("application schemas", "schemas"), + Filesystem: &configkit.FilesystemContext{ + FS: &filesystem, Root: ".", BaseDir: ".", + }, +} +// Supply rootInput in configkit.Underlay{Inputs: []configkit.Input{rootInput}}, +// or use &rootInput as Main when no other main input is needed. +``` + +Remove `Config.SchemaDir`, `WithSchemaFS`, and `WithSchemaFile`, and the internal +option setters/flags used solely by those APIs. This is an intentional pre-1.0 +API migration, not a deprecation period. Do not silently translate options into +a highest-priority layer or preserve last-option-wins behavior. Callers needing +collection-specific roots use ordinary collection mappings with context; the +helper need not grow collection-specific parameters. + +Migration: a directory or filesystem becomes a context-backed root mapping; +a former single-file source becomes a `schema_file` on a configured prompt or a +root mapping plus its basename in `schema_path`. For an unrelated per-request +file, the caller reads/decodes it and supplies `Schema` inline. Host-directory +examples use an explicit `os.DirFS`/ConfigKit filesystem context. Do not infer a +working-directory root. This preserves the requested directory/filesystem +capabilities without preserving a second source-selection system. + +### Internal Resource And Preparation Model + +Separate serializable selections from resource capabilities: + +- Extend domain output selections with inline `Schema` and the configured + `SchemaFile` reference string. The latter is internal, not a new request field. +- Capture an optional applicable root and optional source-relative file binding + with the prompt definition. Use small source-neutral domain values containing + an `fs.FS` and resolved filesystem path. Exclude those capabilities from JSON + and definition hashing. Keep the root separate from the output contract so it + remains available to a runtime `SchemaPath` replacement. +- Configuration binding resolves a field's effective filesystem/base directory + using the current effective-origin lookup and `SourceContext`, as `readContent` + does. Extract binding from reading so schema capture does not call `Open`, + `Stat`, or compile a schema to establish file existence. Apply existing + `filecatalog` path containment rules. Missing context/invalid reference metadata + is a configuration error; missing or malformed file data is a preparation error. +- Add internal `validate.Input` containing `Contract domain.OutputContract` and + optional `File` binding. Change `ValidationPreparer.PrepareValidation` to accept + this input. Non-schema modes need no file. JSON Schema mode selects the inline + document or the one bound file and uses one common compilation pipeline. +- After complete request-contract replacement, the runner selects the binding: + configured `schema_file` uses its captured binding only without a replacement; + `SchemaPath` binds under the selected prompt's applicable root; inline values + need no file. A replacement never borrows the configured inline/file selection. + A missing root for a runtime path is an invalid request. Existing public error + translation remains authoritative. + +Internal names may follow local conventions, but implement these responsibilities +once. Do not carry ConfigKit objects into execution, reread configuration, query +history during execution, or serialize filesystem objects. The selected root is +not a list to search. Root selection happens before opening the requested file. + +## Stage 1 — Add Schema Value Shapes And Ownership + +Read `types.go`, `prompts.go`, `resource_inspection.go`, `convert.go`, domain +output contracts, and the existing schema-document ownership/decoder helpers. + +1. Add the public/configuration/inspection fields selected above, corresponding + internal fields, and the small resource-binding and validation-input values. + Keep selection data separate from the capabilities used to load it. +2. Extend `PromptSource` and output conversions. Preserve false and empty-object + presence; do not use truthiness or map length to decide selection. +3. Establish one shared output-selection check for the effective active mode. + Separate selection validity from schema compilation. Preserve format and + repair-attempt rules. For typed nil runtime inline values, use the standard + JSON null result before counting selections. +4. Add operation-owned inline decoding using the existing JSON document path. + Reuse trusted cloning after capture/decoding. Avoid an encode/decode round trip + at each getter or conversion; ordinary detached getters clone supported trees. +5. Update shallow output-contract/definition copies that now contain mutable + schema values. Do not change schema ownership by accidentally routing these + values through the provider-extra copier. + +Tests: extend focused domain selection and typed-source tests for nil/null, +`false`, `true`, `{}`, conflicting selections, blank references, and inactive +mode behavior. Add a focused runtime inline encoding failure case. Do not run +the full suite while downstream signatures are intentionally incomplete. + +Completion: representations and ownership rules are implemented, with no public +runtime filesystem API or truthiness-based loss of schemas. + +## Stage 2 — Capture Roots Through Native Configuration Inputs + +Work in `internal/configuration/load.go`, its normalization/capture helpers, +the new root source helper, `engine.go`, and internal engine construction. + +1. Add optional roots to decoded application/collection records. Validate only + final effective root values and obtain each root's effective source context. + Build application and collection root bindings once during construction, + using the existing shared metadata snapshot rather than scanning the full + resolved tree per prompt. +2. Capture the applicable root with each prompt under the accepted collection + scope rule. A declared collection root wins even if a subsequently selected + file is missing there. Null clearing restores application-root selection. +3. Implement `SchemaRootSource` and remove the legacy public and internal + schema-source configuration paths described above, including the implicit + default. Remove now-unused schema-specific defaults/imports, not unrelated + artifact-source machinery. +4. Wire the engine toward a source-neutral validator receiving operation inputs, + rather than constructing a validator tied to one engine-wide root. Update + affected configuration/engine tests and fixtures enough to express the new + root inputs; later stages finish preparation integration. + +Tests: root/collection capture with an explicit mapping context, root override +context from another source, null fallback, blank/missing context diagnostics, +and native layer precedence for the mapping helper. Use a counting filesystem +to demonstrate construction does not inspect schema file existence. + +Completion: root selection and context come solely from the final ConfigKit +configuration. No option-specific precedence or implicit root remains. + +## Stage 3 — Capture Prompt-Owned Schema Selections And References + +Work in `internal/configuration/prompts.go`, resource projections, normalization, +and prompt-definition snapshots/hashing. + +1. Extend output capture for all three fields. Validate exactly one effective + active selection, using ConfigKit's existing presence/null data when needed. + Do not reject overwritten contributions or reconstruct native composition. +2. Extract a reusable effective-field resource binder from `readContent`. + Preserve message-file read/close behavior; schema-file capture uses binding + only. Inherited references use their contributing field's context, not the + child's source. Root and file binding use existing path/context rules. +3. For configured active `schema_path`, require an applicable root and validate + path metadata without opening the file. Bind active `schema_file` from that + field's context. Inline selections need neither binding nor root. +4. Extend `PromptOutputResource`, `InspectPrompt`, and public projections to + include selected values/references without opening files. Keep detached + schema trees in public results and preserve existing inspection presence rules. +5. Include configured inline values and file-reference strings in the existing + definition-hash input; exclude resource capabilities. Use the existing hash + algorithm, not a new digest API or file-content read. Root context and schema + file bytes are not newly promised hash inputs. Leave both prepared-message + hash algorithms and their semantics unchanged. + +Tests: imports/inheritance and overrides preserve effective field context; +composition can merge inline mappings and explicitly clear alternatives; +same basenames in different filesystems stay distinct; inactive schema fields +do not trigger resource binding. Inspect a prompt with a missing schema file +successfully, then leave actual file failure for preparation. + +Completion: final selections, source bindings, inspection, and definition +snapshots agree without eager schema I/O or a second composition pass. + +## Stage 4 — Unify Inline And File Schema Preparation + +Work in `internal/validate/validator.go` and the current validator implementation. + +1. Adapt the preparer to `validate.Input`. Replace duplicated engine-root-specific + preparation routes with a source-neutral validator. Migrate internal validator + tests to bound files/inline documents; retain useful path/decoder tests at + their owning helper rather than keeping obsolete constructors just for tests. +2. Load a selected file once through the bound filesystem and existing + context-checked chunk reader. Preserve read/close error handling and the + single-complete-JSON-document decoder. Inline values use their captured or + operation-owned document without a file read. +3. Feed both routes through the existing dialect, self-contained-resource, + schema-location, and root compilation checks. Retain supported IDs, anchors, + internal references, filename independence, and external-resource rejection. + Keep existing canonical escaped retrieval URLs for file sources; use a stable + private absolute retrieval URL for inline documents, without new public IDs. +4. Store the selected document and compiled root in the existing prepared plan. + `SchemaDocument()` returning `false` is a document, not missing metadata. + Use the same value for local checks and provider constraints. +5. Preserve synchronous cancellation checks around reads, decoding, compilation, + and validation. No background dependency workers or new cache is needed. + +Tests: object and boolean schemas work by both routes; equivalent documents +give equivalent validation; invalid schema documents/dialects, external refs, +valid internal refs, exact file JSON numbers, and cancellation retain their +existing classifications. Reuse focused tests rather than duplicating the entire +schema suite for every route. + +Completion: one source-neutral preparation/compilation path produces the existing +operation-local plan with no extra resource reads for `$ref` resolution. + +## Stage 5 — Bind Effective Contracts In Every Execution Path + +Work in `internal/usecase/runner.go`, prepared execution, engine wiring, and +`smoke_inference_case.go`. + +1. Resolve the complete effective output contract first, then its file binding. + Retain surrounding root scope for runtime `SchemaPath`; discard configured + `schema_file` binding and configured inline value on a request replacement. + Store the selected input in preparation state and pass it to the preparer. +2. Keep malformed selection/root requirements within existing invalid-config + construction or invalid-request runtime boundaries. Read/decode/compile + failures remain operational validation errors before generation. Include + selected prompt/reference context while preserving error identities and + cancellation precedence. +3. Keep `Prepare`, `ValidateRequest`, `Run`, and `PrepareExecution` on the shared + preparation path. `RunPrepared` retains/reuses the plan and needs no schema + binding, file read, or compilation at execution time. Preserve callback, + operation-budget, admission, and continuation behavior. +4. Preserve prompt-ID-based structured-output naming and the existing provider + submission path. Repair reuses the same plan/document; it does not reselect + schema forms or roots. Do not invent provider-support assertions. +5. Migrate private smoke-schema setup to an explicit bound private resource and + the same preparer, independent of application schema roots. Update internal + fake preparers to the new signature; preserve the intentional nil-validator + behavior of focused use-case tests. + +Tests: runtime inline/path replacement, no borrowed configured selection, missing +runtime root, and collection-root selection during replacement. Existing +ordinary/prepared, validation-only, continuation, and smoke tests should compile +and exercise the common pipeline. Verify preparation failures do not call a model. + +Completion: all execution entry points select and retain exactly the intended +schema plan, without an out-of-band engine schema root. + +## Stage 6 — Verify Public Projections And Ownership + +Finish public facade conversion and serialization changes in `convert.go`, +`types.go`, `resource_inspection.go`, `json.go`, and prepared notification paths. + +1. Extend existing JSON/YAML contract tests for inline `false`, `{}`, absence, + inspection references, and backward decoding of values omitting new fields. + The prepared JSON projection already inherits public struct fields; avoid a + second serialization DTO. Preserve ordinary manual-value JSON behavior rather + than introducing schema compilation on JSON decode. +2. Test that mutating typed input after its source snapshot, inspection results, + `Prepare` results, `Details`, `OnPrepared`, or injected-client schema metadata + cannot alter captured configuration or retained execution. Extend existing + ownership fixtures rather than duplicating every handle lifecycle test. +3. Keep configured file-reference inspection separate from the schema document + shown in prepared `StructuredOutput`. Public runtime `OutputContract` has no + `SchemaFile`; do not fabricate an inline selection or path merely to fill that + projection. The prepared structured-output document is the selected content. + `ValidationResult.SchemaPath` keeps its explicit-root path meaning and is + empty for the new inline/source-relative forms; no synthetic sentinel path or + new result field is needed. +4. Check every remaining shallow output-contract copy now containing `Schema`. + Trusted internal sharing stays immutable; public copies detach mutable trees. + Do not marshal filesystem bindings, or recompile schemas in inspection/getters. + +Tests: focused typed-source, resource-inspection, prepared/notification, and JSON +contract packages. Retain schema-document tests that exceed provider-extra copy +limits; new inline handling must not route trusted documents through those limits. + +Completion: all exposed values are faithful and caller-owned where the existing +API requires it, with no resource capability leakage or extra schema work. + +## Stage 7 — Verify Mixed Collections And Captured Lifecycles + +Add focused external-package integration coverage using real ConfigKit +composition and injected provider/transport boundaries. Prefer a small fixture +with distinct risks over a Cartesian product of settings. + +1. Build one engine with independent collection roots, application-root fallback, + source-relative schemas, and inline schemas together. Mix embedded-like and + operator filesystem contexts, with identical schema basenames. Replace one + prompt/schema selection while unrelated prompts remain usable. +2. Cover inherited source context and overridden context through actual imports + or layers, including a root inherited from a different source than its prompt. + Exercise explicit null when switching forms and ConfigKit's native whole-value + replacement control. No custom overlay behavior belongs in expected results. +3. Confirm missing files under the chosen root fail without another root search. + Missing/malformed schema files permit engine construction but fail preparation; + missing selection/context/root metadata fails at its configured boundary. +4. Capture a prepared execution, then modify/remove its schema source. The handle + executes and repairs against its captured plan; a later ordinary preparation + observes the changed source. Count reads to protect capture behavior, not + incidental compiler-internal call counts. +5. Confirm equivalent forms submit the selected document to an offline recording + client and validate generated results with those same constraints. Include a + boolean-schema case. Preserve transport error behavior rather than testing + assumptions about live provider support. + +Completion: the mixed Scriptorium/Notarius layout and prepared-plan lifecycle are +covered end to end without application-managed mounts or live infrastructure. + +## Stage 8 — Migrate Documentation And Validate The Assembled Sprint + +1. Update GoDoc and [formats.md](../formats.md) for selection/null/composition, + source context, root scope, inactive modes, preparation failures, and removed + implicit root. Update the [consumer guide](../consumers/pkg-promptkit.md) with + runtime inline replacement and context-backed root/helper migration examples. + Keep exact API declarations in GoDoc and configuration rules in the format + reference; link rather than repeat them everywhere. +2. Update [sources.md](../internal/sources.md), + [runner.md](../internal/runner.md), and the internal overview where responsibilities + changed. Adapt maintained examples/fixtures referencing removed schema options + to inline, source-relative, or explicit context-backed roots as appropriate. + Keep one representative maintained mixed-source example rather than adding an + application asset assembly tutorial. +3. Make the canonical migration documentation cover positional public struct + literals, removed schema options/field, and the absence of an implicit root. + Release preparation will select the version and summarize these changes in + its release note under the release policy. This sprint selects no release + number and does not rewrite published notes or require a publication check. +4. Audit the complete diff against the feature roadmap. Remove old schema-source + wiring, obsolete helper-only tests/defaults, and stale current-state references. + Historical published notes and the original request are not current API owners. + Do not remove unrelated path/artifact machinery or change dependency versions + or the Go consumer minimum for this feature. +5. Follow the development guide's newest locally installed Go selection with + `GOTOOLCHAIN=local` and `GOWORK=off`. Format changed Go files; run + `go test -race ./...`, `go vet ./...`, and `git diff --check`. The complete suite + includes maintained examples; no repeated example gate or benchmark run is + required. Check changed local documentation links. Fix failures using focused + reruns and validate the final assembled changes proportionately. + +Completion: all stages' behavior and tests are implemented, documentation describes +the new configuration/API migration, and assembled validation succeeds. Do not tag, +push, publish dependencies, or require external consumer updates to finish this +sprint. Retire completed roadmaps during normal release preparation. + +## Open Questions + +None. The three product decisions are accepted in the feature roadmap; this plan +selects the remaining API, helper migration, and preparation plumbing details. diff --git a/docs/roadmap/prompt-owned-schemas-request.md b/docs/roadmap/prompt-owned-schemas-request.md new file mode 100644 index 0000000..615a7c3 --- /dev/null +++ b/docs/roadmap/prompt-owned-schemas-request.md @@ -0,0 +1,166 @@ +# Feature Request: Prompt-Owned Inline And Source-Relative Schemas + +**Status:** Requested by the Notarius consumer; proposed behavior, not an +implemented API or an approved implementation plan. + +## Requested Capability + +Allow a prompt's structured-output schema to be supplied directly inline in its +output definition or loaded from a file relative to the source that defines the +schema reference. Use the same effective resource-context model already used by +`content_file`, including imports, inheritance, embedded filesystems, and operator +sources. + +The guiding user observation is that a structured-output schema is part of a +prompt definition. Users generally expect it to behave like any other part of +that definition: they can provide it inline or place it in a separate file loaded +relative to the prompt where it is defined. Requiring a separately assembled, +engine-wide schema filesystem makes that ordinary mental model unnecessarily +hard to express. + +## Consumer Problem + +Notarius is moving to native PromptKit declarations, with embedded defaults and +higher-priority operator prompt sources. It wants to remove its asset registries, +synthetic mounts, file copying, flattened schema namespace, and separate schema +identity/digest machinery. + +PromptKit already composes prompt definitions and resolves message files using +their source contexts. Schema paths currently resolve against a separately +selected schema root. Replacing an embedded prompt with an operator prompt can +therefore require changing the schema source for the whole engine, assembling a +custom filesystem, or copying unrelated default schemas into the replacement +root. An inline schema cannot currently express the simpler alternative either. + +The desired arrangement is straightforward: + +```text +embedded/ + spell-extraction.yml + spell-response.json +operator/ + custom-extraction.yml + custom-response.json +``` + +Each prompt references its nearby schema, or embeds that schema directly. +Replacing one prompt leaves unchanged prompts and their schemas usable. PromptKit +owns this reusable resource resolution; Notarius supplies prompt content and +retains domain interpretation and artifact validation. + +## Why The Current Behavior Appears To Exist + +The original public engine facade exposed separate prompt, profile, and schema +sources (see repository history at `e4899fb`). The current +[`WithSchemaFS` option](../../engine.go) still selects one schema source, and +[internal engine wiring](../../internal/engine/engine.go) constructs a validator +bound to that source. + +This suggests an inheritance from the earlier architecture, rather than a +fundamental requirement that every prompt share one schema root. That is an +inference from the implementation and history, not a documented statement of +the original author's intent. A central root remains useful when an application +selects output contracts programmatically; the request is to support ordinary +prompt-owned schemas as well. + +The newer configuration model already supplies the relevant foundation. +[Configuration ownership ADR-0001](../adr/0001-configuration-boundary.md) includes +effective resource context in the ConfigKit handoff, and the +[`content_file` contract](../formats.md#messages-and-templates) resolves each +effective reference using its own source context. Schema references should have +an equivalent native path through this model. + +## Proposed Consumer Experience + +These output fragments illustrate the two requested forms. `schema` and +`schema_file` are suggested names only; exact syntax and public Go API design +remain upstream decisions. They do not describe currently supported fields. + +Inline: + +```yaml +output: + format: json + validation_mode: json_schema + schema: + type: object + properties: + summary: + type: string + required: [summary] + additionalProperties: false +``` + +File-backed: + +```yaml +output: + format: json + validation_mode: json_schema + schema_file: ./summary-response.json +``` + +Both forms should feed the same native preparation, structural validation, +provider structured-output, and repair behavior. No application schema loader +or registry should be necessary to use them. + +## Resource And Composition Semantics + +- A file reference uses the source context of the effective schema-reference + field, rather than a process working directory or the final child prompt's + location regardless of origin. +- An inherited reference retains its originating context. An overridden reference + uses the overriding source's context. Imports and collection composition follow + the same native rules as other resource-bearing fields. +- Embedded sources and operator files can coexist within one engine. Existing + filesystem-root rules apply; this does not request a second path-security model. +- Inline schema values participate in native configuration composition. Define + presence, replacement/clearing, and selection between inline and file-backed + forms explicitly, using ConfigKit's effective values rather than reconstructing + configuration history in the runner. +- Retain an explicit schema-source route for programmatic request contracts and + existing root-based consumers. Define how it interacts with prompt-owned forms; + avoid ambiguous multi-root searching or silently reinterpreting existing paths. +- Preserve preparation-time loading for schema files and the prepared execution's + captured validation plan. `RunPrepared` should execute with the schema captured + during preparation rather than reopening the source. + +This provides composition through schema references. It does not require a +filename-based schema overlay system: an operator replacement explicitly selects +its schema, while unchanged prompts continue using theirs. + +## Self-Contained Schemas Remain A Separate Concern + +The self-contained-schema decision has a concrete rationale. Local validation +could resolve external files that were absent from the single schema object sent +to the provider. Repository history at `c542b81` records the decision to remove +that mismatch without introducing schema bundling. + +That rationale does not require a global schema root. PromptKit can load a +self-contained schema beside its prompt, or accept it inline, and submit the same +complete object. Preserve the current [self-contained schema semantics](../formats.md#schemas), +including supported in-document references. Loading the root schema from a +prompt-relative file does not imply loading external JSON Schema `$ref` resources. +Schema bundling and network resource loading are outside this request. + +## Suggested Verification And Documentation + +- Equivalent inline and file-backed schemas produce equivalent local validation + and provider constraints. +- Embedded and operator prompts with different schema files work in one engine, + including files with the same basename in different source contexts. +- Replacing one prompt does not require copying or replacing unrelated schemas. +- Imported/inherited references preserve their defining context; overridden + references resolve against the new context independently of working directory. +- Inline/file selection and clearing follow the documented native composition + rules, including deliberate errors for unresolved or ambiguous selections. +- Existing explicit-root and request-level contract behavior has a deliberate, + documented compatibility path. +- Missing or invalid schema files produce useful errors identifying the relevant + prompt/reference. Self-contained-schema behavior remains consistent for both + new forms, and prepared executions retain their captured schema. + +Document the selected syntax, source-context behavior, request-level interaction, +and migration in the format reference, GoDoc, and consumer examples when +implemented. The request does not select a release number or require a new +application-owned asset assembly layer as an interim design. diff --git a/docs/roadmap/prompt-owned-schemas.md b/docs/roadmap/prompt-owned-schemas.md index 60de0e9..70cafd2 100644 --- a/docs/roadmap/prompt-owned-schemas.md +++ b/docs/roadmap/prompt-owned-schemas.md @@ -1,7 +1,9 @@ # Prompt-Owned Inline And Source-Relative Schemas -Status: accepted design decisions and implementation roadmap. All three decisions -are settled. Examples describe target, not implemented, syntax. +Status: accepted feature roadmap. All three decisions are settled. Examples +describe target, not implemented, syntax. The companion +[implementation plan](implementation.md) owns staged execution and concrete API +and migration details. This document authorizes no code changes and selects no release number. ## Purpose And Boundary @@ -137,9 +139,9 @@ a root or migrate their schema selection. Retain the ability to supply explicit schema directories and filesystems through Go configuration helpers, but fit those helpers into ConfigKit's normal source composition. Do not give Go options a special precedence rule or retain a second -runtime override pass. API planning must spell out the helper adaptation and any -migration required; unchanged legacy option behavior is not a requirement where -it conflicts with this decision. +runtime override pass. The companion implementation plan specifies helper +adaptation and migration; unchanged legacy option behavior is not a requirement +where it conflicts with this decision. ### Effective Source Context And Composition @@ -223,7 +225,7 @@ schema pipeline. Do not treat `false` as absence or impose an object-only rule merely for a convenient Go representation. Use existing JSON-compatible handling rather than introduce a numeric-conversion adapter or precision guarantees beyond those of the selected input path. File-backed JSON decoding retains its existing -behavior. Exact public Go types remain API-planning work. +behavior. The companion implementation plan selects concrete public Go types. Preserve existing structured-output naming, validation/repair semantics, and hash responsibilities. Account for new configured fields in existing projections and @@ -240,37 +242,6 @@ the existing independence from the retrieval filename. Use the same schema value for local validation and provider constraints. Schema bundling, remote loading, and new provider-specific dialects are outside this work. -## Work From The Baseline - -1. **Specify concrete API shapes.** Translate the accepted decisions into YAML, - typed prompt sources, and request-level overrides. Define the Go representation - of inline values, preserving absence, empty objects, and boolean schemas. - Specify helper migration without changing ConfigKit's composition rules. -2. **Capture schema ownership.** Extend configuration decoding, normalization, - domain selection, and inspection. Share the existing effective-field resource - resolution mechanics where useful, but separate resource binding from file - reading so schema bytes remain preparation-time data. Capture optional roots - and their effective contexts, then apply the accepted scope-selection rule. -3. **Unify schema preparation.** Extend the validator's input route to consume - the selected inline value or file source. Retain one compilation/plan path for - local checks, provider metadata, and repairs. Adapt explicit-root source helpers - to native ConfigKit composition and remove the implicit root; avoid parallel - validators with divergent semantics. -4. **Complete public integration.** Update output-contract conversion, typed - helpers, prompt inspection, preparation details, and ordinary/prepared runs. - Check request replacement, continuation, and smoke-test paths that use output - contracts. Preserve detachment and avoid leaking filesystem capabilities into - generically renderable inspection values. -5. **Verify and document.** Add focused offline coverage below; update the format - reference, GoDoc, consumer examples, and internal source/validation documentation - when implementation lands. Show embedded and operator collections coexisting, - switching forms, and configuring optional roots. Document migration from the - implicit root and any Go helper/API changes without selecting a release here. - -Internal refactoring is permitted; these steps specify behavior and ownership, -not a permanent package layout. No interim application asset assembly layer is -needed to make the feature usable. - ## Validation Targets Use the consumer request's scenarios with real configuration composition and @@ -375,10 +346,3 @@ runtime-selected bundles by reusing PromptKit's reading, decoding, diagnostics, and preparation lifecycle. It adds a public resource type and ownership/lifetime semantics. Revisit only for a concrete consumer need; configured collections do not require it. Neither approach needs implicit working-directory lookup. - -## Remaining Planning Work - -No product decisions remain open among Decisions 1–3. Implementation planning -still needs concrete public Go shapes and the adaptation/migration of existing -schema-source helpers within the accepted ConfigKit composition model. These -details must preserve the accepted scope and selection rules.