Files
promptkit/docs/roadmap/prompt-owned-schemas.md

19 KiB

Prompt-Owned Inline And Source-Relative Schemas

Status: proposed target and implementation roadmap. Decision 1 is accepted; Decisions 2 and 3 remain open. Examples describe target, not implemented, syntax. This document authorizes no code changes and selects no release number.

Purpose And Boundary

Let prompt collections carry their own structured-output schemas, inline or in nearby files, without requiring applications to assemble an engine-wide schema filesystem. An operator can replace one embedded prompt and select its schema while unrelated prompts continue using their own resources.

Notarius's request supplies the consumer need: remove application asset registries, synthetic mounts, copying, and a flattened schema namespace. PromptKit owns reusable resource resolution and structural validation; Notarius retains content interpretation, domain validation, and artifact handling.

Follow the ConfigKit handoff boundary. ConfigKit owns imports, inheritance, composition, and effective field context. PromptKit interprets the final result. Neither the configuration adapter nor the runner reconstructs source history to decide which schema should win.

Current Baseline

The current checkout includes ConfigKit v1.3.0 and the changes described in the v0.24.0 release notes. Schema support still has these boundaries:

  • Output contracts select schemas through SchemaPath. Configuration capture copies that string into the domain contract; no inline or source-relative schema selection is captured.
  • Schema source options configure a root used by the engine's validator, with an implicit default directory of ".". The schema format contract defines existing path meaning, dialect, self-contained resources, and operational-error behavior.
  • Message files already resolve using the effective field's defining source context. Message content is read at engine construction, while schema files are read during preparation.
  • Schema preparation creates an operation-local plan containing the schema document and compiled validator. Local validation, structural repair, and provider structured-output metadata use that plan. RunPrepared reuses its captured plan without reopening resources.
  • A request-level output contract replaces the entire configured contract. Schema path is currently required for JSON Schema mode and ignored by other modes.

The change needs new selection and resource plumbing, not another schema compiler or application-managed overlay system.

Target End State

Schema Forms

The accepted configuration forms are:

Field Meaning
schema An inline schema value in the prompt's output definition.
schema_file A schema file resolved using that effective field's source context.
schema_path A reference within the applicable explicitly configured schema root.

Inline and source-relative forms are the new capability. The explicit-root route remains available, with optional collection-specific roots and an application-wide default. Decisions 2 and 3 below settle selection and API details. There is no multi-root fallback search or implicit working-directory root.

Illustrative inline output fragment:

output:
  format: json
  validation_mode: json_schema
  schema:
    type: object
    properties:
      summary: {type: string}
    required: [summary]
    additionalProperties: false

The same output can instead select schema_file: ./summary-response.json, whose JSON document expresses that schema. Equivalence means equivalent schema values, validation, and submitted constraints, not identical YAML/JSON source bytes or an assurance that every provider supports every locally valid schema.

Optional Schema Roots

Accept an optional schema_root at the configuration root and within each collections entry. A collection can manage schemas centrally without requiring other collections to adopt that layout. Inline and source-relative forms remain available whether or not a root is configured.

Illustrative configuration:

schema_root: ./application-schemas
collections:
  weatherreporter:
    schema_root: ./schemas
    prompts:
      - id: summarize.alerts
        messages:
          - role: user
            content: Summarize the supplied alerts.
        output:
          format: json
          validation_mode: json_schema
          schema_path: alerts.schema.json

ConfigKit owns composition, layering, inheritance, and effective source context for these fields, just as for other configuration. PromptKit consumes the final values without another override or merge pass. Resolve each declared root against the defining field's effective filesystem and base directory; an overridden root uses the overriding source's context. This supports embedded and host filesystems without assuming that a root is a host path.

After composition, use the selected prompt's collection root when present; otherwise use the application-wide root. Root-level prompts use the application root. This is a scope-selection rule, not an exception to ConfigKit composition. Select one root before opening the schema: a missing file under that root is an error, not a reason to search another root or the prompt's directory.

No root is needed for schema or schema_file. Selecting schema_path without an applicable root is a configuration problem. Remove the implicit "." root as an intentional breaking change; callers relying on it must explicitly select a root or migrate their schema selection.

Retain the ability to supply explicit schema directories and filesystems through Go configuration helpers, but fit those helpers into ConfigKit's normal source composition. Do not give Go options a special precedence rule or retain a second runtime override pass. API planning must spell out the helper adaptation and any migration required; unchanged legacy option behavior is not a requirement where it conflicts with this decision.

Effective Source Context And Composition

Resolve schema_file using the effective field's defining filesystem and base directory, as for content_file. Inherited references retain their origin; overridden references use the overriding source. A child prompt's location does not replace an inherited field's context. Imports and named collections follow the same native rules.

Embedded defaults and operator sources can coexist, including different files with the same basename. Replacing a prompt explicitly selects its schema; it need not replace the engine-wide root or copy unrelated default schemas.

Use existing rooted-filesystem and usable-context rules. Do not add a new path security model, automatic mounts, or schema search paths. A source-relative reference without a usable context needs a diagnostic identifying the field and prompt, rather than an inferred working directory.

Inline values participate in native ConfigKit composition. Interpret selection, null clearing, and conflicts from the final effective values under Decision 2. No PromptKit pass repairs or replays configuration composition. A source-neutral runtime representation may retain the filesystem capability and resolved resource reference needed for preparation; runtime components need not depend on ConfigKit objects or inspect configuration history.

Preparation And Captured Execution

At construction, capture the effective inline value or the file reference and its resource context. Validate selection and context metadata at the configuration boundary. Keep file reading and schema compilation at preparation time; do not read all schema files at construction merely to copy message-loading behavior.

Every selected form feeds the existing validation-plan pipeline. Inline values need no file read; file forms load the selected root document for the operation. Compile and retain that document for local validation and provider submission, then reuse the plan for any structural repair. Preserve existing cancellation, error classification, dialect, and self-contained-schema semantics.

PrepareExecution retains the prepared schema and validator. RunPrepared does not reopen the schema even if its source changes or disappears. Later ordinary runs may read the changed file during their own preparation. Inline configuration remains part of the engine's captured configuration. This does not introduce schema watching, hot reload, or cross-operation caching requirements.

Missing, malformed, or unresolvable selected schema files fail preparation through the existing operational validation-error boundary, with useful prompt/reference context. Invalid generated content remains an ordinary failed validation result. Invalid configuration selection or missing source context is a configuration problem. Avoid promising diagnostics for nonexistent partial validation plans.

Public API And Inspection

Keep complete request-level output-contract replacement. A replacement must not retain the prompt's schema selection or resource context implicitly. The recommended programmatic forms are inline values and the existing explicit-root path; Decision 3 addresses source-relative runtime files.

Extend typed prompt configuration helpers, public output contracts, conversions, and inspection consistently. Inline values and exposed details must follow existing ownership patterns so caller mutation cannot alter captured configuration or a prepared execution. Configuration inspection describes the resolved inline value or reference without opening schema files. Prepared details continue showing the schema document actually selected for the operation.

Accept inline schema values through native configuration/public value mechanisms, including meaningful empty objects and boolean schemas supported by the existing schema pipeline. Do not treat false as absence or impose an object-only rule merely for a convenient Go representation. Use existing JSON-compatible handling rather than introduce a numeric-conversion adapter or precision guarantees beyond those of the selected input path. File-backed JSON decoding retains its existing behavior. Exact public Go types remain API-planning work.

Preserve existing structured-output naming, validation/repair semantics, and hash responsibilities. Account for new configured fields in existing projections and hash inputs where appropriate, without promising that a prompt or message hash identifies runtime schema file contents. No new schema registry, public schema-ID system, or digest API is required by this request.

Self-Contained Resources

Root schema location and JSON Schema reference resolution remain separate. Loading a root beside its prompt does not enable external $ref files or network loading. Preserve supported in-document IDs, anchors, and references, including the existing independence from the retrieval filename. Use the same schema value for local validation and provider constraints. Schema bundling, remote loading, and new provider-specific dialects are outside this work.

Work From The Baseline

  1. Settle configuration and API selection. Resolve the decisions below and sketch YAML, typed prompt sources, and request-level overrides. Specify absence/null, empty values, inactive validation modes, and representation of inline values without changing ConfigKit's composition rules.
  2. Capture schema ownership. Extend configuration decoding, normalization, domain selection, and inspection. Share the existing effective-field resource resolution mechanics where useful, but separate resource binding from file reading so schema bytes remain preparation-time data. Capture optional roots and their effective contexts, then apply the accepted scope-selection rule.
  3. Unify schema preparation. Extend the validator's input route to consume the selected inline value or file source. Retain one compilation/plan path for local checks, provider metadata, and repairs. Adapt explicit-root source helpers to native ConfigKit composition and remove the implicit root; avoid parallel validators with divergent semantics.
  4. Complete public integration. Update output-contract conversion, typed helpers, prompt inspection, preparation details, and ordinary/prepared runs. Check request replacement, continuation, and smoke-test paths that use output contracts. Preserve detachment and avoid leaking filesystem capabilities into generically renderable inspection values.
  5. Verify and document. Add focused offline coverage below; update the format reference, GoDoc, consumer examples, and internal source/validation documentation when implementation lands. Show embedded and operator collections coexisting, switching forms, and configuring optional roots. Document migration from the implicit root and any Go helper/API changes without selecting a release here.

Internal refactoring is permitted; these steps specify behavior and ownership, not a permanent package layout. No interim application asset assembly layer is needed to make the feature usable.

Validation Targets

Use the consumer request's scenarios with real configuration composition and controlled local/embedded filesystems. Keep provider checks offline through the existing injected-client or transport seams.

  • Equivalent inline, source-relative, and explicit-root documents produce the same local constraints and provider schema value through ordinary preparation.
  • Embedded/operator prompts coexist with identical schema basenames; replacing one prompt leaves unrelated schema selections usable.
  • Imports and inheritance retain effective field context; overrides change it. Resolution is independent of process working directory and observes existing root/context rules.
  • Collection and application roots follow native ConfigKit composition and effective field context, including roots supplied through Go helpers. Collection scope wins when configured; otherwise application scope applies. Missing files never trigger a search of another root. Inline/source-relative schemas require no root, and schema_path without an applicable root fails clearly.
  • Native inline composition, null clearing, conflicting forms, empty objects, boolean schemas, and malformed selections follow the chosen contract.
  • Request replacement does not borrow the prompt's schema or context. Explicit schema source helpers follow the accepted composition model and documented migration, without a special Go-option precedence rule.
  • Missing/invalid files fail at preparation, not eager engine construction; prepared execution survives later source modification/removal. Inline input and returned-detail mutation do not alter retained execution state.
  • Existing dialect and external-reference rejection remain consistent across forms; supported in-document references still work. Source-relative loading introduces no extra $ref resource reads.
  • Inspection remains generically renderable and respects its configured-resource boundary. Prepared details and provider submission reflect the selected plan.
  • Representative validation/repair, continuation, cancellation, and smoke paths continue using the common plan without duplicating schema loading or compilation beyond existing self-contained checks.

Decisions And Remaining Questions

1. How Should New File References Coexist With schema_path?

Accepted: add schema and schema_file while preserving schema_path for an explicitly selected root. Roots are optional and configurable per collection and application, as specified in Optional Schema Roots. Use ConfigKit's standard composition and effective source contexts, with no special Go-option precedence. Remove the implicit "." root. The three forms support different collection layouts without requiring application asset assembly or fallback searches; migration must cover the deliberate default-root change and any necessary adaptation of Go helpers.

Alternative considered: replace the selection with one tagged schema-source value covering inline, relative-file, and explicit-root forms. This makes alternatives structurally explicit and may simplify future API representation. It requires configuration and Go migration, or a compatibility translation layer. Separate fields were selected to keep each reference form's meaning direct; the accepted root changes do not reinterpret schema_path as a source-relative file.

2. How Should Selection, Composition, And Clearing Work?

Status: open.

Recommendation: in JSON Schema mode, require exactly one nonnull effective selection among the three forms. Null clears a selection; {} and false remain schema values, not absence. Blank file/path selections are invalid rather than fallback instructions. Preserve existing inactive-mode behavior for legacy paths and apply a consistent rule to new forms rather than introducing unrelated restrictions on unused fields.

Inline mappings compose according to ConfigKit. Switching forms may require explicitly clearing the inherited alternative, for example schema_file: null when supplying schema. Whole-value replacement uses ConfigKit's composition controls; PromptKit does not invent its own replacement pass. This is predictable from final configuration and aligns with the existing message-content selection pattern, though it requires an explicit clear when switching inherited forms.

Alternative: define fixed precedence, such as inline over source-relative file over legacy path. This also works on final effective values and makes a higher-priority form convenient without clearing another field. It can silently leave stale alternatives that reappear when the preferred form is cleared, and makes the effective configuration less obvious to operators. Explicit conflict errors are preferable unless consumers demonstrate substantial value in fallback.

3. How Much Source Selection Should Programmatic Requests Expose?

Status: open.

Recommendation: support inline values and existing SchemaPath in request contracts; keep source-relative schema_file tied to configuration inputs with resource context. Typed configuration sources can supply that context through ConfigKit just as they do for message files. Request replacement is complete and never borrows the prompt's source implicitly. This meets the consumer request without making each runtime output contract a filesystem configuration API.

Alternative: also let runtime contracts supply an explicit filesystem/base context with a schema file reference. That supports per-request schemas from several filesystems without an engine-wide root or caller preloading. It adds source ownership, detachment, and serialization questions to the public runtime API. It is viable for a demonstrated runtime-source use case, but not necessary for prompt-owned collections or inline programmatic schemas. Neither approach needs an implicit working-directory or multi-root lookup.