24 KiB
Prompt-Owned Schemas Implementation Plan
Objective And Authority
Implement prompt-owned-schemas.md. That feature roadmap owns the accepted selection, composition, scope, ownership, and migration decisions. The consumer request is background, not authority for application-specific behavior. Respect ADR-0001: interpret only ConfigKit's final effective configuration and its effective resource context.
The baseline is commit adfccb5, with ConfigKit v1.3.0 and the prepared
message-content hash. This is one end-to-end sprint, divided into sequential
single-prompt stages for a Luna coding agent. Intermediate stages may fail to
compile. Do not add compatibility shims or duplicate implementations merely to
make an intermediate commit coherent. No stage requires human approval, external
publication, or a dependency upgrade. Commit completed units under repository
policy; commits are not review gates.
Start with development.md and the applicable agent, architecture, testing, and documentation policies. Read the format reference, source/validation internals, and existing GoDoc before changing their contracts. Reuse existing file-path, schema compilation, cancellation, and ownership machinery where applicable.
Concrete Design For This Sprint
Public Values And Configuration
- Add
Schema anywithjson:"schema,omitempty"to publicOutputContract. Nil means no inline selection; a nonnil empty object andfalseare selections. PreserveSchemaPath stringand its existing JSON field. Do not add runtimeSchemaFile, filesystem handles, or a standalone schema-loading API. - Add
Schema anyandSchemaFile *stringtoPromptOutputConfig, retainingSchemaPath *string.PromptSourceemitsschemaonly for a nonnull inline value and emits each nonnil reference pointer. As with existing typed helpers, raw ConfigKit mappings express explicit null and composition controls. - Add
Schema anyandSchemaFile *stringtoPromptOutputInspection, withschemaandschema_fileJSON/YAML names andomitempty. Preserve its existing path presence. Expose configured values/references, not filesystem capabilities or file contents. Generic configuration inspection already exposes declared application and collection roots; no additional public root-inspection API is needed. - Decode optional
schema_rootstrings at the application root and collection entries. Absent/null collection roots select the application root; absent/null application roots mean no root. A nonnull blank root is invalid. An explicitly supplied"."is valid when accompanied by usable source context. - In JSON Schema mode, exactly one nonnull effective
schema,schema_file, orschema_pathis required. Reject blank file/path selections and conflicts. Outside that mode, do not resolve, compile, or impose new selection checks on unused schema fields. Retain ordinary ConfigKit decoding semantics.
Use native ConfigKit values for captured configuration. For active runtime inline values, use standard JSON encoding and the existing schema JSON decoder to obtain an operation-owned document; do not introduce a custom numeric bridge or the bounded provider-extra copier. JSON null, including typed nil values encoded as null, means no selection. Empty objects and booleans must survive. Unsupported/cyclic Go values that cannot be encoded fail as invalid request values, not panics. JSON-compatible but invalid schema documents fail compilation at preparation through the existing validation-error boundary. Preserve existing file JSON decoding and its numeric behavior; promise no new cross-input numeric equivalence. Clone captured, already-supported trees using the existing trusted document ownership path, without provider-extra depth/node limits.
Schema-Root Helper Migration
Replace the out-of-band root settings with one mapping helper:
func SchemaRootSource(label, root string) configkit.Source
It returns configkit.Mapping(label, map[string]any{"schema_root": root}),
without opening files or validating raw contributions. Callers place its input
at their chosen native ConfigKit layer and provide a filesystem context, just as
they do for PromptSource. For example, for an fs.FS:
filesystem := configkit.NewFilesystem("application", schemaFS)
rootInput := configkit.Input{
Source: promptkit.SchemaRootSource("application schemas", "schemas"),
Filesystem: &configkit.FilesystemContext{
FS: &filesystem, Root: ".", BaseDir: ".",
},
}
// Supply rootInput in configkit.Underlay{Inputs: []configkit.Input{rootInput}},
// or use &rootInput as Main when no other main input is needed.
Remove Config.SchemaDir, WithSchemaFS, and WithSchemaFile, and the internal
option setters/flags used solely by those APIs. This is an intentional pre-1.0
API migration, not a deprecation period. Do not silently translate options into
a highest-priority layer or preserve last-option-wins behavior. Callers needing
collection-specific roots use ordinary collection mappings with context; the
helper need not grow collection-specific parameters.
Migration: a directory or filesystem becomes a context-backed root mapping;
a former single-file source becomes a schema_file on a configured prompt or a
root mapping plus its basename in schema_path. For an unrelated per-request
file, the caller reads/decodes it and supplies Schema inline. Host-directory
examples use an explicit os.DirFS/ConfigKit filesystem context. Do not infer a
working-directory root. This preserves the requested directory/filesystem
capabilities without preserving a second source-selection system.
Internal Resource And Preparation Model
Separate serializable selections from resource capabilities:
- Extend domain output selections with inline
Schemaand the configuredSchemaFilereference string. The latter is internal, not a new request field. - Capture an optional applicable root and optional source-relative file binding
with the prompt definition. Use small source-neutral domain values containing
an
fs.FSand resolved filesystem path. Exclude those capabilities from JSON and definition hashing. Keep the root separate from the output contract so it remains available to a runtimeSchemaPathreplacement. - Configuration binding resolves a field's effective filesystem/base directory
using the current effective-origin lookup and
SourceContext, asreadContentdoes. Extract binding from reading so schema capture does not callOpen,Stat, or compile a schema to establish file existence. Apply existingfilecatalogpath containment rules. Missing context/invalid reference metadata is a configuration error; missing or malformed file data is a preparation error. - Add internal
validate.InputcontainingContract domain.OutputContractand optionalFilebinding. ChangeValidationPreparer.PrepareValidationto accept this input. Non-schema modes need no file. JSON Schema mode selects the inline document or the one bound file and uses one common compilation pipeline. - After complete request-contract replacement, the runner selects the binding:
configured
schema_fileuses its captured binding only without a replacement;SchemaPathbinds under the selected prompt's applicable root; inline values need no file. A replacement never borrows the configured inline/file selection. A missing root for a runtime path is an invalid request. Existing public error translation remains authoritative.
Internal names may follow local conventions, but implement these responsibilities once. Do not carry ConfigKit objects into execution, reread configuration, query history during execution, or serialize filesystem objects. The selected root is not a list to search. Root selection happens before opening the requested file.
Stage 1 — Add Schema Value Shapes And Ownership
Status: complete.
Read types.go, prompts.go, resource_inspection.go, convert.go, domain
output contracts, and the existing schema-document ownership/decoder helpers.
- Add the public/configuration/inspection fields selected above, corresponding internal fields, and the small resource-binding and validation-input values. Keep selection data separate from the capabilities used to load it.
- Extend
PromptSourceand output conversions. Preserve false and empty-object presence; do not use truthiness or map length to decide selection. - Establish one shared output-selection check for the effective active mode. Separate selection validity from schema compilation. Preserve format and repair-attempt rules. For typed nil runtime inline values, use the standard JSON null result before counting selections.
- Add operation-owned inline decoding using the existing JSON document path. Reuse trusted cloning after capture/decoding. Avoid an encode/decode round trip at each getter or conversion; ordinary detached getters clone supported trees.
- Update shallow output-contract/definition copies that now contain mutable schema values. Do not change schema ownership by accidentally routing these values through the provider-extra copier.
Tests: extend focused domain selection and typed-source tests for nil/null,
false, true, {}, conflicting selections, blank references, and inactive
mode behavior. Add a focused runtime inline encoding failure case. Do not run
the full suite while downstream signatures are intentionally incomplete.
Completion: representations and ownership rules are implemented, with no public runtime filesystem API or truthiness-based loss of schemas.
Stage 2 — Capture Roots Through Native Configuration Inputs
Status: complete.
Work in internal/configuration/load.go, its normalization/capture helpers,
the new root source helper, engine.go, and internal engine construction.
- Add optional roots to decoded application/collection records. Validate only final effective root values and obtain each root's effective source context. Build application and collection root bindings once during construction, using the existing shared metadata snapshot rather than scanning the full resolved tree per prompt.
- Capture the applicable root with each prompt under the accepted collection scope rule. A declared collection root wins even if a subsequently selected file is missing there. Null clearing restores application-root selection.
- Implement
SchemaRootSourceand remove the legacy public and internal schema-source configuration paths described above, including the implicit default. Remove now-unused schema-specific defaults/imports, not unrelated artifact-source machinery. - Wire the engine toward a source-neutral validator receiving operation inputs, rather than constructing a validator tied to one engine-wide root. Update affected configuration/engine tests and fixtures enough to express the new root inputs; later stages finish preparation integration.
Tests: root/collection capture with an explicit mapping context, root override context from another source, null fallback, blank/missing context diagnostics, and native layer precedence for the mapping helper. Use a counting filesystem to demonstrate construction does not inspect schema file existence.
Completion: root selection and context come solely from the final ConfigKit configuration. No option-specific precedence or implicit root remains.
Stage 3 — Capture Prompt-Owned Schema Selections And References
Work in internal/configuration/prompts.go, resource projections, normalization,
and prompt-definition snapshots/hashing.
- Extend output capture for all three fields. Validate exactly one effective active selection, using ConfigKit's existing presence/null data when needed. Do not reject overwritten contributions or reconstruct native composition.
- Extract a reusable effective-field resource binder from
readContent. Preserve message-file read/close behavior; schema-file capture uses binding only. Inherited references use their contributing field's context, not the child's source. Root and file binding use existing path/context rules. - For configured active
schema_path, require an applicable root and validate path metadata without opening the file. Bind activeschema_filefrom that field's context. Inline selections need neither binding nor root. - Extend
PromptOutputResource,InspectPrompt, and public projections to include selected values/references without opening files. Keep detached schema trees in public results and preserve existing inspection presence rules. - Include configured inline values and file-reference strings in the existing definition-hash input; exclude resource capabilities. Use the existing hash algorithm, not a new digest API or file-content read. Root context and schema file bytes are not newly promised hash inputs. Leave both prepared-message hash algorithms and their semantics unchanged.
Tests: imports/inheritance and overrides preserve effective field context; composition can merge inline mappings and explicitly clear alternatives; same basenames in different filesystems stay distinct; inactive schema fields do not trigger resource binding. Inspect a prompt with a missing schema file successfully, then leave actual file failure for preparation.
Completion: final selections, source bindings, inspection, and definition snapshots agree without eager schema I/O or a second composition pass.
Stage 4 — Unify Inline And File Schema Preparation
Work in internal/validate/validator.go and the current validator implementation.
- Adapt the preparer to
validate.Input. Replace duplicated engine-root-specific preparation routes with a source-neutral validator. Migrate internal validator tests to bound files/inline documents; retain useful path/decoder tests at their owning helper rather than keeping obsolete constructors just for tests. - Load a selected file once through the bound filesystem and existing context-checked chunk reader. Preserve read/close error handling and the single-complete-JSON-document decoder. Inline values use their captured or operation-owned document without a file read.
- Feed both routes through the existing dialect, self-contained-resource, schema-location, and root compilation checks. Retain supported IDs, anchors, internal references, filename independence, and external-resource rejection. Keep existing canonical escaped retrieval URLs for file sources; use a stable private absolute retrieval URL for inline documents, without new public IDs.
- Store the selected document and compiled root in the existing prepared plan.
SchemaDocument()returningfalseis a document, not missing metadata. Use the same value for local checks and provider constraints. - Preserve synchronous cancellation checks around reads, decoding, compilation, and validation. No background dependency workers or new cache is needed.
Tests: object and boolean schemas work by both routes; equivalent documents give equivalent validation; invalid schema documents/dialects, external refs, valid internal refs, exact file JSON numbers, and cancellation retain their existing classifications. Reuse focused tests rather than duplicating the entire schema suite for every route.
Completion: one source-neutral preparation/compilation path produces the existing
operation-local plan with no extra resource reads for $ref resolution.
Stage 5 — Bind Effective Contracts In Every Execution Path
Work in internal/usecase/runner.go, prepared execution, engine wiring, and
smoke_inference_case.go.
- Resolve the complete effective output contract first, then its file binding.
Retain surrounding root scope for runtime
SchemaPath; discard configuredschema_filebinding and configured inline value on a request replacement. Store the selected input in preparation state and pass it to the preparer. - Keep malformed selection/root requirements within existing invalid-config construction or invalid-request runtime boundaries. Read/decode/compile failures remain operational validation errors before generation. Include selected prompt/reference context while preserving error identities and cancellation precedence.
- Keep
Prepare,ValidateRequest,Run, andPrepareExecutionon the shared preparation path.RunPreparedretains/reuses the plan and needs no schema binding, file read, or compilation at execution time. Preserve callback, operation-budget, admission, and continuation behavior. - Preserve prompt-ID-based structured-output naming and the existing provider submission path. Repair reuses the same plan/document; it does not reselect schema forms or roots. Do not invent provider-support assertions.
- Migrate private smoke-schema setup to an explicit bound private resource and the same preparer, independent of application schema roots. Update internal fake preparers to the new signature; preserve the intentional nil-validator behavior of focused use-case tests.
Tests: runtime inline/path replacement, no borrowed configured selection, missing runtime root, and collection-root selection during replacement. Existing ordinary/prepared, validation-only, continuation, and smoke tests should compile and exercise the common pipeline. Verify preparation failures do not call a model.
Completion: all execution entry points select and retain exactly the intended schema plan, without an out-of-band engine schema root.
Stage 6 — Verify Public Projections And Ownership
Finish public facade conversion and serialization changes in convert.go,
types.go, resource_inspection.go, json.go, and prepared notification paths.
- Extend existing JSON/YAML contract tests for inline
false,{}, absence, inspection references, and backward decoding of values omitting new fields. The prepared JSON projection already inherits public struct fields; avoid a second serialization DTO. Preserve ordinary manual-value JSON behavior rather than introducing schema compilation on JSON decode. - Test that mutating typed input after its source snapshot, inspection results,
Prepareresults,Details,OnPrepared, or injected-client schema metadata cannot alter captured configuration or retained execution. Extend existing ownership fixtures rather than duplicating every handle lifecycle test. - Keep configured file-reference inspection separate from the schema document
shown in prepared
StructuredOutput. Public runtimeOutputContracthas noSchemaFile; do not fabricate an inline selection or path merely to fill that projection. The prepared structured-output document is the selected content.ValidationResult.SchemaPathkeeps its explicit-root path meaning and is empty for the new inline/source-relative forms; no synthetic sentinel path or new result field is needed. - Check every remaining shallow output-contract copy now containing
Schema. Trusted internal sharing stays immutable; public copies detach mutable trees. Do not marshal filesystem bindings, or recompile schemas in inspection/getters.
Tests: focused typed-source, resource-inspection, prepared/notification, and JSON contract packages. Retain schema-document tests that exceed provider-extra copy limits; new inline handling must not route trusted documents through those limits.
Completion: all exposed values are faithful and caller-owned where the existing API requires it, with no resource capability leakage or extra schema work.
Stage 7 — Verify Mixed Collections And Captured Lifecycles
Add focused external-package integration coverage using real ConfigKit composition and injected provider/transport boundaries. Prefer a small fixture with distinct risks over a Cartesian product of settings.
- Build one engine with independent collection roots, application-root fallback, source-relative schemas, and inline schemas together. Mix embedded-like and operator filesystem contexts, with identical schema basenames. Replace one prompt/schema selection while unrelated prompts remain usable.
- Cover inherited source context and overridden context through actual imports or layers, including a root inherited from a different source than its prompt. Exercise explicit null when switching forms and ConfigKit's native whole-value replacement control. No custom overlay behavior belongs in expected results.
- Confirm missing files under the chosen root fail without another root search. Missing/malformed schema files permit engine construction but fail preparation; missing selection/context/root metadata fails at its configured boundary.
- Capture a prepared execution, then modify/remove its schema source. The handle executes and repairs against its captured plan; a later ordinary preparation observes the changed source. Count reads to protect capture behavior, not incidental compiler-internal call counts.
- Confirm equivalent forms submit the selected document to an offline recording client and validate generated results with those same constraints. Include a boolean-schema case. Preserve transport error behavior rather than testing assumptions about live provider support.
Completion: the mixed Scriptorium/Notarius layout and prepared-plan lifecycle are covered end to end without application-managed mounts or live infrastructure.
Stage 8 — Migrate Documentation And Validate The Assembled Sprint
- Update GoDoc and formats.md for selection/null/composition, source context, root scope, inactive modes, preparation failures, and removed implicit root. Update the consumer guide with runtime inline replacement and context-backed root/helper migration examples. Keep exact API declarations in GoDoc and configuration rules in the format reference; link rather than repeat them everywhere.
- Update sources.md, runner.md, and the internal overview where responsibilities changed. Adapt maintained examples/fixtures referencing removed schema options to inline, source-relative, or explicit context-backed roots as appropriate. Keep one representative maintained mixed-source example rather than adding an application asset assembly tutorial.
- Make the canonical migration documentation cover positional public struct literals, removed schema options/field, and the absence of an implicit root. Release preparation will select the version and summarize these changes in its release note under the release policy. This sprint selects no release number and does not rewrite published notes or require a publication check.
- Audit the complete diff against the feature roadmap. Remove old schema-source wiring, obsolete helper-only tests/defaults, and stale current-state references. Historical published notes and the original request are not current API owners. Do not remove unrelated path/artifact machinery or change dependency versions or the Go consumer minimum for this feature.
- Follow the development guide's newest locally installed Go selection with
GOTOOLCHAIN=localandGOWORK=off. Format changed Go files; rungo test -race ./...,go vet ./..., andgit diff --check. The complete suite includes maintained examples; no repeated example gate or benchmark run is required. Check changed local documentation links. Fix failures using focused reruns and validate the final assembled changes proportionately.
Completion: all stages' behavior and tests are implemented, documentation describes the new configuration/API migration, and assembled validation succeeds. Do not tag, push, publish dependencies, or require external consumer updates to finish this sprint. Retire completed roadmaps during normal release preparation.
Open Questions
None. The three product decisions are accepted in the feature roadmap; this plan selects the remaining API, helper migration, and preparation plumbing details.