Retire completed schema planning documents
This commit is contained in:
@@ -1,414 +0,0 @@
|
||||
# Prompt-Owned Schemas Implementation Plan
|
||||
|
||||
## Objective And Authority
|
||||
|
||||
Implement [prompt-owned-schemas.md](prompt-owned-schemas.md). That feature
|
||||
roadmap owns the accepted selection, composition, scope, ownership, and migration
|
||||
decisions. The [consumer request](prompt-owned-schemas-request.md) is background,
|
||||
not authority for application-specific behavior. Respect
|
||||
[ADR-0001](../adr/0001-configuration-boundary.md): interpret only ConfigKit's final
|
||||
effective configuration and its effective resource context.
|
||||
|
||||
The baseline is commit `adfccb5`, with ConfigKit v1.3.0 and the prepared
|
||||
message-content hash. This is one end-to-end sprint, divided into sequential
|
||||
single-prompt stages for a Luna coding agent. Intermediate stages may fail to
|
||||
compile. Do not add compatibility shims or duplicate implementations merely to
|
||||
make an intermediate commit coherent. No stage requires human approval, external
|
||||
publication, or a dependency upgrade. Commit completed units under repository
|
||||
policy; commits are not review gates.
|
||||
|
||||
Start with [development.md](../development.md) and the applicable
|
||||
[agent](../policy/agents.md), [architecture](../policy/architecture.md),
|
||||
[testing](../policy/testing.md), and [documentation](../policy/documentation.md)
|
||||
policies. Read the [format reference](../formats.md),
|
||||
[source/validation internals](../internal/sources.md), and existing GoDoc before
|
||||
changing their contracts. Reuse existing file-path, schema compilation,
|
||||
cancellation, and ownership machinery where applicable.
|
||||
|
||||
## Concrete Design For This Sprint
|
||||
|
||||
### Public Values And Configuration
|
||||
|
||||
- Add `Schema any` with `json:"schema,omitempty"` to public `OutputContract`.
|
||||
Nil means no inline selection; a nonnil empty object and `false` are selections.
|
||||
Preserve `SchemaPath string` and its existing JSON field. Do not add runtime
|
||||
`SchemaFile`, filesystem handles, or a standalone schema-loading API.
|
||||
- Add `Schema any` and `SchemaFile *string` to `PromptOutputConfig`, retaining
|
||||
`SchemaPath *string`. `PromptSource` emits `schema` only for a nonnull inline
|
||||
value and emits each nonnil reference pointer. As with existing typed helpers,
|
||||
raw ConfigKit mappings express explicit null and composition controls.
|
||||
- Add `Schema any` and `SchemaFile *string` to `PromptOutputInspection`, with
|
||||
`schema` and `schema_file` JSON/YAML names and `omitempty`. Preserve its existing
|
||||
path presence. Expose configured values/references, not filesystem capabilities
|
||||
or file contents. Generic configuration inspection already exposes declared
|
||||
application and collection roots; no additional public root-inspection API is
|
||||
needed.
|
||||
- Decode optional `schema_root` strings at the application root and collection
|
||||
entries. Absent/null collection roots select the application root; absent/null
|
||||
application roots mean no root. A nonnull blank root is invalid. An explicitly
|
||||
supplied `"."` is valid when accompanied by usable source context.
|
||||
- In JSON Schema mode, exactly one nonnull effective `schema`, `schema_file`, or
|
||||
`schema_path` is required. Reject blank file/path selections and conflicts.
|
||||
Outside that mode, do not resolve, compile, or impose new selection checks on
|
||||
unused schema fields. Retain ordinary ConfigKit decoding semantics.
|
||||
|
||||
Use native ConfigKit values for captured configuration. For active runtime
|
||||
inline values, use standard JSON encoding and the existing schema JSON decoder
|
||||
to obtain an operation-owned document; do not introduce a custom numeric bridge
|
||||
or the bounded provider-extra copier. JSON null, including typed nil values
|
||||
encoded as null, means no selection. Empty objects and booleans must survive.
|
||||
Unsupported/cyclic Go values that cannot be encoded fail as invalid request
|
||||
values, not panics. JSON-compatible but invalid schema documents fail compilation
|
||||
at preparation through the existing validation-error boundary. Preserve existing
|
||||
file JSON decoding and its numeric behavior; promise no new cross-input numeric
|
||||
equivalence. Clone captured, already-supported trees using the existing trusted
|
||||
document ownership path, without provider-extra depth/node limits.
|
||||
|
||||
### Schema-Root Helper Migration
|
||||
|
||||
Replace the out-of-band root settings with one mapping helper:
|
||||
|
||||
```go
|
||||
func SchemaRootSource(label, root string) configkit.Source
|
||||
```
|
||||
|
||||
It returns `configkit.Mapping(label, map[string]any{"schema_root": root})`,
|
||||
without opening files or validating raw contributions. Callers place its input
|
||||
at their chosen native ConfigKit layer and provide a filesystem context, just as
|
||||
they do for `PromptSource`. For example, for an `fs.FS`:
|
||||
|
||||
```go
|
||||
filesystem := configkit.NewFilesystem("application", schemaFS)
|
||||
rootInput := configkit.Input{
|
||||
Source: promptkit.SchemaRootSource("application schemas", "schemas"),
|
||||
Filesystem: &configkit.FilesystemContext{
|
||||
FS: &filesystem, Root: ".", BaseDir: ".",
|
||||
},
|
||||
}
|
||||
// Supply rootInput in configkit.Underlay{Inputs: []configkit.Input{rootInput}},
|
||||
// or use &rootInput as Main when no other main input is needed.
|
||||
```
|
||||
|
||||
Remove `Config.SchemaDir`, `WithSchemaFS`, and `WithSchemaFile`, and the internal
|
||||
option setters/flags used solely by those APIs. This is an intentional pre-1.0
|
||||
API migration, not a deprecation period. Do not silently translate options into
|
||||
a highest-priority layer or preserve last-option-wins behavior. Callers needing
|
||||
collection-specific roots use ordinary collection mappings with context; the
|
||||
helper need not grow collection-specific parameters.
|
||||
|
||||
Migration: a directory or filesystem becomes a context-backed root mapping;
|
||||
a former single-file source becomes a `schema_file` on a configured prompt or a
|
||||
root mapping plus its basename in `schema_path`. For an unrelated per-request
|
||||
file, the caller reads/decodes it and supplies `Schema` inline. Host-directory
|
||||
examples use an explicit `os.DirFS`/ConfigKit filesystem context. Do not infer a
|
||||
working-directory root. This preserves the requested directory/filesystem
|
||||
capabilities without preserving a second source-selection system.
|
||||
|
||||
### Internal Resource And Preparation Model
|
||||
|
||||
Separate serializable selections from resource capabilities:
|
||||
|
||||
- Extend domain output selections with inline `Schema` and the configured
|
||||
`SchemaFile` reference string. The latter is internal, not a new request field.
|
||||
- Capture an optional applicable root and optional source-relative file binding
|
||||
with the prompt definition. Use small source-neutral domain values containing
|
||||
an `fs.FS` and resolved filesystem path. Exclude those capabilities from JSON
|
||||
and definition hashing. Keep the root separate from the output contract so it
|
||||
remains available to a runtime `SchemaPath` replacement.
|
||||
- Configuration binding resolves a field's effective filesystem/base directory
|
||||
using the current effective-origin lookup and `SourceContext`, as `readContent`
|
||||
does. Extract binding from reading so schema capture does not call `Open`,
|
||||
`Stat`, or compile a schema to establish file existence. Apply existing
|
||||
`filecatalog` path containment rules. Missing context/invalid reference metadata
|
||||
is a configuration error; missing or malformed file data is a preparation error.
|
||||
- Add internal `validate.Input` containing `Contract domain.OutputContract` and
|
||||
optional `File` binding. Change `ValidationPreparer.PrepareValidation` to accept
|
||||
this input. Non-schema modes need no file. JSON Schema mode selects the inline
|
||||
document or the one bound file and uses one common compilation pipeline.
|
||||
- After complete request-contract replacement, the runner selects the binding:
|
||||
configured `schema_file` uses its captured binding only without a replacement;
|
||||
`SchemaPath` binds under the selected prompt's applicable root; inline values
|
||||
need no file. A replacement never borrows the configured inline/file selection.
|
||||
A missing root for a runtime path is an invalid request. Existing public error
|
||||
translation remains authoritative.
|
||||
|
||||
Internal names may follow local conventions, but implement these responsibilities
|
||||
once. Do not carry ConfigKit objects into execution, reread configuration, query
|
||||
history during execution, or serialize filesystem objects. The selected root is
|
||||
not a list to search. Root selection happens before opening the requested file.
|
||||
|
||||
## Stage 1 — Add Schema Value Shapes And Ownership
|
||||
|
||||
Status: complete.
|
||||
|
||||
Read `types.go`, `prompts.go`, `resource_inspection.go`, `convert.go`, domain
|
||||
output contracts, and the existing schema-document ownership/decoder helpers.
|
||||
|
||||
1. Add the public/configuration/inspection fields selected above, corresponding
|
||||
internal fields, and the small resource-binding and validation-input values.
|
||||
Keep selection data separate from the capabilities used to load it.
|
||||
2. Extend `PromptSource` and output conversions. Preserve false and empty-object
|
||||
presence; do not use truthiness or map length to decide selection.
|
||||
3. Establish one shared output-selection check for the effective active mode.
|
||||
Separate selection validity from schema compilation. Preserve format and
|
||||
repair-attempt rules. For typed nil runtime inline values, use the standard
|
||||
JSON null result before counting selections.
|
||||
4. Add operation-owned inline decoding using the existing JSON document path.
|
||||
Reuse trusted cloning after capture/decoding. Avoid an encode/decode round trip
|
||||
at each getter or conversion; ordinary detached getters clone supported trees.
|
||||
5. Update shallow output-contract/definition copies that now contain mutable
|
||||
schema values. Do not change schema ownership by accidentally routing these
|
||||
values through the provider-extra copier.
|
||||
|
||||
Tests: extend focused domain selection and typed-source tests for nil/null,
|
||||
`false`, `true`, `{}`, conflicting selections, blank references, and inactive
|
||||
mode behavior. Add a focused runtime inline encoding failure case. Do not run
|
||||
the full suite while downstream signatures are intentionally incomplete.
|
||||
|
||||
Completion: representations and ownership rules are implemented, with no public
|
||||
runtime filesystem API or truthiness-based loss of schemas.
|
||||
|
||||
## Stage 2 — Capture Roots Through Native Configuration Inputs
|
||||
|
||||
Status: complete.
|
||||
|
||||
Work in `internal/configuration/load.go`, its normalization/capture helpers,
|
||||
the new root source helper, `engine.go`, and internal engine construction.
|
||||
|
||||
1. Add optional roots to decoded application/collection records. Validate only
|
||||
final effective root values and obtain each root's effective source context.
|
||||
Build application and collection root bindings once during construction,
|
||||
using the existing shared metadata snapshot rather than scanning the full
|
||||
resolved tree per prompt.
|
||||
2. Capture the applicable root with each prompt under the accepted collection
|
||||
scope rule. A declared collection root wins even if a subsequently selected
|
||||
file is missing there. Null clearing restores application-root selection.
|
||||
3. Implement `SchemaRootSource` and remove the legacy public and internal
|
||||
schema-source configuration paths described above, including the implicit
|
||||
default. Remove now-unused schema-specific defaults/imports, not unrelated
|
||||
artifact-source machinery.
|
||||
4. Wire the engine toward a source-neutral validator receiving operation inputs,
|
||||
rather than constructing a validator tied to one engine-wide root. Update
|
||||
affected configuration/engine tests and fixtures enough to express the new
|
||||
root inputs; later stages finish preparation integration.
|
||||
|
||||
Tests: root/collection capture with an explicit mapping context, root override
|
||||
context from another source, null fallback, blank/missing context diagnostics,
|
||||
and native layer precedence for the mapping helper. Use a counting filesystem
|
||||
to demonstrate construction does not inspect schema file existence.
|
||||
|
||||
Completion: root selection and context come solely from the final ConfigKit
|
||||
configuration. No option-specific precedence or implicit root remains.
|
||||
|
||||
## Stage 3 — Capture Prompt-Owned Schema Selections And References
|
||||
|
||||
Status: complete.
|
||||
|
||||
Work in `internal/configuration/prompts.go`, resource projections, normalization,
|
||||
and prompt-definition snapshots/hashing.
|
||||
|
||||
1. Extend output capture for all three fields. Validate exactly one effective
|
||||
active selection, using ConfigKit's existing presence/null data when needed.
|
||||
Do not reject overwritten contributions or reconstruct native composition.
|
||||
2. Extract a reusable effective-field resource binder from `readContent`.
|
||||
Preserve message-file read/close behavior; schema-file capture uses binding
|
||||
only. Inherited references use their contributing field's context, not the
|
||||
child's source. Root and file binding use existing path/context rules.
|
||||
3. For configured active `schema_path`, require an applicable root and validate
|
||||
path metadata without opening the file. Bind active `schema_file` from that
|
||||
field's context. Inline selections need neither binding nor root.
|
||||
4. Extend `PromptOutputResource`, `InspectPrompt`, and public projections to
|
||||
include selected values/references without opening files. Keep detached
|
||||
schema trees in public results and preserve existing inspection presence rules.
|
||||
5. Include configured inline values and file-reference strings in the existing
|
||||
definition-hash input; exclude resource capabilities. Use the existing hash
|
||||
algorithm, not a new digest API or file-content read. Root context and schema
|
||||
file bytes are not newly promised hash inputs. Leave both prepared-message
|
||||
hash algorithms and their semantics unchanged.
|
||||
|
||||
Tests: imports/inheritance and overrides preserve effective field context;
|
||||
composition can merge inline mappings and explicitly clear alternatives;
|
||||
same basenames in different filesystems stay distinct; inactive schema fields
|
||||
do not trigger resource binding. Inspect a prompt with a missing schema file
|
||||
successfully, then leave actual file failure for preparation.
|
||||
|
||||
Completion: final selections, source bindings, inspection, and definition
|
||||
snapshots agree without eager schema I/O or a second composition pass.
|
||||
|
||||
## Stage 4 — Unify Inline And File Schema Preparation
|
||||
|
||||
Status: complete.
|
||||
|
||||
Work in `internal/validate/validator.go` and the current validator implementation.
|
||||
|
||||
1. Adapt the preparer to `validate.Input`. Replace duplicated engine-root-specific
|
||||
preparation routes with a source-neutral validator. Migrate internal validator
|
||||
tests to bound files/inline documents; retain useful path/decoder tests at
|
||||
their owning helper rather than keeping obsolete constructors just for tests.
|
||||
2. Load a selected file once through the bound filesystem and existing
|
||||
context-checked chunk reader. Preserve read/close error handling and the
|
||||
single-complete-JSON-document decoder. Inline values use their captured or
|
||||
operation-owned document without a file read.
|
||||
3. Feed both routes through the existing dialect, self-contained-resource,
|
||||
schema-location, and root compilation checks. Retain supported IDs, anchors,
|
||||
internal references, filename independence, and external-resource rejection.
|
||||
Keep existing canonical escaped retrieval URLs for file sources; use a stable
|
||||
private absolute retrieval URL for inline documents, without new public IDs.
|
||||
4. Store the selected document and compiled root in the existing prepared plan.
|
||||
`SchemaDocument()` returning `false` is a document, not missing metadata.
|
||||
Use the same value for local checks and provider constraints.
|
||||
5. Preserve synchronous cancellation checks around reads, decoding, compilation,
|
||||
and validation. No background dependency workers or new cache is needed.
|
||||
|
||||
Tests: object and boolean schemas work by both routes; equivalent documents
|
||||
give equivalent validation; invalid schema documents/dialects, external refs,
|
||||
valid internal refs, exact file JSON numbers, and cancellation retain their
|
||||
existing classifications. Reuse focused tests rather than duplicating the entire
|
||||
schema suite for every route.
|
||||
|
||||
Completion: one source-neutral preparation/compilation path produces the existing
|
||||
operation-local plan with no extra resource reads for `$ref` resolution.
|
||||
|
||||
## Stage 5 — Bind Effective Contracts In Every Execution Path
|
||||
|
||||
Status: complete.
|
||||
|
||||
Work in `internal/usecase/runner.go`, prepared execution, engine wiring, and
|
||||
`smoke_inference_case.go`.
|
||||
|
||||
1. Resolve the complete effective output contract first, then its file binding.
|
||||
Retain surrounding root scope for runtime `SchemaPath`; discard configured
|
||||
`schema_file` binding and configured inline value on a request replacement.
|
||||
Store the selected input in preparation state and pass it to the preparer.
|
||||
2. Keep malformed selection/root requirements within existing invalid-config
|
||||
construction or invalid-request runtime boundaries. Read/decode/compile
|
||||
failures remain operational validation errors before generation. Include
|
||||
selected prompt/reference context while preserving error identities and
|
||||
cancellation precedence.
|
||||
3. Keep `Prepare`, `ValidateRequest`, `Run`, and `PrepareExecution` on the shared
|
||||
preparation path. `RunPrepared` retains/reuses the plan and needs no schema
|
||||
binding, file read, or compilation at execution time. Preserve callback,
|
||||
operation-budget, admission, and continuation behavior.
|
||||
4. Preserve prompt-ID-based structured-output naming and the existing provider
|
||||
submission path. Repair reuses the same plan/document; it does not reselect
|
||||
schema forms or roots. Do not invent provider-support assertions.
|
||||
5. Migrate private smoke-schema setup to an explicit bound private resource and
|
||||
the same preparer, independent of application schema roots. Update internal
|
||||
fake preparers to the new signature; preserve the intentional nil-validator
|
||||
behavior of focused use-case tests.
|
||||
|
||||
Tests: runtime inline/path replacement, no borrowed configured selection, missing
|
||||
runtime root, and collection-root selection during replacement. Existing
|
||||
ordinary/prepared, validation-only, continuation, and smoke tests should compile
|
||||
and exercise the common pipeline. Verify preparation failures do not call a model.
|
||||
|
||||
Completion: all execution entry points select and retain exactly the intended
|
||||
schema plan, without an out-of-band engine schema root.
|
||||
|
||||
## Stage 6 — Verify Public Projections And Ownership
|
||||
|
||||
Status: complete.
|
||||
|
||||
Finish public facade conversion and serialization changes in `convert.go`,
|
||||
`types.go`, `resource_inspection.go`, `json.go`, and prepared notification paths.
|
||||
|
||||
1. Extend existing JSON/YAML contract tests for inline `false`, `{}`, absence,
|
||||
inspection references, and backward decoding of values omitting new fields.
|
||||
The prepared JSON projection already inherits public struct fields; avoid a
|
||||
second serialization DTO. Preserve ordinary manual-value JSON behavior rather
|
||||
than introducing schema compilation on JSON decode.
|
||||
2. Test that mutating typed input after its source snapshot, inspection results,
|
||||
`Prepare` results, `Details`, `OnPrepared`, or injected-client schema metadata
|
||||
cannot alter captured configuration or retained execution. Extend existing
|
||||
ownership fixtures rather than duplicating every handle lifecycle test.
|
||||
3. Keep configured file-reference inspection separate from the schema document
|
||||
shown in prepared `StructuredOutput`. Public runtime `OutputContract` has no
|
||||
`SchemaFile`; do not fabricate an inline selection or path merely to fill that
|
||||
projection. The prepared structured-output document is the selected content.
|
||||
`ValidationResult.SchemaPath` keeps its explicit-root path meaning and is
|
||||
empty for the new inline/source-relative forms; no synthetic sentinel path or
|
||||
new result field is needed.
|
||||
4. Check every remaining shallow output-contract copy now containing `Schema`.
|
||||
Trusted internal sharing stays immutable; public copies detach mutable trees.
|
||||
Do not marshal filesystem bindings, or recompile schemas in inspection/getters.
|
||||
|
||||
Tests: focused typed-source, resource-inspection, prepared/notification, and JSON
|
||||
contract packages. Retain schema-document tests that exceed provider-extra copy
|
||||
limits; new inline handling must not route trusted documents through those limits.
|
||||
|
||||
Completion: all exposed values are faithful and caller-owned where the existing
|
||||
API requires it, with no resource capability leakage or extra schema work.
|
||||
|
||||
## Stage 7 — Verify Mixed Collections And Captured Lifecycles
|
||||
|
||||
Status: complete.
|
||||
|
||||
Add focused external-package integration coverage using real ConfigKit
|
||||
composition and injected provider/transport boundaries. Prefer a small fixture
|
||||
with distinct risks over a Cartesian product of settings.
|
||||
|
||||
1. Build one engine with independent collection roots, application-root fallback,
|
||||
source-relative schemas, and inline schemas together. Mix embedded-like and
|
||||
operator filesystem contexts, with identical schema basenames. Replace one
|
||||
prompt/schema selection while unrelated prompts remain usable.
|
||||
2. Cover inherited source context and overridden context through actual imports
|
||||
or layers, including a root inherited from a different source than its prompt.
|
||||
Exercise explicit null when switching forms and ConfigKit's native whole-value
|
||||
replacement control. No custom overlay behavior belongs in expected results.
|
||||
3. Confirm missing files under the chosen root fail without another root search.
|
||||
Missing/malformed schema files permit engine construction but fail preparation;
|
||||
missing selection/context/root metadata fails at its configured boundary.
|
||||
4. Capture a prepared execution, then modify/remove its schema source. The handle
|
||||
executes and repairs against its captured plan; a later ordinary preparation
|
||||
observes the changed source. Count reads to protect capture behavior, not
|
||||
incidental compiler-internal call counts.
|
||||
5. Confirm equivalent forms submit the selected document to an offline recording
|
||||
client and validate generated results with those same constraints. Include a
|
||||
boolean-schema case. Preserve transport error behavior rather than testing
|
||||
assumptions about live provider support.
|
||||
|
||||
Completion: the mixed Scriptorium/Notarius layout and prepared-plan lifecycle are
|
||||
covered end to end without application-managed mounts or live infrastructure.
|
||||
|
||||
## Stage 8 — Migrate Documentation And Validate The Assembled Sprint
|
||||
|
||||
Status: complete.
|
||||
|
||||
1. Update GoDoc and [formats.md](../formats.md) for selection/null/composition,
|
||||
source context, root scope, inactive modes, preparation failures, and removed
|
||||
implicit root. Update the [consumer guide](../consumers/pkg-promptkit.md) with
|
||||
runtime inline replacement and context-backed root/helper migration examples.
|
||||
Keep exact API declarations in GoDoc and configuration rules in the format
|
||||
reference; link rather than repeat them everywhere.
|
||||
2. Update [sources.md](../internal/sources.md),
|
||||
[runner.md](../internal/runner.md), and the internal overview where responsibilities
|
||||
changed. Adapt maintained examples/fixtures referencing removed schema options
|
||||
to inline, source-relative, or explicit context-backed roots as appropriate.
|
||||
Keep one representative maintained mixed-source example rather than adding an
|
||||
application asset assembly tutorial.
|
||||
3. Make the canonical migration documentation cover positional public struct
|
||||
literals, removed schema options/field, and the absence of an implicit root.
|
||||
Release preparation will select the version and summarize these changes in
|
||||
its release note under the release policy. This sprint selects no release
|
||||
number and does not rewrite published notes or require a publication check.
|
||||
4. Audit the complete diff against the feature roadmap. Remove old schema-source
|
||||
wiring, obsolete helper-only tests/defaults, and stale current-state references.
|
||||
Historical published notes and the original request are not current API owners.
|
||||
Do not remove unrelated path/artifact machinery or change dependency versions
|
||||
or the Go consumer minimum for this feature.
|
||||
5. Follow the development guide's newest locally installed Go selection with
|
||||
`GOTOOLCHAIN=local` and `GOWORK=off`. Format changed Go files; run
|
||||
`go test -race ./...`, `go vet ./...`, and `git diff --check`. The complete suite
|
||||
includes maintained examples; no repeated example gate or benchmark run is
|
||||
required. Check changed local documentation links. Fix failures using focused
|
||||
reruns and validate the final assembled changes proportionately.
|
||||
|
||||
Completion: all stages' behavior and tests are implemented, documentation describes
|
||||
the new configuration/API migration, and assembled validation succeeds. Do not tag,
|
||||
push, publish dependencies, or require external consumer updates to finish this
|
||||
sprint. Retire completed roadmaps during normal release preparation.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None. The three product decisions are accepted in the feature roadmap; this plan
|
||||
selects the remaining API, helper migration, and preparation plumbing details.
|
||||
Reference in New Issue
Block a user