Internal Sources And Validation

Purpose

This document describes Promptkit's implemented configuration, artifact, rendering, and output-validation behavior. The architecture policy owns dependency rules, while the framework format reference owns fields, defaults, and source semantics. Internal packages are not consumer APIs.

Typed Configuration Loading

internal/configuration is the single adapter from ConfigKit's final effective configuration to execution-owned definitions. It performs one layered load, captures effective presence (including explicit null), records normalization and defaults through resolution, and validates all effective root and collection backend, profile, and prompt values. Replaced lower-layer domain values are not separately validated. It indexes root resources under their unqualified IDs and collection resources under their full namespace/local-key IDs while retaining ConfigKit's effective tree in its authored shape. It builds an immutable backend index and presence-aware profile targets, applying final backend defaults only after profile inheritance. Runtime lookups use those final indexes and exact stored references; they do not rewrite or resolve resources from ConfigKit source history. The runtime/domain packages do not depend on ConfigKit.

ConfigKit owns source headers, imports, composition, and source evidence. Promptkit does not preparse contributions: it validates the final effective values once, captures the indexed identities and their original resource contexts, and retains ConfigKit's result for explicit operator inspection. Accepted non-main source headers are consumed during composition; the main source declaration remains available in that inspection view. Prompt and collection release labels are descriptive metadata, not selection inputs. The format reference owns the exact source-header and collection contracts.

The ConfigurationInspection.Diff facade method compares two retained results through a private wrapper and a narrow internal/configuration.Diff bridge. The wrapper keeps Result[document] inside the adapter; the bridge delegates once to ConfigKit v0.8.0 and returns its public difference records. Comparison uses only frozen effective values and does not reopen sources, resolve runtime credentials, or enter the execution path. ConfigKit applies sensitivity from both results; reports may still contain ordinary application data. See the consumer comparison guide and ConfigKit's v0.8.0 comparison contract.

After resource materialization and effective-profile snapshot construction, the adapter captures lexically ordered identity inventories for backends, profiles, and prompts, per-kind resource namespace sets, and metadata for effective collection declarations. Listing methods return caller-owned copies from those captured values without consulting ConfigKit or copying resource definitions.

The root DirectoryInputs and FilesystemInputs helpers discover sorted YAML files under an explicit root, provide resource context, and leave imports disabled. Each returned slice belongs to one strict-composition group; file order does not confer priority. Caller underlays are composed in their given order, followed by the optional main input. The configuration tests exercise the real loader and final-value handoff.

The public CollectionInput helper mounts one caller-selected source at collections/<namespace> and enables ConfigKit imports for that input. The helper does not read the source; ConfigKit retains responsibility for loading, composition, imports, and source-local resource context. Imported or inherited content_file values continue resolving relative to the source that contributed the effective field, not to a flattened runtime identity. Explicit InspectConfiguration views expose the retained effective configuration and provenance for operator review; runtime selection and inventories do not derive from source history.

Prompt Definitions

The configuration adapter indexes each prompt by exact ID, materializes file-backed message content using the exact effective field's origin and resource context, and checks static message/session template syntax at construction. It stores captured content, compiled templates, declared input metadata, and a definition hash. Alongside the unchanged hashed definition, it retains content-type and description presence for each declared input and provides an internal requirement snapshot with those declarations and the captured parses. Metadata values come from the final normalized declaration; absent and null fields remain distinguishable from explicit strings, including empty strings. It closes content-file handles before returning and retains no source capability for public inspection.

Prompt materialization shares a lazy metadata snapshot across root and collection resources during construction, avoiding a full metadata copy per collection.

Role and cache-control normalization is applied to execution-owned message copies. ConfigKit v0.8.0 retains effective input message metadata and its resource origins; recording a sequence-wide normalization in the retained configuration would replace origins of unchanged content-file fields.

The runner selects full definitions from this index for preparation and uses captured prompt metadata for InspectPrompt and InspectRequirements; the internal requirement snapshot lookup uses the selected prompt ID alone. These paths do not rescan configuration or reopen message files. InspectPrompt returns declared metadata without rendering or resolving profile, input, or schema resources, and its public shape and prompt hashes do not include the captured presence state. Runtime rendering still checks request variables and input helpers. The prompt adapter tests cover resource origins, context containment, capture, and selection.

The configuration adapter shares ID-only selection across its internal prompt lookups. The execution snapshot lookup returns an owned prompt definition and cached hash; metadata inspection returns owned input metadata without copying execution templates. Profile selection likewise has an identity-only lookup for consumers that do not need the execution target, while the full profile lookup continues to return an owned target and deep copy of provider extras. These lookups use captured indexes and do not query ConfigKit again.

The internal/prompt requirement walker uses the captured parse trees and declaration metadata without executing templates or reading artifacts. It collects literal input names, direct root caller keys, and literal-key index lookups. For if and with, it classifies branches when their conditions are simple supported values: direct values, literals, negation, or two-string equality/inequality. It uses Go template truthiness, so a supplied nonempty string such as false is true. Missing decision values and content-dependent conditions leave dependency-bearing branches conditional and report incomplete discovery; unresolved conditions with no branch dependencies do not by themselves make discovery incomplete. Recognizable references on known skipped paths remain optional inventory entries with an inactive-use explanation, unless another applicable use establishes a stronger classification. Skipped expressions are scanned only for syntactically recognizable names and are not evaluated. A direct caller value or literal-key lookup can also supply an input name, including through simple parentheses or a direct pipeline to input; names are used exactly as supplied. Unresolved names and unsupported dependency-bearing expressions produce structured limitations while retaining recognizable references. Where an unresolved name or scope could affect existing optional entries, those entries are reported as conditional rather than inactive-only or definitively optional. Direct root-map with forms retain caller-map identity. Known scalar values remain separate from unknown scopes; accesses through untracked local maps report incomplete discovery rather than silently losing caller keys. Unsupported name expressions reuse value analysis's conservative syntax scan instead of recursively rescanning the same subtree. Invoked named templates are scanned conservatively, and dependency-bearing range scopes make discovery incomplete. Uncalled definitions and bypassed session templates are not traversed. This internal walker does not change rendering behavior.

Profiles

ConfigKit resolves profile inheritance before the handoff. Within a collection, extends names a sibling local key; it is not a cross-collection reference. The configuration adapter validates final effective resource references and field values, then builds target snapshots with backend defaults and presence/suppression state. A profile can remain partial until runtime overrides complete it. InspectProfile selects that captured target without prompt, artifact, schema, credential lookup, or provider work. Preparation applies runtime overrides and then checks target completeness. Identity-only profile selection can identify a partial captured profile without copying its target or requiring readiness. The target tests cover effective target interpretation.

The internal engine delegates listing through a narrow identity-inventory interface directly to the configuration snapshot. The execution runner has no listing responsibility.

Ordinary Artifacts

internal/artifact accepts explicitly typed inline references even when their body is empty. It also resolves unrestricted, caller-selected paths only when they identify regular operating-system files, checking that condition before and after opening the file. It copies content into an artifact, records metadata and an opaque content-equality value, and applies a content-type fallback.

Regular files are read synchronously in bounded chunks. The opened file's size is only an initial-capacity hint, capped at 1 MiB; reads continue to EOF, and the returned body determines artifact size and hash. If an overreported hint leaves unused capacity greater than the actual body length, the completed body is copied to release the excess backing storage, including when the body is empty. Cancellation is checked before opening, before and after every read, and before publishing the artifact, so a canceled read never publishes partial content. The ordinary reader does not detach file reads into background goroutines.

This ordinary reader does not implement an inbound HTTP security boundary. In particular, it does not constrain files to an application root or impose an HTTP request-size policy. Scriptorium's restricted HTTP reader remains an application concern outside Promptkit. The artifact reader tests own the implemented reader behavior and failures.

Rendering

internal/prompt renders captured, pre-parsed definition templates using named artifacts and variables. It clones templates to bind request-specific input helpers without mutating shared state. Within one render, each referenced artifact body is converted to text lazily and cached by input name for reuse across the session and every message; the cache is not shared across renders. Conversion uses bounded chunks and preserves the artifact bytes exactly.

Template parsing occurs during engine construction. Runtime execution remains synchronous. The renderer checks cancellation around execution boundaries, between artifact conversion chunks, around each message, and before publishing the complete prompt. It cannot interrupt template work already in progress and never publishes a partial prompt after observing cancellation. It validates and canonicalizes message roles before carrying roles, session IDs, and cache control into the rendered prompt. Message and session templates execute into per-call string builders, so their rendered strings do not require a final buffer-to-string copy. The renderer tests own rendering behavior.

Schemas And Output Validation

internal/validate provides validators backed by an operating-system filesystem or an fs.FS. Invalid generated content is returned as a validation result; inability to load, register, or compile a schema is an operational error. Basic validation checks body emptiness on the byte slice directly instead of first making a response-sized string copy.

Every preparation operation creates one operation-local validation plan. None, basic, and JSON modes retain the effective output contract without source access. JSON Schema mode loads the root document once, resolves and compiles each transitive reference, and retains the compiled validator. Schema compiler resources use canonical escaped file or private-scheme URLs; loaders decode their paths once and enforce the configured source boundary. The provider-facing structured-output metadata uses the root document captured by the same plan.

Schema preparation and execution remain synchronous. Promptkit checks cancellation before and after source resolution, JSON decoding, compilation, and validation, and between bounded schema-read chunks. Once cancellation is observed it returns the context error without publishing a partial plan or validation result, even when a compiler or validator has just returned a different error or a successful result. An fs.FS method or JSON Schema dependency call already in progress cannot be preempted; Promptkit waits for that call to return and then gives cancellation precedence. Validation does not detach dependency work into background goroutines.

Prepare discards its validation plan after returning metadata. Run retains its plan for initial and repaired-output validation, then discards it with the operation. PrepareExecution retains the plan in its private frozen payload; RunPrepared uses that plan without reopening configuration, message, input, or schema resources or rerendering the request. A later ordinary Run uses the same captured configuration but prepares request artifacts and schemas afresh.

The validator tests own basic, JSON, JSON Schema, source resolution, schema loading, compilation, frozen-reference behavior, content-failure behavior, and the synchronous cancellation boundary. Prepared execution orchestration is owned by the use-case tests.

15 KiB Raw Blame History