promptkit/docs/internal/sources.md

Internal Sources And Validation

Purpose

This document describes Promptkit's implemented internal source, artifact, rendering, and output-validation behavior. The architecture policy owns the library boundary and dependency rules. None of these internal packages is a supported consumer API, and the root engine assembles them behind its public source options and values. The framework format reference owns the exact file fields, validation modes, built-in catalog, and source precedence.

Prompt Definitions

internal/promptdef uses one source-neutral flow for prompt selection and normalization. That flow scans normalized YAML ID and version metadata, requires one strictly decoded document per file, classifies errors for the selected definition, detects duplicates, and normalizes the exact match. Small operating-system and fs.FS adapters own discovery, byte reads, display paths, content opening, and root containment. Each lookup remains a point-in-time scan: definitions and catalogs are not cached, and file-backed message content is opened only for the exact selected candidate.

Operating-system sources enforce containment against canonical roots and targets so symlinks cannot escape. Injected fs.FS sources enforce containment in their clean relative path namespace. A single-file source uses the selected prompt file's containing directory as its root. Every content path must be relative and is opened from its exact parsed text after a separate blank check; contained parent components and whitespace-bearing names remain valid.

Exact prompt inspection performs one point-in-time lookup through that same repository and validates referenced message content before returning declared metadata. It does not parse templates or read profile, input, or schema sources, and it does not retain the definition for a later execution.

Its package tests own prompt selection, strict decoding, definition validation, duplicate detection, and source containment: prompt-definition repository tests.

Profiles And Built-Ins

internal/profile loads and validates execution profiles from an operating-system filesystem or an fs.FS. A file contains exactly one YAML document and its trimmed YAML id is its only selection identity; filenames do not confer authority. Each point lookup reads discovered files once for their metadata and reuses the selected file's bytes for strict decoding; unrelated profiles are not fully decoded. Strict selected decoding recognizes the optional backend field, trims its value, and requires a model plus at least one non-blank backend or endpoint. File-backed extra_params values are validated and defensively copied through the shared bounded JSON-value owner before a profile is published. OpenAI-compatible reserved-field policy remains with the model-client and backend-registry owners.

The overlay repository consults the next repository only when the higher-precedence repository reports that a profile is absent. A reliably selected malformed profile stops fallback, while an unrelated malformed file does not become authoritative through its filename. Loading does not check backend registry membership because the available registry belongs to the assembled engine; the runner checks membership during preparation and exact profile inspection.

The root engine assembles profile repositories in precedence order: in-memory profiles, one ordinary configured source, an application fallback source, then the embedded built-in catalog. An explicit file or fs.FS profile source replaces Config.ProfileDir within the ordinary configured-source category.

Exact profile inspection performs one point-in-time lookup through those profile sources and checks the resolved target without reading prompt, input, or schema sources. It does not retain that lookup for a later execution.

internal/profile/builtin embeds the maintained built-in profile catalog. Every embedded profile selects openrouter and inherits its endpoint and credential environment-variable name from the built-in backend registry rather than repeating those values. Profile loading and overlay behavior are owned by the profile repository tests, while catalog completeness, the backend-selection invariant, and duplicate IDs are owned by the built-in repository tests.

Ordinary Artifacts

internal/artifact accepts explicitly typed inline references even when their body is empty. It also resolves unrestricted, caller-selected paths only when they identify regular operating-system files, checking that condition before and after opening the file. It copies content into an artifact, records metadata and an opaque content-equality value, and applies a content-type fallback.

Regular files are read synchronously in bounded chunks. Cancellation is checked before opening, before and after every read, and before publishing the artifact, so a canceled read never publishes partial content. The ordinary reader does not detach file reads into background goroutines.

This ordinary reader does not implement an inbound HTTP security boundary. In particular, it does not constrain files to an application root or impose an HTTP request-size policy. Scriptorium's restricted HTTP reader remains an application concern outside Promptkit. The artifact reader tests own the implemented reader behavior and failures.

Rendering

internal/prompt renders definition messages as Go templates using named artifacts and variables. It carries message roles, session IDs, and cache control into the rendered prompt. The renderer tests own rendering behavior.

Schemas And Output Validation

internal/validate provides validators backed by an operating-system filesystem or an fs.FS. Invalid generated content is returned as a validation result; inability to load, register, or compile a schema is an operational error.

For executable preparation, the built-in validators create a frozen validation plan. None, basic, and JSON modes retain the effective output contract without source access. JSON Schema mode loads the root document, resolves and compiles every transitive reference during preparation, and retains the compiled validator. The provider-facing structured-output metadata uses that same captured root document.

PrepareExecution also completes prompt and profile selection, artifact loading and hashing, session and message rendering, and target resolution. RunPrepared uses the retained source-derived state and validation plan; it does not reopen prompt, profile, input, or schema sources and does not rerender the request. By contrast, ordinary Prepare produces a preparation value only: a later Run performs its own source resolution and preparation.

The validator tests own basic, JSON, JSON Schema, source resolution, schema loading, compilation, frozen-reference behavior, and content-failure behavior. Prepared execution orchestration is owned by the use-case tests.