promptkit/docs/roadmap/prompt-inspection.md

Prompt-Definition Inspection

Status: Accepted.

Purpose

Allow consumers to look up one prompt definition by ID and optional version and inspect its declared interface without supplying placeholder inputs, resolving a profile, rendering templates, loading schemas, or invoking a model.

This provides a focused configuration-validation boundary for Weatherreporter, whose need is recorded in its Promptkit wishlist, while preserving Promptkit's deferred, request-oriented source model.

Motivation

Weatherreporter has a fixed application-owned registry of report definitions. It needs to verify that every configured prompt exists and declares the expected inputs and output workflow before collecting weather data or starting provider work.

The current workaround is to construct synthetic artifacts and variables and call Engine.Prepare. That validates much more than the application needs: prompt inputs are loaded, templates and session IDs are rendered, a profile and backend are resolved, credentials are checked, and JSON Schemas may be loaded and compiled.

Promptkit already owns prompt source selection, exact ID and version lookup, strict definition decoding, referenced content-file resolution, and prompt hashing. It should expose that cohesive subset directly rather than requiring consumers to reproduce its rules or maintain placeholder execution fixtures.

Consumer Workflow

The target public workflow is:

inspection, err := engine.InspectPrompt(ctx, promptID, promptVersion)
if err != nil {
	// Reject or report the configured prompt.
	return
}

for _, input := range inspection.Inputs {
	// Compare the declared prompt interface with application configuration.
}

The target public surface is:

type PromptInputDefinition struct {
	Name        string
	Required    bool
	ContentType string
	Description string
}

type PromptInspection struct {
	PromptID         string
	PromptVersion    string
	PromptHash       string
	DefaultProfileID string
	Inputs           []PromptInputDefinition
	OutputContract   OutputContract
}

func (e *Engine) InspectPrompt(
	ctx context.Context,
	promptID string,
	promptVersion string,
) (*PromptInspection, error)

The exported declarations and GoDoc will own the exact implemented contract. The important public shape is exact lookup through an existing engine, one caller-owned inspection value, declared input metadata, the declared output contract, and the same opaque prompt equality value used by preparation.

PromptInspection and PromptInputDefinition do not need stable JSON representations. Consumers that persist application configuration or diagnostics can define their own format and select the fields they require. The existing stable representations of OutputContract, OutputFormat, and ValidationMode remain unchanged.

Lookup And Source Selection

InspectPrompt requires a non-blank prompt ID and performs the same case-sensitive exact selection used by ordinary preparation. Inspection does not trim or otherwise canonicalize a non-blank ID or version independently of that execution path.

When promptVersion is non-empty, the configured prompt source must contain exactly one matching ID and version pair. When it is empty, the source must contain exactly one definition with the requested ID; multiple matching versions remain an ambiguous source-selection error.

Lookup uses the engine's normally selected prompt source:

• the last WithPromptFS or WithPromptFile option replaces earlier prompt source options and Config.PromptDir; or
• Config.PromptDir supplies the source when no prompt source option is present.

Inspection does not merge prompt sources, fall back after a malformed match, or introduce a new public prompt repository. The format reference remains the canonical owner of prompt source and exact selection behavior.

Structural Validation

A successful inspection proves that the selected definition can be loaded through the ordinary prompt repository and satisfies its source-level structural rules. This includes:

• deterministic YAML discovery and exact definition selection;
• strict decoding and validation of required identity, version, messages, and output fields;
• normalization and validation of input declarations, message roles, cache control, the default-profile identifier, and output-contract fields; and
• contained, readable resolution of every referenced message content_file.

Referenced message content participates in validation and hashing but is not returned. Inspection does not parse or execute Go templates, resolve template variables or input helpers, or enforce the runtime presence and media type of declared inputs. Those checks remain part of rendering and preparation.

The returned OutputContract is the normalized contract declared by the prompt definition. For json_schema validation, structural inspection proves that a non-blank schema path is declared and returns that path. It does not open, resolve, compile, or return the schema document or its references. Schema-source validation remains part of executable preparation and any future explicit full-source validation feature.

Inspection returns DefaultProfileID as declared metadata. It does not look up that profile, resolve a backend or execution target, check credentials, or prove that the profile is executable. Consumers may call InspectProfile separately when their application policy requires that additional check.

Result Semantics

The result exposes the complete declared input metadata in definition order: name, required status, content type, and human-readable description. It does not expose message templates, session-ID templates, cache-control details, prompt descriptions, raw YAML, source paths, rendered messages, or schema bodies.

PromptID and PromptVersion are the validated identity read from the selected definition. PromptHash is an opaque equality value for the complete loaded definition, including referenced message content. For the same selected definition and observed source state, it must equal the PreparedRun.PromptHash produced by ordinary preparation.

Consumers may compare PromptHash values but must not depend on their algorithm, encoding, length, or suitability as a security proof. Promptkit may change the representation in a future release together with the existing prepared-run hash contract.

Ownership, Consistency, And Concurrency

Each successful call returns a caller-owned snapshot. Mutating the result or its input slice cannot affect the engine, later inspections, or later preparation.

Inspection is safe to call concurrently under the engine's existing repository contracts. It does not mutate prompt sources or install a global cache.

For filesystem-backed and mutable fs.FS sources, the result describes the state observed by that lookup. It does not freeze the definition for a later Prepare, PrepareExecution, or Run, and a source may change between operations. Consumers requiring an exact preflight-to-execution snapshot should use the existing prepared-execution workflow.

Errors And Cancellation

The operation uses existing public error categories:

• a blank or whitespace-only prompt ID matches ErrInvalidRequest;
• an absent exact ID or version matches ErrPromptNotFound and not ErrPromptLoad;
• read, strict-decode, validation, duplicate, ambiguity, referenced-content, and prompt-hashing failures match ErrPromptLoad; and
• a nil engine receiver matches ErrInvalidConfig.

Errors preserve useful underlying collaborator and context identities through errors.Is where the existing facade does so, without exposing internal package types. Context cancellation governs the complete lookup, and no partial inspection result is returned on failure.

Inspection cannot return profile, credential, artifact, rendering, schema, capacity, validation, or model-generation errors because it does not perform those operations.

Compatibility And Boundaries

This feature is additive. Existing prompt formats, prompt source selection, Prepare, prepared execution, Run, profile inspection, and stable JSON contracts remain unchanged.

The method belongs on the root Engine facade. Prompt repositories and internal domain definitions remain implementation details, and consumers cannot use inspection to replace or mutate registered prompt definitions. Engine construction retains its ordinary configuration requirements; this feature does not introduce a separate prompt-only engine.

Inspection is intentionally definition-oriented rather than execution-oriented. It reports what the prompt declares, not an effective request after profile selection, per-run overrides, artifacts, variables, schema resolution, or rendering.

Documentation

When implemented, documentation should retain these ownership boundaries:

• exported declarations and GoDoc own the exact method, result, ownership, hash, error, and cancellation contracts;
• the Promptkit consumer guide explains configuration-time prompt inspection and distinguishes it from profile inspection, Prepare, and prepared execution;
• the format reference continues to own prompt fields, source selection, content-file resolution, and version behavior; and
• internal documentation describes shared prompt loading and hashing without duplicating the public contract.

Non-Goals

This work does not include:

• enumerating, searching, or filtering prompt definitions;
• validating every definition across one or more prompt sources;
• returning raw definitions, YAML, source paths, message bodies, rendered messages, prompt descriptions, or session-ID templates;
• parsing or rendering templates or accepting placeholder inputs and variables;
• resolving the default profile, backend, effective execution target, or credentials;
• loading, compiling, or returning JSON Schema documents or transitive references;
• validating application-specific relationships among prompts;
• freezing a mutable prompt source for later execution;
• exposing or mutating an internal prompt repository;
• adding a stable JSON representation for the new inspection values; or
• changing current prompt selection, hashing, preparation, or execution behavior.

Target End State

After this work:

• consumers can validate one configured prompt without inventing artifacts, variables, or an executable profile;
• lookup uses ordinary prompt-source selection and exact ID/version semantics;
• a successful result proves that the selected definition and its referenced message content are structurally loadable;
• callers receive validated identity, complete declared input metadata, the default profile ID, the normalized output contract, and an opaque prompt equality value;
• the equality value matches ordinary preparation for the same selected definition and observed source state;
• no prompt body, rendered message, schema body, credential, or execution target is exposed;
• returned values are caller-owned and safe to inspect concurrently; and
• broader corpus validation, enumeration, schema validation, and executable preparation remain separate responsibilities.