Add internal prompt inspection

docs/roadmap/future.md

@@ -33,19 +33,9 @@ consumers.
 
 ## Ideas
 
-### Prompt-definition inspection
+Prompt-definition inspection has been selected for active planning in the
-
+[focused feature roadmap](prompt-inspection.md). The remaining idea is still
-Provide exact prompt-definition lookup without rendering, placeholder inputs,
+available for future selection.
-profile resolution, or model generation, as requested by
-[Weatherreporter](weatherreporter-promptkit-wishlist.md#priority-2-prompt-definition-inspection).
-
-- Return caller-owned identity, version, input definitions, default-profile,
-  output-contract, and opaque definition-equality information.
-- Apply ordinary prompt-source precedence and exact ID/version selection.
-- Validate the selected definition and referenced prompt content
-  structurally, without returning source bodies or rendered messages.
-- Leave complete cross-source corpus validation and enumeration outside the
-  initial inspection contract.
 
 ### Structured capacity errors
 

docs/roadmap/prompt-inspection.md

@@ -0,0 +1,268 @@
+# Prompt-Definition Inspection
+
+**Status:** Accepted.
+
+## Purpose
+
+Allow consumers to look up one prompt definition by ID and optional version
+and inspect its declared interface without supplying placeholder inputs,
+resolving a profile, rendering templates, loading schemas, or invoking a
+model.
+
+This provides a focused configuration-validation boundary for Weatherreporter,
+whose need is recorded in its
+[Promptkit wishlist](weatherreporter-promptkit-wishlist.md#priority-2-prompt-definition-inspection),
+while preserving Promptkit's deferred, request-oriented source model.
+
+## Motivation
+
+Weatherreporter has a fixed application-owned registry of report definitions.
+It needs to verify that every configured prompt exists and declares the
+expected inputs and output workflow before collecting weather data or starting
+provider work.
+
+The current workaround is to construct synthetic artifacts and variables and
+call `Engine.Prepare`. That validates much more than the application needs:
+prompt inputs are loaded, templates and session IDs are rendered, a profile
+and backend are resolved, credentials are checked, and JSON Schemas may be
+loaded and compiled.
+
+Promptkit already owns prompt source selection, exact ID and version lookup,
+strict definition decoding, referenced content-file resolution, and prompt
+hashing. It should expose that cohesive subset directly rather than requiring
+consumers to reproduce its rules or maintain placeholder execution fixtures.
+
+## Consumer Workflow
+
+The target public workflow is:
+
+```go
+inspection, err := engine.InspectPrompt(ctx, promptID, promptVersion)
+if err != nil {
+	// Reject or report the configured prompt.
+	return
+}
+
+for _, input := range inspection.Inputs {
+	// Compare the declared prompt interface with application configuration.
+}
+```
+
+The target public surface is:
+
+```go
+type PromptInputDefinition struct {
+	Name        string
+	Required    bool
+	ContentType string
+	Description string
+}
+
+type PromptInspection struct {
+	PromptID         string
+	PromptVersion    string
+	PromptHash       string
+	DefaultProfileID string
+	Inputs           []PromptInputDefinition
+	OutputContract   OutputContract
+}
+
+func (e *Engine) InspectPrompt(
+	ctx context.Context,
+	promptID string,
+	promptVersion string,
+) (*PromptInspection, error)
+```
+
+The exported declarations and GoDoc will own the exact implemented contract.
+The important public shape is exact lookup through an existing engine, one
+caller-owned inspection value, declared input metadata, the declared output
+contract, and the same opaque prompt equality value used by preparation.
+
+`PromptInspection` and `PromptInputDefinition` do not need stable JSON
+representations. Consumers that persist application configuration or
+diagnostics can define their own format and select the fields they require.
+The existing stable representations of `OutputContract`, `OutputFormat`, and
+`ValidationMode` remain unchanged.
+
+## Lookup And Source Selection
+
+`InspectPrompt` requires a non-blank prompt ID and performs the same
+case-sensitive exact selection used by ordinary preparation. Inspection does
+not trim or otherwise canonicalize a non-blank ID or version independently of
+that execution path.
+
+When `promptVersion` is non-empty, the configured prompt source must contain
+exactly one matching ID and version pair. When it is empty, the source must
+contain exactly one definition with the requested ID; multiple matching
+versions remain an ambiguous source-selection error.
+
+Lookup uses the engine's normally selected prompt source:
+
+- the last `WithPromptFS` or `WithPromptFile` option replaces earlier prompt
+  source options and `Config.PromptDir`; or
+- `Config.PromptDir` supplies the source when no prompt source option is
+  present.
+
+Inspection does not merge prompt sources, fall back after a malformed match,
+or introduce a new public prompt repository. The
+[format reference](../formats.md#prompt-definitions) remains the canonical
+owner of prompt source and exact selection behavior.
+
+## Structural Validation
+
+A successful inspection proves that the selected definition can be loaded
+through the ordinary prompt repository and satisfies its source-level
+structural rules. This includes:
+
+- deterministic YAML discovery and exact definition selection;
+- strict decoding and validation of required identity, version, messages, and
+  output fields;
+- normalization and validation of input declarations, message roles, cache
+  control, the default-profile identifier, and output-contract fields; and
+- contained, readable resolution of every referenced message
+  `content_file`.
+
+Referenced message content participates in validation and hashing but is not
+returned. Inspection does not parse or execute Go templates, resolve template
+variables or input helpers, or enforce the runtime presence and media type of
+declared inputs. Those checks remain part of rendering and preparation.
+
+The returned `OutputContract` is the normalized contract declared by the
+prompt definition. For `json_schema` validation, structural inspection proves
+that a non-blank schema path is declared and returns that path. It does not
+open, resolve, compile, or return the schema document or its references.
+Schema-source validation remains part of executable preparation and any
+future explicit full-source validation feature.
+
+Inspection returns `DefaultProfileID` as declared metadata. It does not look
+up that profile, resolve a backend or execution target, check credentials, or
+prove that the profile is executable. Consumers may call `InspectProfile`
+separately when their application policy requires that additional check.
+
+## Result Semantics
+
+The result exposes the complete declared input metadata in definition order:
+name, required status, content type, and human-readable description. It does
+not expose message templates, session-ID templates, cache-control details,
+prompt descriptions, raw YAML, source paths, rendered messages, or schema
+bodies.
+
+`PromptID` and `PromptVersion` are the validated identity read from the
+selected definition. `PromptHash` is an opaque equality value for the complete
+loaded definition, including referenced message content. For the same selected
+definition and observed source state, it must equal the `PreparedRun.PromptHash`
+produced by ordinary preparation.
+
+Consumers may compare `PromptHash` values but must not depend on their
+algorithm, encoding, length, or suitability as a security proof. Promptkit may
+change the representation in a future release together with the existing
+prepared-run hash contract.
+
+## Ownership, Consistency, And Concurrency
+
+Each successful call returns a caller-owned snapshot. Mutating the result or
+its input slice cannot affect the engine, later inspections, or later
+preparation.
+
+Inspection is safe to call concurrently under the engine's existing
+repository contracts. It does not mutate prompt sources or install a global
+cache.
+
+For filesystem-backed and mutable `fs.FS` sources, the result describes the
+state observed by that lookup. It does not freeze the definition for a later
+`Prepare`, `PrepareExecution`, or `Run`, and a source may change between
+operations. Consumers requiring an exact preflight-to-execution snapshot
+should use the existing prepared-execution workflow.
+
+## Errors And Cancellation
+
+The operation uses existing public error categories:
+
+- a blank or whitespace-only prompt ID matches `ErrInvalidRequest`;
+- an absent exact ID or version matches `ErrPromptNotFound` and not
+  `ErrPromptLoad`;
+- read, strict-decode, validation, duplicate, ambiguity, referenced-content,
+  and prompt-hashing failures match `ErrPromptLoad`; and
+- a nil engine receiver matches `ErrInvalidConfig`.
+
+Errors preserve useful underlying collaborator and context identities through
+`errors.Is` where the existing facade does so, without exposing internal
+package types. Context cancellation governs the complete lookup, and no
+partial inspection result is returned on failure.
+
+Inspection cannot return profile, credential, artifact, rendering, schema,
+capacity, validation, or model-generation errors because it does not perform
+those operations.
+
+## Compatibility And Boundaries
+
+This feature is additive. Existing prompt formats, prompt source selection,
+`Prepare`, prepared execution, `Run`, profile inspection, and stable JSON
+contracts remain unchanged.
+
+The method belongs on the root `Engine` facade. Prompt repositories and
+internal domain definitions remain implementation details, and consumers
+cannot use inspection to replace or mutate registered prompt definitions.
+Engine construction retains its ordinary configuration requirements; this
+feature does not introduce a separate prompt-only engine.
+
+Inspection is intentionally definition-oriented rather than
+execution-oriented. It reports what the prompt declares, not an effective
+request after profile selection, per-run overrides, artifacts, variables,
+schema resolution, or rendering.
+
+## Documentation
+
+When implemented, documentation should retain these ownership boundaries:
+
+- exported declarations and GoDoc own the exact method, result, ownership,
+  hash, error, and cancellation contracts;
+- the Promptkit consumer guide explains configuration-time prompt inspection
+  and distinguishes it from profile inspection, `Prepare`, and prepared
+  execution;
+- the format reference continues to own prompt fields, source selection,
+  content-file resolution, and version behavior; and
+- internal documentation describes shared prompt loading and hashing without
+  duplicating the public contract.
+
+## Non-Goals
+
+This work does not include:
+
+- enumerating, searching, or filtering prompt definitions;
+- validating every definition across one or more prompt sources;
+- returning raw definitions, YAML, source paths, message bodies, rendered
+  messages, prompt descriptions, or session-ID templates;
+- parsing or rendering templates or accepting placeholder inputs and
+  variables;
+- resolving the default profile, backend, effective execution target, or
+  credentials;
+- loading, compiling, or returning JSON Schema documents or transitive
+  references;
+- validating application-specific relationships among prompts;
+- freezing a mutable prompt source for later execution;
+- exposing or mutating an internal prompt repository;
+- adding a stable JSON representation for the new inspection values; or
+- changing current prompt selection, hashing, preparation, or execution
+  behavior.
+
+## Target End State
+
+After this work:
+
+- consumers can validate one configured prompt without inventing artifacts,
+  variables, or an executable profile;
+- lookup uses ordinary prompt-source selection and exact ID/version semantics;
+- a successful result proves that the selected definition and its referenced
+  message content are structurally loadable;
+- callers receive validated identity, complete declared input metadata, the
+  default profile ID, the normalized output contract, and an opaque prompt
+  equality value;
+- the equality value matches ordinary preparation for the same selected
+  definition and observed source state;
+- no prompt body, rendered message, schema body, credential, or execution
+  target is exposed;
+- returned values are caller-owned and safe to inspect concurrently; and
+- broader corpus validation, enumeration, schema validation, and executable
+  preparation remain separate responsibilities.

docs/roadmap/weatherreporter-promptkit-wishlist.md

@@ -107,7 +107,7 @@ describes the actual execution.
 ## Priority 2: Prompt-Definition Inspection
 
 **Disposition:** Accepted into the
-[future catalog](future.md#prompt-definition-inspection).
+[prompt-definition inspection](prompt-inspection.md) feature roadmap.
 
 ### Downstream need
 

internal/domain/domain.go

@@ -145,6 +145,16 @@ type PromptDefinition struct {
 	Validation     OutputContract          `yaml:"validation"`
 }
 
+// PromptInspection is the resolved result of exact prompt inspection.
+type PromptInspection struct {
+	PromptID         string
+	PromptVersion    string
+	PromptHash       string
+	DefaultProfileID string
+	Inputs           []PromptInput
+	OutputContract   OutputContract
+}
+
 // PromptInput describes one named input expected by a prompt definition.
 type PromptInput struct {
 	Name        string `yaml:"name"`

internal/usecase/prompt_inspection.go

@@ -0,0 +1,73 @@
+package usecase
+
+import (
+	"context"
+	"fmt"
+	"strings"
+
+	"gitea.maximumdirect.net/eric/promptkit/internal/domain"
+)
+
+type resolvedPromptDefinition struct {
+	definition *domain.PromptDefinition
+	hash       string
+}
+
+func (r *Runner) resolvePromptDefinition(
+	ctx context.Context,
+	promptID string,
+	promptVersion string,
+) (*resolvedPromptDefinition, error) {
+	if strings.TrimSpace(promptID) == "" {
+		return nil, fmt.Errorf("%w: prompt id is required", ErrInvalidRequest)
+	}
+	if r == nil || r.promptDefs == nil {
+		return nil, fmt.Errorf("%w: prompt repository is not configured", ErrPromptLoad)
+	}
+
+	definition, err := r.promptDefs.GetPromptDefinition(ctx, promptID, promptVersion)
+	if err != nil {
+		return nil, fmt.Errorf("%w: %w", ErrPromptLoad, err)
+	}
+	if definition == nil {
+		return nil, fmt.Errorf("%w: prompt repository returned nil definition", ErrPromptLoad)
+	}
+
+	hash, err := hashPromptDefinition(definition)
+	if err != nil {
+		return nil, fmt.Errorf("%w: failed to hash prompt definition: %v", ErrPromptLoad, err)
+	}
+	return &resolvedPromptDefinition{definition: definition, hash: hash}, nil
+}
+
+// InspectPrompt resolves one explicit prompt without execution work.
+func (r *Runner) InspectPrompt(
+	ctx context.Context,
+	promptID string,
+	promptVersion string,
+) (*domain.PromptInspection, error) {
+	if strings.TrimSpace(promptID) == "" {
+		return nil, fmt.Errorf("%w: prompt id is required", ErrInvalidRequest)
+	}
+	select {
+	case <-ctx.Done():
+		return nil, fmt.Errorf("%w: %w", ErrPromptLoad, ctx.Err())
+	default:
+	}
+
+	selection, err := r.resolvePromptDefinition(ctx, promptID, promptVersion)
+	if err != nil {
+		return nil, err
+	}
+	inputs := make([]domain.PromptInput, len(selection.definition.Inputs))
+	copy(inputs, selection.definition.Inputs)
+
+	return &domain.PromptInspection{
+		PromptID:         selection.definition.ID,
+		PromptVersion:    selection.definition.Version,
+		PromptHash:       selection.hash,
+		DefaultProfileID: selection.definition.DefaultProfile,
+		Inputs:           inputs,
+		OutputContract:   selection.definition.Validation,
+	}, nil
+}

internal/usecase/prompt_inspection_test.go

@@ -0,0 +1,157 @@
+package usecase
+
+import (
+	"context"
+	"errors"
+	"reflect"
+	"testing"
+
+	"gitea.maximumdirect.net/eric/promptkit/internal/domain"
+	"gitea.maximumdirect.net/eric/promptkit/internal/promptdef"
+)
+
+type inspectionPromptRepository struct {
+	definition *domain.PromptDefinition
+	err        error
+	calls      int
+	id         string
+	version    string
+}
+
+func (r *inspectionPromptRepository) GetPromptDefinition(
+	_ context.Context,
+	id string,
+	version string,
+) (*domain.PromptDefinition, error) {
+	r.calls++
+	r.id = id
+	r.version = version
+	if r.err != nil {
+		return nil, r.err
+	}
+	return r.definition, nil
+}
+
+func TestRunnerInspectPromptResolvesOneDefinitionWithoutExecutionCollaborators(t *testing.T) {
+	definition := &domain.PromptDefinition{
+		ID:             "normalized.prompt",
+		Version:        "1.2.3",
+		DefaultProfile: "not-resolved",
+		Inputs: []domain.PromptInput{
+			{Name: "document", Required: true, ContentType: "text/plain", Description: "Source document."},
+			{Name: "audience", ContentType: "text/plain", Description: "Intended reader."},
+		},
+		Validation: domain.OutputContract{
+			Format:         domain.FormatJSON,
+			ValidationMode: domain.ValidationJSONSchema,
+			SchemaPath:     "schemas/result.json",
+			RepairAttempts: 2,
+		},
+	}
+	repository := &inspectionPromptRepository{definition: definition}
+	runner := &Runner{promptDefs: repository}
+
+	inspection, err := runner.InspectPrompt(context.Background(), " prompt-id ", " version ")
+	if err != nil {
+		t.Fatalf("inspect prompt: %v", err)
+	}
+	wantHash, err := hashPromptDefinition(definition)
+	if err != nil {
+		t.Fatalf("hash prompt definition: %v", err)
+	}
+	if repository.calls != 1 || repository.id != " prompt-id " || repository.version != " version " {
+		t.Fatalf("prompt lookup=(calls=%d id=%q version=%q), want one unchanged lookup", repository.calls, repository.id, repository.version)
+	}
+	if inspection.PromptID != definition.ID ||
+		inspection.PromptVersion != definition.Version ||
+		inspection.PromptHash != wantHash ||
+		inspection.DefaultProfileID != definition.DefaultProfile ||
+		!reflect.DeepEqual(inspection.Inputs, definition.Inputs) ||
+		inspection.OutputContract != definition.Validation {
+		t.Fatalf("inspection=%#v, want definition metadata", inspection)
+	}
+
+	inspection.Inputs[0].Name = "changed"
+	second, err := runner.InspectPrompt(context.Background(), " prompt-id ", " version ")
+	if err != nil {
+		t.Fatalf("inspect prompt again: %v", err)
+	}
+	if definition.Inputs[0].Name != "document" || second.Inputs[0].Name != "document" {
+		t.Fatalf("inspection input mutation escaped caller result: definition=%#v next=%#v", definition.Inputs, second.Inputs)
+	}
+}
+
+func TestRunnerInspectPromptClassifiesFailuresWithoutRepositoryWorkAfterCancellation(t *testing.T) {
+	t.Run("blank ID", func(t *testing.T) {
+		repository := &inspectionPromptRepository{}
+		_, err := (&Runner{promptDefs: repository}).InspectPrompt(context.Background(), " \t ", "version")
+		if !errors.Is(err, ErrInvalidRequest) || repository.calls != 0 {
+			t.Fatalf("blank inspection=(%v, calls=%d), want invalid request without lookup", err, repository.calls)
+		}
+	})
+
+	t.Run("canceled context", func(t *testing.T) {
+		ctx, cancel := context.WithCancel(context.Background())
+		cancel()
+		repository := &inspectionPromptRepository{}
+		_, err := (&Runner{promptDefs: repository}).InspectPrompt(ctx, "prompt", "version")
+		if !errors.Is(err, ErrPromptLoad) || !errors.Is(err, context.Canceled) || repository.calls != 0 {
+			t.Fatalf("canceled inspection=(%v, calls=%d), want prompt load and context identities without lookup", err, repository.calls)
+		}
+	})
+
+	t.Run("missing prompt", func(t *testing.T) {
+		repository := &inspectionPromptRepository{err: promptdef.ErrPromptDefinitionNotFound}
+		_, err := (&Runner{promptDefs: repository}).InspectPrompt(context.Background(), "missing", "version")
+		if !errors.Is(err, ErrPromptLoad) || !errors.Is(err, promptdef.ErrPromptDefinitionNotFound) {
+			t.Fatalf("missing prompt error=%v, want prompt load and not-found identities", err)
+		}
+	})
+
+	t.Run("defensive prompt dependencies", func(t *testing.T) {
+		var nilRunner *Runner
+		cases := []struct {
+			name   string
+			runner *Runner
+		}{
+			{name: "nil runner", runner: nilRunner},
+			{name: "nil repository", runner: &Runner{}},
+			{name: "nil definition", runner: &Runner{promptDefs: &inspectionPromptRepository{}}},
+		}
+		for _, tc := range cases {
+			t.Run(tc.name, func(t *testing.T) {
+				_, err := tc.runner.InspectPrompt(context.Background(), "prompt", "version")
+				if !errors.Is(err, ErrPromptLoad) {
+					t.Fatalf("inspection error=%v, want prompt load", err)
+				}
+			})
+		}
+	})
+}
+
+func TestRunnerPrepareUsesThePromptInspectionSelectionAndHash(t *testing.T) {
+	definition := promptDef(domain.FormatMarkdown, domain.ValidationBasic, 0)
+	repository := &fakePromptRepo{def: definition}
+	runner := NewRunner(
+		repository,
+		&fakeExecutionProfileRepo{profiles: map[string]*domain.ExecutionProfile{"exec": defaultExecutionProfile()}},
+		nil,
+		&fakeArtifactReader{},
+		&fakeRenderer{rendered: &domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hello"}}}},
+		nil,
+		nil,
+		nil,
+	)
+
+	inspection, err := runner.InspectPrompt(context.Background(), definition.ID, definition.Version)
+	if err != nil {
+		t.Fatalf("inspect prompt: %v", err)
+	}
+	prepared, err := runner.Prepare(context.Background(), domain.RunRequest{PromptID: definition.ID, PromptVersion: definition.Version, ProfileID: "exec"})
+	if err != nil {
+		t.Fatalf("prepare prompt: %v", err)
+	}
+	if inspection.PromptHash != prepared.PromptHash {
+		t.Fatalf("inspection hash=%q, preparation hash=%q", inspection.PromptHash, prepared.PromptHash)
+	}
+}

internal/usecase/runner.go

@@ -266,14 +266,12 @@ func (r *Runner) resolvePreparation(
 		return nil, fmt.Errorf("%w: session_id: %v", ErrInvalidRequest, err)
 	}
 
-	def, err := r.promptDefs.GetPromptDefinition(ctx, req.PromptID, req.PromptVersion)
+	promptSelection, err := r.resolvePromptDefinition(ctx, req.PromptID, req.PromptVersion)
 	if err != nil {
-		return nil, fmt.Errorf("%w: %w", ErrPromptLoad, err)
+		return nil, err
-	}
-	promptDefinitionHash, err := hashPromptDefinition(def)
-	if err != nil {
-		return nil, fmt.Errorf("%w: failed to hash prompt definition: %v", ErrPromptLoad, err)
 	}
+	def := promptSelection.definition
+	promptDefinitionHash := promptSelection.hash
 
 	selectedProfileID := strings.TrimSpace(req.ProfileID)
 	if selectedProfileID == "" {