promptkit/docs/roadmap/inspection.md

# Effective Prompt Requirement Inspection

Status: Implementation planned; the feature is not implemented. The
[implementation plan](implementation.md) owns the ordered migration steps and
implements the agreed analysis scope and inventory below. This document owns
the target end state and migration scope.

## Purpose

Let consumers discover a selected prompt's caller requirements before its input
artifacts exist. Scriptorium can expose that information to operators and to
Narratio, which can compare its planned bindings with Promptkit's exact input
names before undertaking expensive pipeline work.

The originating request is `docs/roadmap/scriptorium-feature-request.md`.
The capability is application-neutral: Promptkit describes prompt requirements;
consumers interpret them for their own configuration, presentation, and workflow.

## Current Baseline

At review baseline `b595dca`:

- [InspectPrompt](../consumers/pkg-promptkit.md#inspect-a-prompt-before-preparation)
  selects a captured prompt and reports declared inputs and output metadata.
  It does not analyze templates or resolve the execution profile.
- [InspectProfile](../consumers/pkg-promptkit.md#inspect-a-profile-before-prompt-work)
  reports one captured profile's effective target separately.
- [Prepare](../consumers/pkg-promptkit.md#prepare-without-model-execution)
  selects the execution context, checks target requirements, loads artifacts
  and applicable schemas, and renders messages. It needs actual input data.
- The configuration adapter captures final effective definitions, message
  contents, and compiled message/session templates during engine construction.
  Runtime rendering uses Go templates, string-valued caller variables, and the
  `input` helper. Declared required inputs are checked independently of template
  branches; active helper calls also require their named inputs.
- Declared input metadata currently uses string fields for content type and
  description. The new result needs to identify a declaration and distinguish
  available metadata from absent metadata, including effective presence where
  necessary; current empty strings alone do not establish source-field presence.

The [format reference](../formats.md#messages-and-templates),
[internal source document](../internal/sources.md), and existing GoDoc own the
implemented semantics. No existing operation provides artifact-free,
context-sensitive requirement discovery.

## Target Public Operation

Add a separate read-only engine operation, planned as
`Engine.InspectRequirements`, with a dedicated request and result. Preserve
`InspectPrompt`, `InspectProfile`, preparation, and execution contracts.

The request supplies a prompt ID, optional version, optional profile selector,
supplied template variables, and an optional direct session ID. It needs no
artifact references or contents, output path, API key, or execution settings
unrelated to discovery.

Selection follows equivalent execution semantics: exact identities, omitted
version ambiguity, request profile or prompt default, and direct session-ID
normalization and bypass. Return selected prompt ID/version and effective
profile ID. Profile selection establishes identity, not generation readiness;
profiles that depend on later execution overrides need not be completed here.

Use the configured engine's captured definitions. Source discovery and layering
remain consumer choices. Results describe that captured context, not a promise
about a later engine or later execution with changed variables.

## Result And Requirement Meanings

Expose public Go values with documented classifications and predictable ordering.
Exact symbol and field spelling belongs in the implementation plan
and GoDoc. No Promptkit JSON wire schema is part of this feature.

The result contains:

- Selected prompt and profile identities.
- Known input names, preserving execution identity, with required, optional,
  or conditional classification for the supplied context.
- Declaration information, descriptions, and content-type metadata where
  available. A template-discovered undeclared input must not look like a
  declared input with known metadata. Report existing media-type metadata,
  not an invented input schema or inferred format constraint.
- An inactive-use explanation for statically identifiable undeclared inputs
  referenced only on known inactive paths. These are optional inventory entries,
  not bindings the current context needs. Keep this explanation separate from
  declared descriptions.
- Variable requirements separately from artifact inputs, with requirement
  classification and whether supplied values satisfy the known requirement.
  Report presence and actual supported semantics, not the supplied values.
- A discovery-completeness indicator and structured uncertainty reasons, with
  safe explanations and relevant identifiers where available.

Required means the caller must supply the input for this context. A declared
required input stays required even if no active template references it. A
declared optional input becomes required when an active template use needs it.
Optional means it is known not to be required for this context; conditional
means its requirement depends on unresolved context. Do not use optional to
hide uncertainty. Known required declarations retain that classification even
when other requirements are conditional or undiscoverable.

Include statically identifiable undeclared references on known inactive paths
as optional entries with an inactive-use explanation. A required or conditional
use elsewhere takes precedence over that inactive-only classification and
explanation. Inactivity itself is not discovery uncertainty.

An unresolved dynamic name can refer to an otherwise optional input entry. Its
absence from recognized active uses alone does not establish that it is
definitely optional; classification and uncertainty must account for that
possibility.

Known input names should be deduplicated across declarations, messages, and
session templates. Aggregate uses without discarding an established required
use. Conditions need not expose raw expressions; machine-readable reason
categories and safe summaries are sufficient for basic consumer handling.

Completeness describes discovery, not readiness or satisfaction. For example,
a missing variable used only to insert text can be a known unsatisfied
requirement while the complete input set is still discoverable. If that
variable selects an input name or branch, its absence can also make discovery
incomplete. An empty complete input set means no artifact bindings are needed;
a complete inventory containing only optional entries also needs no artifact
bindings. An empty incomplete set does not establish that fact. Consumers must
distinguish the inventory from applicable requirements, rather than treating
every reported name as a needed binding.

## Context-Sensitive Discovery Boundary

Analyze parsed templates rather than searching raw text or rendering with
placeholder artifacts. Use supplied variable values where they determine
branches and input names, following execution's presence, truthiness, scope,
and evaluation semantics. An empty string is not an absent variable, and a
nonempty string must not be reinterpreted as a Boolean merely from its spelling.

Include dependencies from messages and the applicable session-ID template.
A direct nonblank session ID bypasses that template, removing only requirements
introduced solely by it; declarations and message uses remain relevant.
The bypassed session template is also excluded from the inactive inventory.

Literal `input` references are discoverable without reading bodies, including
undeclared references. Artifact content being unavailable does not by itself
make discovery incomplete: it does so when the content affects which further
requirements apply. Variable-derived names are discoverable when supported
analysis can determine the exact name from the supplied context.

When missing variables, content-dependent decisions, unresolved names, or
analysis limitations prevent determination, preserve known requirements and
return successful incomplete discovery. Structured reasons should distinguish
these situations without requiring diagnostic-text matching. Unsupported
analysis is not a new prompt-validity rule. Completeness is asserted only when
the supported analysis establishes the requirement set; it does not promise
universal analysis of every valid Go template.

The selected practical analysis core and inactive inventory are described below.
They do not change execution or expand the template language.

## Effects, Errors, And Disclosure

Inspection uses captured prompt resources and needs no future artifact files,
credential lookup, model access, capacity admission, or execution artifact
creation. It does not prepare an output-validation schema or validate future
input bytes. Necessary configuration/resource failures can still prevent engine
construction or inspection; this is not a general resource preflight.

Use existing public selection/error identities wherever applicable, including
invalid requests, missing prompt/profile selections, and ambiguous versions.
Preserve ordinary context cancellation behavior. Discovery uncertainty is a
result state, distinct from an operational or selection error. Expose any
additional necessary classifications through the public facade, not diagnostic
text parsing or internal types.

Results and safe diagnostics exclude credential values, artifact payloads,
raw/rendered prompt content, and unrelated supplied variable values. Selected
identities, input/variable names, and declared descriptions are intentional
operator metadata. A resolved dynamic input identifier may equal a supplied
variable value; reporting that identifier is necessary to describe the binding.
Keep the disclosure exception limited to its role as an identifier, without
echoing the variable map or the expression that produced it. Scriptorium owns
the presentation and disclosure of this metadata to its callers.

A complete result describes requirements, not successful rendering or generation,
future artifact validity, or provider availability. Incomplete results remain
useful without authorizing consumers to treat unknown dependencies as absent.

## Work Needed From The Baseline

Add public request/result values and the engine method through the existing
facade. Put selection coordination and analysis in internal components with
cohesive ownership. Reuse existing prompt/profile selection, session override,
and template parsing semantics rather than introducing a second preparation
path or a general-purpose template interpreter.

Make the captured templates and effective declaration metadata available to
internal inspection. Retain any required metadata presence at the ConfigKit
handoff. Apply [ADR 0001](../adr/0001-configuration-boundary.md): inspection
interprets final effective definitions, not raw sources, configuration history,
or intermediate artifacts. Operator provenance remains a separate explicit API.

Implement requirement aggregation, context-sensitive analysis, and safe public
uncertainty/error conversion. Keep request-specific analysis separate from
shared captured state. Prefer direct traversals and indexed aggregation; avoid
enumerating every possible branch combination or repeated full-template scans.
Predictable result ordering does not require a stronger complexity guarantee
or a new global cache.

Add focused analyzer tests and public-facade contract coverage under the
[testing policy](../policy/testing.md). Use real captured definitions and
templates, synthetic variables, and explicit unavailable artifacts. Compare
complete discovery with representative execution semantics. Test incomplete
results separately from errors and verify disclosure boundaries and absence
of artifact reads/provider calls. Do not retest ConfigKit's loading matrix.

When implementation lands, update GoDoc, the consumer guide, relevant format
cross-references, and the internal component/source documents. Describe the
analysis boundary and examples of complete, unsatisfied, and incomplete results
in their canonical owners. Release notes explain the new capability; they do
not own its contract. The implementation plan owns implementation sequencing.

## Acceptance Outcomes

- A required `transcript` is reported exactly, allowing a consumer to detect a
  planned binding under the wrong name before the transcript exists.
- Literal undeclared helper references are discoverable without making their
  absence from declarations a new definition error.
- Declared required inputs remain required in inactive branches; optional
  declarations remain optional where no applicable use makes them required.
- Statically identifiable inactive undeclared references are optional entries
  with an inactive-use explanation; active and unresolved uses of the same name
  take precedence. These entries do not introduce variable requirements or
  incompleteness solely from skipped expressions.
- Supported variable-dependent branches and names agree with execution;
  unresolved content/name/analysis cases retain known entries and uncertainty.
- Version/profile/default selection and session-template bypass agree with
  execution without credential or generation-readiness checks.
- Variables are reported separately, with absent and empty supplied values
  interpreted according to the existing template semantics.
- Artifact-free complete results, including all-optional inventories, are
  distinguishable from empty partial discovery.
- Errors remain publicly classifiable; results and safe diagnostics disclose
  identifiers and declared metadata without unrelated content or secret values.
- Existing inspection and execution APIs retain their contracts. Pipeline
  planning, binding validation, CLI/HTTP exposure, presentation, JSON schemas,
  graph rendering, batch interfaces, and provider preflight remain downstream.

## Selected Analysis Scope And Inventory

### Initial Template Analysis Coverage

Implement a documented practical core: literal input calls,
direct caller-variable access, ordinary `if`/`with` control flow, and simple
variable-derived names and comparisons where their values are determinable.
Account for template variable scope and short-circuit behavior for supported
expressions. Additional constructs can be supported where straightforward;
otherwise report incompleteness when they affect requirement discovery. The
implementation plan specifies the exact expression families for this baseline.
This covers the representative consumer needs while keeping completeness
honest and avoiding a second general-purpose interpreter.

### Inactive Undeclared Reference Inventory

Always include declarations and discovered active or unresolved dependencies.
Also include statically identifiable undeclared references on known inactive
paths as optional entries with an inactive-use explanation. This broader
inventory helps operators explore prompt behavior while preserving the
distinction between inventory and applicable dependencies.

Collect inactive names from syntax without evaluating skipped expressions.
The initial scope covers literal helper names and equivalent parenthesized or
piped literal forms. Skipped dynamic names need not be resolved, and skipped
variable accesses do not become caller requirements or uncertainty reasons.
Include known skipped branches and short-circuit operands, without expanding
analysis to never-invoked template definitions or a bypassed session template.
Preserve declared optional metadata and required declarations; when another
use makes a name required or conditional, its inactive occurrence does not
override that classification or imply that the name is inactive-only.