Document prompt inspection API

This commit is contained in:
2026-07-30 21:06:33 +00:00
parent e920168b30
commit fc3255967e
9 changed files with 68 additions and 68 deletions

View File

@@ -40,6 +40,30 @@ validation, and default transport behavior. Source discovery, format
validation, and profile precedence are defined by the
[framework format reference](../formats.md).
## Inspect A Prompt Before Preparation
Use [`Engine.InspectPrompt`](../../engine.go) to check one configured prompt's
declared inputs and output workflow without creating placeholder inputs or
resolving a profile:
```go
inspection, err := engine.InspectPrompt(ctx, "meeting.summary", "")
if err != nil {
return err
}
for _, input := range inspection.Inputs {
// Compare the declared input with application configuration.
}
```
Use this configuration-time boundary when the application needs only the
declared prompt interface. Use `InspectProfile` separately when it must also
check a configured profile. Use `Prepare` when it needs inputs, schemas, or
rendered messages, and use prepared execution when that work must remain tied
to later execution. The method's [GoDoc](../../engine.go) owns exact fields,
hash, ownership, and error semantics.
## Prepare Without Model Execution
[`Engine.Prepare`](../../engine.go) resolves the selected prompt and profile,

View File

@@ -58,6 +58,11 @@ When a request omits a version, the selected prompt ID must identify exactly
one definition. When it supplies a version, the ID and version pair must be
unique.
Exact prompt inspection uses this same configured source, strict decoding,
referenced content-file resolution, and ID/version selection. It reports the
selected definition's declared metadata without changing the prompt format or
executing the definition.
### Inputs
Each `inputs` item has these fields:

View File

@@ -11,7 +11,7 @@ contributor workflow and validation.
| Component | Implemented responsibility | References |
| --- | --- | --- |
| Root `promptkit` package | Provides the supported engine facade, source, backend-registration, and injection options, public request, result, and profile-inspection values, opaque prepared-execution handles, profile construction, extension interfaces, value conversion, redacted formatting, public error mapping, and engine-local assembly. | [Package GoDoc](../../doc.go), [prepared execution](../../prepared_execution.go), [backend API](../../backends.go), [engine assembly](../../engine.go) |
| Root `promptkit` package | Provides the supported engine facade, source, backend-registration, and injection options, public request, result, prompt-inspection, and profile-inspection values, opaque prepared-execution handles, profile construction, extension interfaces, value conversion, redacted formatting, public error mapping, and engine-local assembly. | [Package GoDoc](../../doc.go), [prepared execution](../../prepared_execution.go), [backend API](../../backends.go), [engine assembly](../../engine.go) |
| `examples/go-library/prepare` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, and `Prepare`. It is not a public library package. | [Example program](../../examples/go-library/prepare/main.go) |
| `examples/go-library/run` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, an injected deterministic model client, and `Run`. It is not a public library package. | [Example program](../../examples/go-library/run/main.go) |
| `internal/backend` | Constructs each engine's immutable registry from the built-in OpenRouter definition and consumer additions, validates and defensively copies definitions through the shared JSON-value package, and consumes the LLM-owned OpenAI-compatible reserved request-field rule. | [Backend registry](../../internal/backend/registry.go) |
@@ -27,7 +27,7 @@ contributor workflow and validation.
| `internal/artifact` | Resolves ordinary inline and unrestricted caller-selected file references into copied artifacts with metadata and hashes. | [Internal sources and validation](sources.md) |
| `internal/validate` | Validates basic, JSON, and JSON Schema output using operating-system filesystem or `fs.FS` schema sources and creates frozen validation plans for prepared execution. | [Framework formats](../formats.md#schemas), [internal sources and validation](sources.md) |
| `internal/llm` | Defines the internal generation boundary and implements outbound OpenAI-compatible chat requests from resolved execution targets, including response decoding, authentication, deadline handling, and ownership of the OpenAI-compatible reserved request-field policy. | [Internal model client](llm.md) |
| `internal/usecase` | Resolves profiles, backends, and targets for exact inspection and request settings for preparation, and coordinates ordinary execution and one-attempt prepared execution across internal sources, rendering, artifact loading, generation, validation, capacity, and optional repair. | [Internal runner](runner.md), [prepared-execution implementation](../../internal/usecase/prepared_execution.go) |
| `internal/usecase` | Resolves prompt definitions and hashes, profiles, backends, and targets for exact inspection and request settings for preparation, and coordinates ordinary execution and one-attempt prepared execution across internal sources, rendering, artifact loading, generation, validation, capacity, and optional repair. | [Internal runner](runner.md), [prepared-execution implementation](../../internal/usecase/prepared_execution.go) |
The root package assembles these internal components without exposing their
representations. Consumers depend only on the root facade.

View File

@@ -28,6 +28,19 @@ constructor does not enable one.
Each invocation carries its state in request, prepared-run, and result values.
The runner has no durable run or session store.
## Shared Prompt Selection
The runner uses one prompt-selection and hashing boundary for ordinary
preparation and exact prompt inspection. Preparation retains its early
request-ID check before direct-session normalization; both operations then use
the configured prompt repository to select one definition, load referenced
message content, and calculate the same prompt hash.
Inspection stops after that structural lookup. It does not parse templates or
touch profile, artifact, schema, renderer, validator, admission, or model
collaborators. The root [`Engine.InspectPrompt`](../../engine.go) GoDoc owns
the public operation's exact contract.
## Shared Profile Selection
The runner uses one profile-selection and target-resolution boundary for

View File

@@ -16,6 +16,11 @@ validation modes, built-in catalog, and source precedence.
definitions, selects an ID and optional version, and resolves file-backed
message content within the selected operating-system or `fs.FS` source.
Exact prompt inspection performs one point-in-time lookup through that same
repository and validates referenced message content before returning declared
metadata. It does not parse templates or read profile, input, or schema
sources, and it does not retain the definition for a later execution.
Its package tests own prompt selection, strict decoding, definition validation,
duplicate detection, and source containment:
[prompt-definition repository tests](../../internal/promptdef/repository_test.go).

View File

@@ -33,10 +33,6 @@ consumers.
## Ideas
Prompt-definition inspection has been selected for active planning in the
[focused feature roadmap](prompt-inspection.md). The remaining idea is still
available for future selection.
### Structured capacity errors
Add safe structured context to backend admission rejection, as requested by

View File

@@ -1,6 +1,6 @@
# Prompt-Definition Inspection Implementation Plan
**Status:** Accepted.
**Status:** Complete.
## Purpose

View File

@@ -1,6 +1,6 @@
# Prompt-Definition Inspection
**Status:** Accepted.
**Status:** Complete.
## Purpose

View File

@@ -106,8 +106,9 @@ describes the actual execution.
## Priority 2: Prompt-Definition Inspection
**Disposition:** Accepted into the
[prompt-definition inspection](prompt-inspection.md) feature roadmap.
**Disposition:** Implemented as
[`Engine.InspectPrompt`](../../engine.go). See the
[consumer guidance](../consumers/pkg-promptkit.md#inspect-a-prompt-before-preparation).
### Downstream need
@@ -123,65 +124,22 @@ response schemas that implement those reports. It needs to validate that the
report registry and embedded prompt corpus agree before weather collection or
provider execution.
### Current integration option
### Previous integration option
Weatherreporter can maintain synthetic data-package fixtures and call
`Engine.Prepare` for every report prompt during tests. Runtime validation can
also occur through the ordinary per-report preparation stage.
Before prompt inspection was available, Weatherreporter could maintain
synthetic data-package fixtures and call `Engine.Prepare` for every report
prompt during tests. Runtime validation could also occur through the ordinary
per-report preparation stage.
This works, but it requires complete placeholder inputs and profile resolution
when the application primarily wants to inspect prompt identity and declared
contracts.
### Requested capability
Add exact prompt-definition lookup without rendering or generation:
```go
type PromptInfo struct {
PromptID string
PromptVersion string
PromptHash string
DefaultProfileID string
Inputs []InputDefinition
OutputContract OutputContract
}
func (e *Engine) ResolvePrompt(
ctx context.Context,
promptID string,
promptVersion string,
) (PromptInfo, error)
```
The exact returned shape may differ. Weatherreporter needs enough information
to verify prompt existence, version selection, declared inputs, default
profile identity, output format, validation mode, and schema selection without
supplying synthetic prompt input.
### Design considerations
- Use ordinary PromptKit prompt-source precedence and exact ID/version
selection.
- Fully load and structurally validate the selected prompt definition.
- Validate referenced prompt content files without rendering their templates.
- Resolve and validate the selected output contract and schema reference where
practical.
- Return an opaque prompt-definition equality value rather than raw source
bytes.
- Do not return rendered messages, schema bodies, profile credentials, or
another source of sensitive content.
- Preserve typed or sentinel errors for missing and invalid prompts.
- Return caller-owned values.
- Enumeration of all known prompts is not required for Weatherreporter; exact
lookup is sufficient.
This required complete placeholder inputs and profile resolution when the
application primarily wanted to inspect prompt identity and declared contracts.
### Value to Weatherreporter
This would let Weatherreporter directly verify that every report prompt
exists, requires the curated `data_package` input, and declares the expected
Markdown or JSON Schema output contract. It would reduce synthetic test setup
and move failures ahead of weather collection.
The implemented interface lets Weatherreporter directly verify that every
report prompt exists, requires the curated `data_package` input, and declares
the expected Markdown or JSON Schema output contract. It reduces synthetic
test setup and moves failures ahead of weather collection.
## Priority 3: Prompt-Independent Profile Inspection
@@ -403,11 +361,10 @@ Weatherreporter would be:
1. Add executable preparation handles, ideally sharing implementation with an
atomic detailed-run API.
2. Add prompt-definition inspection.
3. Consider eager source validation after evaluating whether the two exact
2. Consider eager source validation after evaluating whether the two exact
inspection APIs are sufficient.
4. Add structured generation errors.
5. Add structured capacity errors and semantic execution-target fingerprints
3. Add structured generation errors.
4. Add structured capacity errors and semantic execution-target fingerprints
as lower-priority operational improvements.
The first item removes the only material integration workaround. Prompt and