diff --git a/docs/consumers/pkg-promptkit.md b/docs/consumers/pkg-promptkit.md index d8b5ced..2f55f8c 100644 --- a/docs/consumers/pkg-promptkit.md +++ b/docs/consumers/pkg-promptkit.md @@ -40,6 +40,30 @@ validation, and default transport behavior. Source discovery, format validation, and profile precedence are defined by the [framework format reference](../formats.md). +## Inspect A Prompt Before Preparation + +Use [`Engine.InspectPrompt`](../../engine.go) to check one configured prompt's +declared inputs and output workflow without creating placeholder inputs or +resolving a profile: + +```go +inspection, err := engine.InspectPrompt(ctx, "meeting.summary", "") +if err != nil { + return err +} + +for _, input := range inspection.Inputs { + // Compare the declared input with application configuration. +} +``` + +Use this configuration-time boundary when the application needs only the +declared prompt interface. Use `InspectProfile` separately when it must also +check a configured profile. Use `Prepare` when it needs inputs, schemas, or +rendered messages, and use prepared execution when that work must remain tied +to later execution. The method's [GoDoc](../../engine.go) owns exact fields, +hash, ownership, and error semantics. + ## Prepare Without Model Execution [`Engine.Prepare`](../../engine.go) resolves the selected prompt and profile, diff --git a/docs/formats.md b/docs/formats.md index 01fd4d9..8994fdf 100644 --- a/docs/formats.md +++ b/docs/formats.md @@ -58,6 +58,11 @@ When a request omits a version, the selected prompt ID must identify exactly one definition. When it supplies a version, the ID and version pair must be unique. +Exact prompt inspection uses this same configured source, strict decoding, +referenced content-file resolution, and ID/version selection. It reports the +selected definition's declared metadata without changing the prompt format or +executing the definition. + ### Inputs Each `inputs` item has these fields: diff --git a/docs/internal/overview.md b/docs/internal/overview.md index 77e9ffa..b475fe0 100644 --- a/docs/internal/overview.md +++ b/docs/internal/overview.md @@ -11,7 +11,7 @@ contributor workflow and validation. | Component | Implemented responsibility | References | | --- | --- | --- | -| Root `promptkit` package | Provides the supported engine facade, source, backend-registration, and injection options, public request, result, and profile-inspection values, opaque prepared-execution handles, profile construction, extension interfaces, value conversion, redacted formatting, public error mapping, and engine-local assembly. | [Package GoDoc](../../doc.go), [prepared execution](../../prepared_execution.go), [backend API](../../backends.go), [engine assembly](../../engine.go) | +| Root `promptkit` package | Provides the supported engine facade, source, backend-registration, and injection options, public request, result, prompt-inspection, and profile-inspection values, opaque prepared-execution handles, profile construction, extension interfaces, value conversion, redacted formatting, public error mapping, and engine-local assembly. | [Package GoDoc](../../doc.go), [prepared execution](../../prepared_execution.go), [backend API](../../backends.go), [engine assembly](../../engine.go) | | `examples/go-library/prepare` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, and `Prepare`. It is not a public library package. | [Example program](../../examples/go-library/prepare/main.go) | | `examples/go-library/run` | Demonstrates an offline downstream consumer using a prompt file, in-memory profile, inline input, an injected deterministic model client, and `Run`. It is not a public library package. | [Example program](../../examples/go-library/run/main.go) | | `internal/backend` | Constructs each engine's immutable registry from the built-in OpenRouter definition and consumer additions, validates and defensively copies definitions through the shared JSON-value package, and consumes the LLM-owned OpenAI-compatible reserved request-field rule. | [Backend registry](../../internal/backend/registry.go) | @@ -27,7 +27,7 @@ contributor workflow and validation. | `internal/artifact` | Resolves ordinary inline and unrestricted caller-selected file references into copied artifacts with metadata and hashes. | [Internal sources and validation](sources.md) | | `internal/validate` | Validates basic, JSON, and JSON Schema output using operating-system filesystem or `fs.FS` schema sources and creates frozen validation plans for prepared execution. | [Framework formats](../formats.md#schemas), [internal sources and validation](sources.md) | | `internal/llm` | Defines the internal generation boundary and implements outbound OpenAI-compatible chat requests from resolved execution targets, including response decoding, authentication, deadline handling, and ownership of the OpenAI-compatible reserved request-field policy. | [Internal model client](llm.md) | -| `internal/usecase` | Resolves profiles, backends, and targets for exact inspection and request settings for preparation, and coordinates ordinary execution and one-attempt prepared execution across internal sources, rendering, artifact loading, generation, validation, capacity, and optional repair. | [Internal runner](runner.md), [prepared-execution implementation](../../internal/usecase/prepared_execution.go) | +| `internal/usecase` | Resolves prompt definitions and hashes, profiles, backends, and targets for exact inspection and request settings for preparation, and coordinates ordinary execution and one-attempt prepared execution across internal sources, rendering, artifact loading, generation, validation, capacity, and optional repair. | [Internal runner](runner.md), [prepared-execution implementation](../../internal/usecase/prepared_execution.go) | The root package assembles these internal components without exposing their representations. Consumers depend only on the root facade. diff --git a/docs/internal/runner.md b/docs/internal/runner.md index 04d55a2..5257bdf 100644 --- a/docs/internal/runner.md +++ b/docs/internal/runner.md @@ -28,6 +28,19 @@ constructor does not enable one. Each invocation carries its state in request, prepared-run, and result values. The runner has no durable run or session store. +## Shared Prompt Selection + +The runner uses one prompt-selection and hashing boundary for ordinary +preparation and exact prompt inspection. Preparation retains its early +request-ID check before direct-session normalization; both operations then use +the configured prompt repository to select one definition, load referenced +message content, and calculate the same prompt hash. + +Inspection stops after that structural lookup. It does not parse templates or +touch profile, artifact, schema, renderer, validator, admission, or model +collaborators. The root [`Engine.InspectPrompt`](../../engine.go) GoDoc owns +the public operation's exact contract. + ## Shared Profile Selection The runner uses one profile-selection and target-resolution boundary for diff --git a/docs/internal/sources.md b/docs/internal/sources.md index c8fa62c..3773e9b 100644 --- a/docs/internal/sources.md +++ b/docs/internal/sources.md @@ -16,6 +16,11 @@ validation modes, built-in catalog, and source precedence. definitions, selects an ID and optional version, and resolves file-backed message content within the selected operating-system or `fs.FS` source. +Exact prompt inspection performs one point-in-time lookup through that same +repository and validates referenced message content before returning declared +metadata. It does not parse templates or read profile, input, or schema +sources, and it does not retain the definition for a later execution. + Its package tests own prompt selection, strict decoding, definition validation, duplicate detection, and source containment: [prompt-definition repository tests](../../internal/promptdef/repository_test.go). diff --git a/docs/roadmap/future.md b/docs/roadmap/future.md index ba8aece..8bf17ae 100644 --- a/docs/roadmap/future.md +++ b/docs/roadmap/future.md @@ -33,10 +33,6 @@ consumers. ## Ideas -Prompt-definition inspection has been selected for active planning in the -[focused feature roadmap](prompt-inspection.md). The remaining idea is still -available for future selection. - ### Structured capacity errors Add safe structured context to backend admission rejection, as requested by diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 4e22d5d..06f818e 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,6 +1,6 @@ # Prompt-Definition Inspection Implementation Plan -**Status:** Accepted. +**Status:** Complete. ## Purpose diff --git a/docs/roadmap/prompt-inspection.md b/docs/roadmap/prompt-inspection.md index 58dc0a8..9975b4c 100644 --- a/docs/roadmap/prompt-inspection.md +++ b/docs/roadmap/prompt-inspection.md @@ -1,6 +1,6 @@ # Prompt-Definition Inspection -**Status:** Accepted. +**Status:** Complete. ## Purpose diff --git a/docs/roadmap/weatherreporter-promptkit-wishlist.md b/docs/roadmap/weatherreporter-promptkit-wishlist.md index b24d9ec..189e6e4 100644 --- a/docs/roadmap/weatherreporter-promptkit-wishlist.md +++ b/docs/roadmap/weatherreporter-promptkit-wishlist.md @@ -106,8 +106,9 @@ describes the actual execution. ## Priority 2: Prompt-Definition Inspection -**Disposition:** Accepted into the -[prompt-definition inspection](prompt-inspection.md) feature roadmap. +**Disposition:** Implemented as +[`Engine.InspectPrompt`](../../engine.go). See the +[consumer guidance](../consumers/pkg-promptkit.md#inspect-a-prompt-before-preparation). ### Downstream need @@ -123,65 +124,22 @@ response schemas that implement those reports. It needs to validate that the report registry and embedded prompt corpus agree before weather collection or provider execution. -### Current integration option +### Previous integration option -Weatherreporter can maintain synthetic data-package fixtures and call -`Engine.Prepare` for every report prompt during tests. Runtime validation can -also occur through the ordinary per-report preparation stage. +Before prompt inspection was available, Weatherreporter could maintain +synthetic data-package fixtures and call `Engine.Prepare` for every report +prompt during tests. Runtime validation could also occur through the ordinary +per-report preparation stage. -This works, but it requires complete placeholder inputs and profile resolution -when the application primarily wants to inspect prompt identity and declared -contracts. - -### Requested capability - -Add exact prompt-definition lookup without rendering or generation: - -```go -type PromptInfo struct { - PromptID string - PromptVersion string - PromptHash string - DefaultProfileID string - Inputs []InputDefinition - OutputContract OutputContract -} - -func (e *Engine) ResolvePrompt( - ctx context.Context, - promptID string, - promptVersion string, -) (PromptInfo, error) -``` - -The exact returned shape may differ. Weatherreporter needs enough information -to verify prompt existence, version selection, declared inputs, default -profile identity, output format, validation mode, and schema selection without -supplying synthetic prompt input. - -### Design considerations - -- Use ordinary PromptKit prompt-source precedence and exact ID/version - selection. -- Fully load and structurally validate the selected prompt definition. -- Validate referenced prompt content files without rendering their templates. -- Resolve and validate the selected output contract and schema reference where - practical. -- Return an opaque prompt-definition equality value rather than raw source - bytes. -- Do not return rendered messages, schema bodies, profile credentials, or - another source of sensitive content. -- Preserve typed or sentinel errors for missing and invalid prompts. -- Return caller-owned values. -- Enumeration of all known prompts is not required for Weatherreporter; exact - lookup is sufficient. +This required complete placeholder inputs and profile resolution when the +application primarily wanted to inspect prompt identity and declared contracts. ### Value to Weatherreporter -This would let Weatherreporter directly verify that every report prompt -exists, requires the curated `data_package` input, and declares the expected -Markdown or JSON Schema output contract. It would reduce synthetic test setup -and move failures ahead of weather collection. +The implemented interface lets Weatherreporter directly verify that every +report prompt exists, requires the curated `data_package` input, and declares +the expected Markdown or JSON Schema output contract. It reduces synthetic +test setup and moves failures ahead of weather collection. ## Priority 3: Prompt-Independent Profile Inspection @@ -403,11 +361,10 @@ Weatherreporter would be: 1. Add executable preparation handles, ideally sharing implementation with an atomic detailed-run API. -2. Add prompt-definition inspection. -3. Consider eager source validation after evaluating whether the two exact +2. Consider eager source validation after evaluating whether the two exact inspection APIs are sufficient. -4. Add structured generation errors. -5. Add structured capacity errors and semantic execution-target fingerprints +3. Add structured generation errors. +4. Add structured capacity errors and semantic execution-target fingerprints as lower-priority operational improvements. The first item removes the only material integration workaround. Prompt and