Document prompt inspection API
This commit is contained in:
@@ -106,8 +106,9 @@ describes the actual execution.
|
||||
|
||||
## Priority 2: Prompt-Definition Inspection
|
||||
|
||||
**Disposition:** Accepted into the
|
||||
[prompt-definition inspection](prompt-inspection.md) feature roadmap.
|
||||
**Disposition:** Implemented as
|
||||
[`Engine.InspectPrompt`](../../engine.go). See the
|
||||
[consumer guidance](../consumers/pkg-promptkit.md#inspect-a-prompt-before-preparation).
|
||||
|
||||
### Downstream need
|
||||
|
||||
@@ -123,65 +124,22 @@ response schemas that implement those reports. It needs to validate that the
|
||||
report registry and embedded prompt corpus agree before weather collection or
|
||||
provider execution.
|
||||
|
||||
### Current integration option
|
||||
### Previous integration option
|
||||
|
||||
Weatherreporter can maintain synthetic data-package fixtures and call
|
||||
`Engine.Prepare` for every report prompt during tests. Runtime validation can
|
||||
also occur through the ordinary per-report preparation stage.
|
||||
Before prompt inspection was available, Weatherreporter could maintain
|
||||
synthetic data-package fixtures and call `Engine.Prepare` for every report
|
||||
prompt during tests. Runtime validation could also occur through the ordinary
|
||||
per-report preparation stage.
|
||||
|
||||
This works, but it requires complete placeholder inputs and profile resolution
|
||||
when the application primarily wants to inspect prompt identity and declared
|
||||
contracts.
|
||||
|
||||
### Requested capability
|
||||
|
||||
Add exact prompt-definition lookup without rendering or generation:
|
||||
|
||||
```go
|
||||
type PromptInfo struct {
|
||||
PromptID string
|
||||
PromptVersion string
|
||||
PromptHash string
|
||||
DefaultProfileID string
|
||||
Inputs []InputDefinition
|
||||
OutputContract OutputContract
|
||||
}
|
||||
|
||||
func (e *Engine) ResolvePrompt(
|
||||
ctx context.Context,
|
||||
promptID string,
|
||||
promptVersion string,
|
||||
) (PromptInfo, error)
|
||||
```
|
||||
|
||||
The exact returned shape may differ. Weatherreporter needs enough information
|
||||
to verify prompt existence, version selection, declared inputs, default
|
||||
profile identity, output format, validation mode, and schema selection without
|
||||
supplying synthetic prompt input.
|
||||
|
||||
### Design considerations
|
||||
|
||||
- Use ordinary PromptKit prompt-source precedence and exact ID/version
|
||||
selection.
|
||||
- Fully load and structurally validate the selected prompt definition.
|
||||
- Validate referenced prompt content files without rendering their templates.
|
||||
- Resolve and validate the selected output contract and schema reference where
|
||||
practical.
|
||||
- Return an opaque prompt-definition equality value rather than raw source
|
||||
bytes.
|
||||
- Do not return rendered messages, schema bodies, profile credentials, or
|
||||
another source of sensitive content.
|
||||
- Preserve typed or sentinel errors for missing and invalid prompts.
|
||||
- Return caller-owned values.
|
||||
- Enumeration of all known prompts is not required for Weatherreporter; exact
|
||||
lookup is sufficient.
|
||||
This required complete placeholder inputs and profile resolution when the
|
||||
application primarily wanted to inspect prompt identity and declared contracts.
|
||||
|
||||
### Value to Weatherreporter
|
||||
|
||||
This would let Weatherreporter directly verify that every report prompt
|
||||
exists, requires the curated `data_package` input, and declares the expected
|
||||
Markdown or JSON Schema output contract. It would reduce synthetic test setup
|
||||
and move failures ahead of weather collection.
|
||||
The implemented interface lets Weatherreporter directly verify that every
|
||||
report prompt exists, requires the curated `data_package` input, and declares
|
||||
the expected Markdown or JSON Schema output contract. It reduces synthetic
|
||||
test setup and moves failures ahead of weather collection.
|
||||
|
||||
## Priority 3: Prompt-Independent Profile Inspection
|
||||
|
||||
@@ -403,11 +361,10 @@ Weatherreporter would be:
|
||||
|
||||
1. Add executable preparation handles, ideally sharing implementation with an
|
||||
atomic detailed-run API.
|
||||
2. Add prompt-definition inspection.
|
||||
3. Consider eager source validation after evaluating whether the two exact
|
||||
2. Consider eager source validation after evaluating whether the two exact
|
||||
inspection APIs are sufficient.
|
||||
4. Add structured generation errors.
|
||||
5. Add structured capacity errors and semantic execution-target fingerprints
|
||||
3. Add structured generation errors.
|
||||
4. Add structured capacity errors and semantic execution-target fingerprints
|
||||
as lower-priority operational improvements.
|
||||
|
||||
The first item removes the only material integration workaround. Prompt and
|
||||
|
||||
Reference in New Issue
Block a user