|
|
|
|
@@ -106,8 +106,9 @@ describes the actual execution.
|
|
|
|
|
|
|
|
|
|
## Priority 2: Prompt-Definition Inspection
|
|
|
|
|
|
|
|
|
|
**Disposition:** Accepted into the
|
|
|
|
|
[prompt-definition inspection](prompt-inspection.md) feature roadmap.
|
|
|
|
|
**Disposition:** Implemented as
|
|
|
|
|
[`Engine.InspectPrompt`](../../engine.go). See the
|
|
|
|
|
[consumer guidance](../consumers/pkg-promptkit.md#inspect-a-prompt-before-preparation).
|
|
|
|
|
|
|
|
|
|
### Downstream need
|
|
|
|
|
|
|
|
|
|
@@ -123,65 +124,22 @@ response schemas that implement those reports. It needs to validate that the
|
|
|
|
|
report registry and embedded prompt corpus agree before weather collection or
|
|
|
|
|
provider execution.
|
|
|
|
|
|
|
|
|
|
### Current integration option
|
|
|
|
|
### Previous integration option
|
|
|
|
|
|
|
|
|
|
Weatherreporter can maintain synthetic data-package fixtures and call
|
|
|
|
|
`Engine.Prepare` for every report prompt during tests. Runtime validation can
|
|
|
|
|
also occur through the ordinary per-report preparation stage.
|
|
|
|
|
Before prompt inspection was available, Weatherreporter could maintain
|
|
|
|
|
synthetic data-package fixtures and call `Engine.Prepare` for every report
|
|
|
|
|
prompt during tests. Runtime validation could also occur through the ordinary
|
|
|
|
|
per-report preparation stage.
|
|
|
|
|
|
|
|
|
|
This works, but it requires complete placeholder inputs and profile resolution
|
|
|
|
|
when the application primarily wants to inspect prompt identity and declared
|
|
|
|
|
contracts.
|
|
|
|
|
|
|
|
|
|
### Requested capability
|
|
|
|
|
|
|
|
|
|
Add exact prompt-definition lookup without rendering or generation:
|
|
|
|
|
|
|
|
|
|
```go
|
|
|
|
|
type PromptInfo struct {
|
|
|
|
|
PromptID string
|
|
|
|
|
PromptVersion string
|
|
|
|
|
PromptHash string
|
|
|
|
|
DefaultProfileID string
|
|
|
|
|
Inputs []InputDefinition
|
|
|
|
|
OutputContract OutputContract
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
func (e *Engine) ResolvePrompt(
|
|
|
|
|
ctx context.Context,
|
|
|
|
|
promptID string,
|
|
|
|
|
promptVersion string,
|
|
|
|
|
) (PromptInfo, error)
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
The exact returned shape may differ. Weatherreporter needs enough information
|
|
|
|
|
to verify prompt existence, version selection, declared inputs, default
|
|
|
|
|
profile identity, output format, validation mode, and schema selection without
|
|
|
|
|
supplying synthetic prompt input.
|
|
|
|
|
|
|
|
|
|
### Design considerations
|
|
|
|
|
|
|
|
|
|
- Use ordinary PromptKit prompt-source precedence and exact ID/version
|
|
|
|
|
selection.
|
|
|
|
|
- Fully load and structurally validate the selected prompt definition.
|
|
|
|
|
- Validate referenced prompt content files without rendering their templates.
|
|
|
|
|
- Resolve and validate the selected output contract and schema reference where
|
|
|
|
|
practical.
|
|
|
|
|
- Return an opaque prompt-definition equality value rather than raw source
|
|
|
|
|
bytes.
|
|
|
|
|
- Do not return rendered messages, schema bodies, profile credentials, or
|
|
|
|
|
another source of sensitive content.
|
|
|
|
|
- Preserve typed or sentinel errors for missing and invalid prompts.
|
|
|
|
|
- Return caller-owned values.
|
|
|
|
|
- Enumeration of all known prompts is not required for Weatherreporter; exact
|
|
|
|
|
lookup is sufficient.
|
|
|
|
|
This required complete placeholder inputs and profile resolution when the
|
|
|
|
|
application primarily wanted to inspect prompt identity and declared contracts.
|
|
|
|
|
|
|
|
|
|
### Value to Weatherreporter
|
|
|
|
|
|
|
|
|
|
This would let Weatherreporter directly verify that every report prompt
|
|
|
|
|
exists, requires the curated `data_package` input, and declares the expected
|
|
|
|
|
Markdown or JSON Schema output contract. It would reduce synthetic test setup
|
|
|
|
|
and move failures ahead of weather collection.
|
|
|
|
|
The implemented interface lets Weatherreporter directly verify that every
|
|
|
|
|
report prompt exists, requires the curated `data_package` input, and declares
|
|
|
|
|
the expected Markdown or JSON Schema output contract. It reduces synthetic
|
|
|
|
|
test setup and moves failures ahead of weather collection.
|
|
|
|
|
|
|
|
|
|
## Priority 3: Prompt-Independent Profile Inspection
|
|
|
|
|
|
|
|
|
|
@@ -403,11 +361,10 @@ Weatherreporter would be:
|
|
|
|
|
|
|
|
|
|
1. Add executable preparation handles, ideally sharing implementation with an
|
|
|
|
|
atomic detailed-run API.
|
|
|
|
|
2. Add prompt-definition inspection.
|
|
|
|
|
3. Consider eager source validation after evaluating whether the two exact
|
|
|
|
|
2. Consider eager source validation after evaluating whether the two exact
|
|
|
|
|
inspection APIs are sufficient.
|
|
|
|
|
4. Add structured generation errors.
|
|
|
|
|
5. Add structured capacity errors and semantic execution-target fingerprints
|
|
|
|
|
3. Add structured generation errors.
|
|
|
|
|
4. Add structured capacity errors and semantic execution-target fingerprints
|
|
|
|
|
as lower-priority operational improvements.
|
|
|
|
|
|
|
|
|
|
The first item removes the only material integration workaround. Prompt and
|
|
|
|
|
|
