Document profile inspection API
This commit is contained in:
@@ -88,8 +88,9 @@ corresponds atomically to the actual execution.
|
||||
|
||||
## Priority 2: Prompt-Independent Profile Inspection
|
||||
|
||||
**Disposition:** Covered by the accepted
|
||||
[prompt-independent profile inspection](profile-inspection.md) roadmap.
|
||||
**Disposition:** Implemented as
|
||||
[`Engine.InspectProfile`](../../engine.go). See the
|
||||
[consumer guidance](../consumers/pkg-promptkit.md#inspect-a-profile-before-prompt-work).
|
||||
|
||||
### Downstream need
|
||||
|
||||
@@ -104,62 +105,22 @@ needs to determine whether:
|
||||
|
||||
This validation should not require model generation.
|
||||
|
||||
### Current integration
|
||||
### Previous integration
|
||||
|
||||
Notarius constructs a synthetic prompt using `testing/fstest.MapFS`, supplies a
|
||||
dummy transcript, and calls `Engine.Prepare` solely to exercise profile and
|
||||
backend resolution. This works, but prompt preparation is serving as a
|
||||
substitute for a profile-inspection interface.
|
||||
|
||||
### Requested capability
|
||||
|
||||
Add a prompt-independent profile-resolution API, for example:
|
||||
|
||||
```go
|
||||
type ResolvedProfile struct {
|
||||
ProfileID string
|
||||
BackendID string
|
||||
EffectiveTarget ExecutionTarget
|
||||
APIKeyEnv string
|
||||
}
|
||||
|
||||
func (e *Engine) ResolveProfile(
|
||||
ctx context.Context,
|
||||
profileID string,
|
||||
) (ResolvedProfile, error)
|
||||
```
|
||||
|
||||
The returned shape may differ, but it should provide enough information for a
|
||||
consumer to validate an explicit profile selection without inventing a prompt
|
||||
or supplying placeholder inputs.
|
||||
|
||||
### Design considerations
|
||||
|
||||
- Resolve built-in, file-backed, and programmatic profiles using normal
|
||||
PromptKit precedence.
|
||||
- Validate that a referenced backend registration exists.
|
||||
- Do not resolve, retain, or expose credential values.
|
||||
- Report credential requirements, such as an environment-variable name, so
|
||||
the consuming application can decide whether availability is required at
|
||||
configuration-validation time or only at execution time.
|
||||
- Return caller-owned values.
|
||||
- Preserve typed or sentinel error classification for missing and invalid
|
||||
profiles.
|
||||
- Consider accepting an `ExecutionTargetOverride` if consumers need to inspect
|
||||
the same effective target that a run-level override would produce.
|
||||
- Enumeration of all profiles is not required for the Notarius use case; exact
|
||||
lookup by ID is sufficient.
|
||||
Before profile inspection was available, Notarius constructed a synthetic
|
||||
prompt using `testing/fstest.MapFS`, supplied a dummy transcript, and called
|
||||
`Engine.Prepare` solely to exercise profile and backend resolution.
|
||||
|
||||
### Value to Notarius
|
||||
|
||||
This would eliminate a synthetic production-only prompt fixture and establish
|
||||
a direct, supported contract for configuration-time profile and backend
|
||||
validation.
|
||||
The implemented interface eliminates a synthetic production-only prompt
|
||||
fixture and establishes a direct, supported contract for configuration-time
|
||||
profile and backend validation.
|
||||
|
||||
## Priority 3: Semantic Execution-Target Fingerprints
|
||||
|
||||
**Disposition:** Deferred until prompt-independent profile inspection defines
|
||||
the resolved target whose configuration identity would be fingerprinted.
|
||||
**Disposition:** Deferred pending a separate semantic-equality design for
|
||||
resolved execution targets.
|
||||
|
||||
### Downstream need
|
||||
|
||||
@@ -348,11 +309,9 @@ If the PromptKit team chooses to pursue these ideas, the most useful order for
|
||||
Notarius would be:
|
||||
|
||||
1. Add atomic execution that returns prepared details and the completed result.
|
||||
2. Add prompt-independent profile inspection.
|
||||
3. Add a semantic execution-target digest, preferably as part of profile
|
||||
2. Add a semantic execution-target digest, preferably alongside profile
|
||||
inspection.
|
||||
4. Add a typed capacity error carrying backend identity.
|
||||
3. Add a typed capacity error carrying backend identity.
|
||||
|
||||
The first two address concrete workarounds in current Notarius code. The third
|
||||
would improve checkpoint correctness and reduce coupling. The fourth is
|
||||
operational polish.
|
||||
The first addresses a concrete execution workaround. The second would improve
|
||||
checkpoint correctness and reduce coupling. The third is operational polish.
|
||||
|
||||
Reference in New Issue
Block a user