Document profile inspection API

This commit is contained in:
2026-07-30 19:57:07 +00:00
parent 242eace4a7
commit dde48a31fc
10 changed files with 90 additions and 113 deletions

View File

@@ -88,8 +88,9 @@ corresponds atomically to the actual execution.
## Priority 2: Prompt-Independent Profile Inspection
**Disposition:** Covered by the accepted
[prompt-independent profile inspection](profile-inspection.md) roadmap.
**Disposition:** Implemented as
[`Engine.InspectProfile`](../../engine.go). See the
[consumer guidance](../consumers/pkg-promptkit.md#inspect-a-profile-before-prompt-work).
### Downstream need
@@ -104,62 +105,22 @@ needs to determine whether:
This validation should not require model generation.
### Current integration
### Previous integration
Notarius constructs a synthetic prompt using `testing/fstest.MapFS`, supplies a
dummy transcript, and calls `Engine.Prepare` solely to exercise profile and
backend resolution. This works, but prompt preparation is serving as a
substitute for a profile-inspection interface.
### Requested capability
Add a prompt-independent profile-resolution API, for example:
```go
type ResolvedProfile struct {
ProfileID string
BackendID string
EffectiveTarget ExecutionTarget
APIKeyEnv string
}
func (e *Engine) ResolveProfile(
ctx context.Context,
profileID string,
) (ResolvedProfile, error)
```
The returned shape may differ, but it should provide enough information for a
consumer to validate an explicit profile selection without inventing a prompt
or supplying placeholder inputs.
### Design considerations
- Resolve built-in, file-backed, and programmatic profiles using normal
PromptKit precedence.
- Validate that a referenced backend registration exists.
- Do not resolve, retain, or expose credential values.
- Report credential requirements, such as an environment-variable name, so
the consuming application can decide whether availability is required at
configuration-validation time or only at execution time.
- Return caller-owned values.
- Preserve typed or sentinel error classification for missing and invalid
profiles.
- Consider accepting an `ExecutionTargetOverride` if consumers need to inspect
the same effective target that a run-level override would produce.
- Enumeration of all profiles is not required for the Notarius use case; exact
lookup by ID is sufficient.
Before profile inspection was available, Notarius constructed a synthetic
prompt using `testing/fstest.MapFS`, supplied a dummy transcript, and called
`Engine.Prepare` solely to exercise profile and backend resolution.
### Value to Notarius
This would eliminate a synthetic production-only prompt fixture and establish
a direct, supported contract for configuration-time profile and backend
validation.
The implemented interface eliminates a synthetic production-only prompt
fixture and establishes a direct, supported contract for configuration-time
profile and backend validation.
## Priority 3: Semantic Execution-Target Fingerprints
**Disposition:** Deferred until prompt-independent profile inspection defines
the resolved target whose configuration identity would be fingerprinted.
**Disposition:** Deferred pending a separate semantic-equality design for
resolved execution targets.
### Downstream need
@@ -348,11 +309,9 @@ If the PromptKit team chooses to pursue these ideas, the most useful order for
Notarius would be:
1. Add atomic execution that returns prepared details and the completed result.
2. Add prompt-independent profile inspection.
3. Add a semantic execution-target digest, preferably as part of profile
2. Add a semantic execution-target digest, preferably alongside profile
inspection.
4. Add a typed capacity error carrying backend identity.
3. Add a typed capacity error carrying backend identity.
The first two address concrete workarounds in current Notarius code. The third
would improve checkpoint correctness and reduce coupling. The fourth is
operational polish.
The first addresses a concrete execution workaround. The second would improve
checkpoint correctness and reduce coupling. The third is operational polish.