promptkit/docs/roadmap/profile-inspection.md

# Prompt-Independent Profile Inspection

**Status:** Complete.

## Purpose

Allow consumers to look up one execution profile by ID and inspect its
structurally resolved model target without selecting a prompt, supplying
placeholder inputs, checking credential availability, or invoking a model.

This provides a direct configuration-validation boundary for
[Notarius](notarius-promptkit-wishlist.md#priority-2-prompt-independent-profile-inspection)
and
[Weatherreporter](weatherreporter-promptkit-wishlist.md#priority-3-prompt-independent-profile-inspection).

## Motivation

Both downstream consumers need to reject invalid configured profile IDs before
starting application work. They currently have to construct a synthetic prompt
and call `Engine.Prepare` merely to exercise profile loading, backend lookup,
and execution-target resolution.

That workaround couples profile validation to unrelated prompt definitions,
fixture inputs, rendering, schema behavior, and current credential
availability. Promptkit already owns profile precedence, backend membership,
and target resolution, so it should expose that cohesive capability directly.

## Consumer Workflow

The target public workflow is:

```go
inspection, err := engine.InspectProfile(ctx, profileID)
if err != nil {
	// Reject or report the configured profile.
	return
}

target := inspection.EffectiveModelParams
if target.APIKeyEnv != "" {
	// Apply application policy for the named environment variable.
}
```

The target public surface is:

```go
type ProfileInspection struct {
	ProfileID           string
	EffectiveModelParams ExecutionTarget
	APIKeyRequired      bool
}

func (e *Engine) InspectProfile(
	ctx context.Context,
	profileID string,
) (*ProfileInspection, error)
```

The declarations and GoDoc will own the exact implemented contract. The
important public shape is exact lookup through the existing engine, one
caller-owned inspection value, the resolved `ExecutionTarget`, and an explicit
signal for a direct API-key requirement.

`EffectiveModelParams.BackendID` identifies the selected registered backend
and remains empty for endpoint-only profiles.
`EffectiveModelParams.APIKeyEnv` reports the effective credential environment
variable name. `APIKeyRequired` reports that the profile requires a direct
request credential instead. These states are mutually exclusive after normal
profile and backend precedence is applied.

`ProfileInspection` does not need a stable JSON representation. Consumers that
persist application configuration or diagnostics can select the fields their
own format requires.

## Lookup And Precedence

`InspectProfile` requires a non-blank explicit profile ID. It trims surrounding
whitespace and otherwise performs the same case-sensitive exact lookup used by
ordinary execution.

Lookup applies the engine's normal profile-source precedence:

- programmatic profiles supplied through `WithProfiles`;
- the configured file, directory, or `fs.FS` profile source; and
- the built-in profile catalog.

A valid higher-precedence match shadows a lower-precedence profile with the
same ID. A malformed or unreadable higher-precedence match fails rather than
silently falling back. The
[profile format reference](../formats.md#profile-definitions) remains the
canonical owner of profile-source and file-format behavior.

Inspection never derives a profile ID from a prompt's `default_profile`; the
caller is inspecting one explicitly named profile.

## Structural Resolution

Inspection loads and validates the selected profile, verifies that a named
backend exists in the engine's immutable backend registry, and applies normal
framework-default, backend, and profile precedence to produce the effective
target.

For the same engine state and profile ID, with no per-run execution override,
the inspected target must match the target that ordinary preparation would
resolve before applying request credentials and checking their availability.
This equivalence must use one shared resolution path rather than a second set
of precedence rules.

Structural resolution includes:

- backend routing identity;
- endpoint and model;
- sampling, token, timeout, service-tier, and reasoning settings;
- the effective credential environment-variable name or direct-key
  requirement; and
- deeply copied provider-specific extra parameters.

Inspection returns the effective target rather than a raw profile definition.
This keeps framework and backend defaults visible to consumers without
creating a second public profile-loading interface.

The result does not include request-override presence because no
`ExecutionTargetOverride` participates in inspection.

## Credentials And Sensitive Data

Inspection reports credential requirements but never resolves, retains, or
returns a credential value.

The operation does not read the named environment variable and succeeds when
that variable is absent or blank. It accepts neither a direct API key nor an
API-key environment override. Consumers decide whether credential availability
must be enforced during application configuration, while `Prepare`,
`PrepareExecution`, `Run`, and `RunPrepared` retain their execution-time
credential contracts.

Error messages, formatting, and returned values must not expose environment
values or other resolved secrets. Existing restrictions against raw API keys
in profile sources remain unchanged.

## Ownership, Consistency, And Concurrency

Each successful call returns a caller-owned snapshot. Mutating the returned
target or any nested extra-parameter map or slice cannot affect the engine,
later inspection, or later execution.

Inspection is safe to call concurrently under the engine's existing immutable
registry and repository contracts. It does not mutate profile sources or
cache a result globally.

For filesystem-backed sources, an inspection describes the state observed by
that call. It does not freeze the profile for a later `Run`; a source may
change between operations. Consumers requiring an exact preflight-to-execution
snapshot should use the existing prepared-execution workflow.

## Errors And Cancellation

The operation uses existing public error categories:

- a blank profile ID matches `ErrInvalidRequest`;
- an absent exact ID matches `ErrProfileNotFound` and not `ErrProfileLoad`;
- read, decode, validation, and source-selection failures match
  `ErrProfileLoad`; and
- an unknown referenced backend or an invalid structurally resolved target
  matches `ErrProfileLoad`.

Errors should preserve useful underlying collaborator and context identities
through `errors.Is` where the existing facade does so, without exposing
internal package types. Context cancellation governs inspection and no partial
inspection result is returned on failure.

Missing credential values are not inspection errors. The operation cannot
return capacity or model-generation failures because it performs neither
backend admission nor generation.

## Compatibility And Boundaries

This feature is additive. Existing profile formats, source precedence,
backend registration, `Prepare`, prepared execution, and `Run` behavior remain
unchanged.

The method belongs on the root `Engine` facade. Profile repositories and the
backend registry remain internal implementation details, and no new public
repository interface is introduced.

Inspection does not require prompt lookup, rendering, artifact loading, schema
loading, validation, backend-capacity admission, or model-client access.
Engine construction retains its ordinary configuration requirements; this
feature does not introduce a separate profile-only engine.

## Documentation

The completed documentation set has these ownership boundaries:

- exported declarations and GoDoc own the exact method, result, ownership,
  credential, error, and cancellation contracts;
- the promptkit consumer guide explains configuration-time profile inspection
  and distinguishes it from `Prepare` and prepared execution;
- the profile format reference continues to own profile fields and source
  precedence; and
- internal documentation describes shared profile and target resolution
  without duplicating public contracts.

## Non-Goals

This work does not include:

- enumerating or searching profiles;
- returning raw profile definitions or profile source paths;
- accepting per-run execution overrides, direct API keys, or API-key
  environment overrides;
- checking environment-variable contents or other credential availability;
- semantic execution-target fingerprints or profile hashes;
- prompt-definition inspection or prompt default-profile resolution;
- full-corpus validation across every profile source;
- freezing a filesystem-backed profile for later execution;
- exposing backend concurrency limits, queue state, or capacity policy;
- model generation, provider health checks, or endpoint connectivity tests;
- dynamic backend or profile registration after engine construction; or
- changing current profile, backend, prepared-execution, or stable JSON
  contracts.

## Target End State

After this work:

- consumers can validate one configured profile without inventing a prompt or
  placeholder inputs;
- lookup observes ordinary programmatic, configured-source, and built-in
  precedence;
- a successful result proves that the profile exists, is valid, references a
  registered backend when applicable, and resolves to a structurally valid
  effective target;
- the inspected target matches ordinary preparation for the same profile and
  engine state before per-run overrides and credential availability checks;
- credential requirements are visible without reading or exposing credential
  values;
- returned targets and nested data are caller-owned;
- inspection performs no rendering, source loading unrelated to the profile,
  capacity admission, or model work;
- existing execution workflows and compatibility contracts remain unchanged;
  and
- Promptkit owns reusable profile validation while downstream applications
  retain configuration policy, persistence, logging, and credential-timing
  decisions.