promptkit/docs/roadmap/implementation.md

Prompt-Independent Profile Inspection Implementation Plan

Status: Ready for implementation.

Purpose

This document is the decision-complete implementation plan for prompt-independent profile inspection. It is written for a coding agent that will implement each stage in order.

The feature roadmap owns the motivation, consumer workflow, policy choices, compatibility requirements, non-goals, and target end state. This document owns the concrete design, file-level changes, implementation sequence, test ownership, documentation work, validation commands, and completion gates.

Implementation Rules

• Complete the stages in order. Keep the repository compiling and the focused tests passing at every stage boundary.
• Preserve unrelated working-tree changes. In particular, retain the accepted feature roadmap and the corresponding future-roadmap and downstream-wishlist edits that may already be uncommitted.
• Follow every policy under docs/policy/, the task-specific reading guide in docs/development.md, and the accepted behavior in profile-inspection.md.
• Keep the supported API in the root promptkit package. Profile repositories, backend resolution, and target assembly remain below Go's internal/ boundary.
• Reuse the exact profile-source, backend-membership, and execution-target precedence used by Prepare, PrepareExecution, and Run. Do not create a second profile loader, backend registry, or target-merging implementation.
• Preserve the observable behavior and error ordering of existing preparation and execution methods. The new inspection operation must not require a prompt, prompt default profile, request override, credential value, capacity admission, model client, renderer, artifact reader, schema source, or validator.
• Treat structural validation as the existing profile, backend, and effective target invariants. Do not add new endpoint URL policy, provider connectivity checks, reserved-extra-parameter policy, credential syntax, or file-format validation rules as part of this feature.
• Never read, retain, return, format, or log an environment credential value. The result may contain only the effective environment-variable name and the direct-key-required boolean.
• Return caller-owned public values. In particular, nested ExecutionTarget.ExtraParams maps, slices, and objects must not alias engine-owned state or another inspection result.
• Keep tests lean and behavior-focused. Reuse existing profile repository, backend registry, target precedence, and preparation tests rather than duplicating their complete matrices.
• Update exact contracts in GoDoc with the exported declarations. Update current-state consumer, format, and internal documentation only after the corresponding code exists.
• Do not add release notes, change a module version, create a release, or tag a commit as part of this work.

Fixed Design

Public API

Add this root-package value immediately after ExecutionTarget in types.go:

// ProfileInspection is the caller-owned result of Engine.InspectProfile.
// The exact GoDoc is specified below.
type ProfileInspection struct {
	ProfileID            string
	EffectiveModelParams ExecutionTarget
	APIKeyRequired       bool
}

Add this method to Engine in engine.go, immediately before Prepare:

func (e *Engine) InspectProfile(
	ctx context.Context,
	profileID string,
) (*ProfileInspection, error)

Do not accept RunRequest, ExecutionTargetOverride, an API key, an environment override, or options on this method. Exact inspection of one explicit profile ID is the complete operation.

ProfileInspection has no stable JSON contract. Do not add JSON tags, MarshalJSON, UnmarshalJSON, or a custom string representation. Ordinary Go encoding of the exported fields is not prohibited, but consumers must not be promised compatibility for that encoding.

The exact type and method GoDoc must establish:

• surrounding whitespace is trimmed from profileID; the resulting nonblank ID is looked up exactly and case-sensitively;
• the result applies the engine's ordinary in-memory, configured-source, and built-in profile precedence;
• EffectiveModelParams contains framework defaults overlaid by the selected backend and then the selected profile, with no request override;
• EffectiveModelParams.BackendID is empty for endpoint-only profiles;
• EffectiveModelParams.APIKeyEnv is an environment-variable name and never its value;
• APIKeyRequired means a direct request credential is required and is mutually exclusive with a nonblank effective APIKeyEnv;
• the method never derives an ID from a prompt's default_profile;
• the method performs no prompt lookup, rendering, artifact or schema work, backend admission, provider connectivity check, or model generation;
• credential availability is not checked, so an absent or blank named environment variable is not an error;
• the returned value and all nested mutable values are caller-owned;
• filesystem-backed inspection is a point-in-time lookup and does not freeze a profile for a later execution;
• a nil engine matches ErrInvalidConfig;
• a blank ID matches ErrInvalidRequest;
• an absent exact ID matches ErrProfileNotFound and not ErrProfileLoad;
• malformed or unreadable profile data, an unknown backend, or an invalid resolved target matches ErrProfileLoad;
• cancellation observed during profile loading matches ErrProfileLoad while preserving the context error through errors.Is; and
• the method returns no partial result on error.

Update the Engine type GoDoc to include InspectProfile among the operations safe for concurrent calls. Do not imply that injected collaborators become safe when their existing contracts do not provide that guarantee.

Update doc.go in the same stage:

• include Engine.InspectProfile in the package's operation list;
• include inspection in the concurrency and caller-ownership summary; and
• list ProfileInspection among construction and inspection values without a stable JSON representation.

Do not add ProfileInspection to the stable JSON list.

Internal Result

Add this internal value to internal/domain/domain.go near ExecutionProfile and ExecutionTarget:

// ProfileInspection is the resolved result of exact profile inspection.
type ProfileInspection struct {
	ProfileID            string
	EffectiveModelParams ExecutionTarget
	APIKeyRequired       bool
}

Do not add JSON or YAML tags. The internal value is a use-case result, not a file format or persistence contract.

Add a root conversion in convert.go:

func fromDomainProfileInspection(
	inspection *domain.ProfileInspection,
) *ProfileInspection

The conversion must:

• return nil for a nil internal value;
• copy the scalar fields;
• convert the target through the existing fromDomainExecutionTarget; and
• therefore use the existing recursive copyAnyMap boundary for nested extra parameters.

Do not add APIKeyRequired to the public ExecutionTarget. Keeping the requirement on ProfileInspection preserves the existing stable JSON and general execution-target contract.

Shared Profile Selection

Add internal/usecase/profile_inspection.go. Define one private selection value:

type resolvedProfileSelection struct {
	id      string
	profile *domain.ExecutionProfile
	backend *domain.Backend
}

Add a private runner helper:

func (r *Runner) resolveProfileSelection(
	ctx context.Context,
	profileID string,
) (*resolvedProfileSelection, error)

The helper must perform these operations in order:

1. trim surrounding whitespace from profileID;
2. reject a blank result with ErrInvalidRequest;
3. reject a nil runner profile repository with ErrProfileLoad rather than panicking;
4. call the existing profile.Repository.GetProfile exactly once;
5. wrap every repository error with ErrProfileLoad while preserving the underlying error with %w;
6. reject a nil profile returned without an error as ErrProfileLoad;
7. make a value copy of the selected profile before normalization so the runner does not mutate repository-owned state;
8. trim the copied profile's backend ID;
9. when that ID is nonblank, resolve it exactly once through the existing BackendResolver, preserving the current unknown-backend ErrProfileLoad wrapping and backend ID context; and
10. return the normalized exact ID, copied profile, and optional defensive backend value.

Do not inspect all sources, enumerate profiles, expose a source path, fall back after a malformed higher-precedence match, or infer a backend from an endpoint or model.

Refactor the profile-loading and backend-resolution block in Runner.resolvePreparation to call resolveProfileSelection. Leave prompt selection, prompt default-profile selection, prompt hashing, target overrides, request credentials, output-contract resolution, artifact work, and all later ordering where they currently occur.

This refactor must preserve:

• explicit request profile selection over prompt default_profile;
• the existing ErrProfileRequired result when neither exists;
• exact repository and backend lookup behavior;
• existing public not-found versus profile-load classification;
• request override precedence and target-presence tracking;
• credential checking during ordinary preparation; and
• current Prepare, PrepareExecution, Run, and RunPrepared behavior.

Shared Target Resolution And Structural Validation

Continue using the existing resolveExecutionTarget for framework, backend, profile, and optional request precedence. Do not duplicate or move the individual merge rules into the inspection code.

Extract the two existing effective-target requiredness checks from resolvePreparation into a private pure helper:

func validateResolvedExecutionTarget(
	target domain.ExecutionTarget,
) error

It checks only that the trimmed endpoint and model are nonblank and returns a plain descriptive error. It does not validate URL syntax, contact the endpoint, validate credentials, or introduce new parameter rules.

resolvePreparation calls this helper at the same current point: after request override resolution and direct API-key assignment, and before validateAPIKey. It wraps a failure with ErrInvalidRequest, preserving ordinary preparation behavior.

Add the internal inspection method:

func (r *Runner) InspectProfile(
	ctx context.Context,
	profileID string,
) (*domain.ProfileInspection, error)

It performs these operations:

1. trim and reject a blank ID as ErrInvalidRequest;
2. if the supplied context is already canceled, return an error wrapping both ErrProfileLoad and ctx.Err() without touching the repository;
3. call resolveProfileSelection;
4. call resolveExecutionTarget with the selected backend, selected profile, and a nil request override;
5. wrap any target-resolution error with ErrProfileLoad;
6. call validateResolvedExecutionTarget and wrap a failure with ErrProfileLoad;
7. defensively clear target.APIKey;
8. copy target.APIKeyRequired into the result's APIKeyRequired; and
9. return the normalized profile ID and effective target.

Do not call validateAPIKey, os.Getenv, prompt repositories, artifact readers, renderers, validators, capacity admission, output repair, or the model client. Do not populate request target-presence state.

The internal ExecutionTarget may retain its private APIKeyRequired field; the root conversion intentionally omits that private field from the public target and publishes the separate inspection boolean.

Root Facade And Error Mapping

Engine.InspectProfile follows the existing facade pattern:

1. reject a nil engine or nil runner with ErrInvalidConfig;
2. pass the context and string ID directly to Runner.InspectProfile;
3. map internal failures through the existing mapPublicError; and
4. convert a successful result with fromDomainProfileInspection.

No request conversion is needed. Do not add a public sentinel or typed error.

The existing error mapping already has the required ordering:

• an underlying profile.ErrProfileNotFound maps to ErrProfileNotFound before the enclosing use-case ErrProfileLoad is considered;
• other use-case ErrProfileLoad failures map to ErrProfileLoad; and
• usecase.ErrInvalidRequest maps to ErrInvalidRequest.

Do not reorder or otherwise change errors.go unless a focused test proves that the existing mapping does not meet this plan. Preserve underlying repository, backend, and context errors through errors.Is.

Credentials, Ownership, And Concurrency

Inspection has no input through which a direct credential value can enter. Backend and file-profile APIKeyEnv values remain names only. An in-memory profile with APIKeyRequired clears an inherited backend environment name through the existing target merge behavior.

The implementation must succeed for:

• a backend or profile naming an unset environment variable;
• a backend or profile naming an environment variable whose value is blank;
• an in-memory profile requiring a direct key when no key is supplied; and
• a profile requiring no credential.

The result expresses these effective states:

EffectiveModelParams.APIKeyEnv	APIKeyRequired	Meaning
nonblank	false	The profile resolves to the named environment source.
blank	true	A later execution must supply a direct key or explicit request environment override.
blank	false	The resolved target declares no credential requirement.

The implementation must never produce a successful public result with both a nonblank APIKeyEnv and APIKeyRequired == true.

Public ownership is enforced at the root conversion boundary. A consumer may mutate the returned target and arbitrarily nested JSON-compatible extra parameters without affecting:

• the engine registry or profile repository;
• a later InspectProfile call;
• Prepare, prepared execution, or Run; or
• another result already returned to a caller.

No new mutable engine state, cache, global registry, lock, or goroutine is needed. Concurrency safety follows from the existing immutable registry and repository contracts plus per-call values.

Test Ownership

Add focused internal tests in internal/usecase/profile_inspection_test.go. Use small repository and backend fakes already present in the package where practical; do not build a parallel fixture framework.

The internal tests own:

• blank-ID rejection;
• pre-canceled context classification without repository access;
• one exact profile and backend lookup;
• framework-default, backend, and profile target precedence through the existing resolver;
• endpoint-only behavior;
• effective environment-name reporting without availability checks;
• direct-key-required behavior clearing an inherited backend environment name;
• missing-profile and unknown-backend wrapping; and
• nil repository, nil returned profile, and invalid resolved target defenses only if these cases are not already cheaply covered through existing runner fakes.

Keep the internal matrix compact. Profile parser tests continue to own YAML, duplicates, raw-key rejection, ranges, and source discovery. Backend tests continue to own registration validation. Existing target tests continue to own every merge field and numeric override boundary.

Add external-package public contract tests in public_contract_test.go. They own:

• the exported method and result shape through normal Go use;
• a nil engine returning ErrInvalidConfig;
• blank, missing, malformed, and unknown-backend public error identities, including that not-found does not also match ErrProfileLoad;
• a pre-canceled inspection preserving both ErrProfileLoad and the context error without consulting the profile repository;
• successful inspection with an absent credential environment value;
• no prompt dependency, demonstrated with an empty configured prompt fs.FS;
• no model invocation, using the existing deterministic fake client;
• the three credential-requirement result states;
• an endpoint-only profile's empty backend ID;
• deep caller ownership of nested extra parameters across repeated inspections; and
• representative equivalence between InspectProfile.EffectiveModelParams and Prepare.EffectiveModelParams for the same engine, profile, and source state with no request execution override.

Combine closely related assertions into a few readable behavioral tests. Do not add a JSON golden test because ProfileInspection deliberately has no stable JSON contract. Do not duplicate the complete repository-precedence, target-field, parser-error, or credential-execution suites at the root layer.

Existing tests that must continue passing without semantic edits include:

• profile repository and built-in fallback tests;
• backend registry defensive-copy and lookup tests;
• execution-target precedence tests;
• Prepare and Run profile/backend equivalence tests;
• prepared-execution snapshot and credential tests; and
• public error mapping tests.

Documentation Ownership

After the implementation and public tests pass, update current-state documentation:

• docs/consumers/pkg-promptkit.md: add a concise task-oriented section near profile selection showing Engine.InspectProfile, explaining when to use it instead of a synthetic Prepare, how to interpret APIKeyEnv and APIKeyRequired, and that credential enforcement timing remains application policy. Link to the exact GoDoc rather than restating every error and field contract.
• docs/formats.md: update profile selection and backend-membership wording so exact inspection is recognized as another consumer of the existing source and profile precedence. Do not redefine the exported method here.
• docs/internal/runner.md: describe the shared profile-selection boundary and explain that inspection stops after structural target resolution, before credential availability and all prompt-dependent work.
• docs/internal/sources.md: record that exact profile inspection performs one point-in-time profile-source lookup without reading prompt, input, or schema sources, and replace any ambiguous use of “inspection value” for PreparedRun with “preparation value.”
• docs/internal/overview.md: add profile inspection to the existing root facade and internal/usecase responsibility descriptions. Do not add a new component or package row.
• docs/roadmap/future.md: remove the statement that profile inspection is in active planning and leave the remaining unselected ideas intact.
• docs/roadmap/notarius-promptkit-wishlist.md and docs/roadmap/weatherreporter-promptkit-wishlist.md: change the profile inspection disposition from accepted planning to implemented behavior and link to the durable consumer guidance or GoDoc rather than treating the roadmap as current-state documentation.
• docs/roadmap/profile-inspection.md: change its status to Complete only after the code, tests, current-state documentation, and full validation are complete.
• docs/roadmap/implementation.md: change its status to Complete only after every completion gate in this plan is satisfied.

Do not update release guidance in this feature implementation. A later release pass decides whether the change warrants a supplemental release document.

Stage 1: Implement Shared Internal Profile Resolution

Objective

Add the internal inspection result and operation, share profile/backend selection and structural target validation with ordinary preparation, and prove the internal behavior without publishing the root API yet.

Implementation Prompt

Implement only Stage 1 of docs/roadmap/implementation.md. Read the complete feature roadmap and the implementation rules and fixed design above before editing.

1. Add domain.ProfileInspection to internal/domain/domain.go without serialization tags.
2. Add internal/usecase/profile_inspection.go with resolvedProfileSelection, resolveProfileSelection, validateResolvedExecutionTarget, and Runner.InspectProfile exactly as specified.
3. Refactor Runner.resolvePreparation in internal/usecase/runner.go to use the shared selection and target-validation helpers while preserving its current ordering, error classification, target overrides, credentials, and results.
4. Add lean behavioral tests in internal/usecase/profile_inspection_test.go.
5. Run the focused validation below and repair regressions before ending the stage.

Do not add the root public type or method, update current-state documentation, or alter profile formats, backend registration, credential availability, capacity, model-client, or provider behavior in this stage.

Focused Validation

Run:

gofmt -w internal/domain/domain.go \
  internal/usecase/profile_inspection.go \
  internal/usecase/profile_inspection_test.go \
  internal/usecase/runner.go
go test ./internal/usecase ./internal/profile ./internal/profile/builtin \
  ./internal/backend
go test ./internal/usecase -run \
  'TestRunner(InspectProfile|Prepare|Run|PrepareExecution|RunPrepared)'
go vet ./internal/usecase ./internal/profile ./internal/backend

Completion Gate

Stage 1 is complete only when:

• inspection reaches profile and backend resolution without any prompt or execution collaborator;
• profile/backend selection is shared with ordinary preparation;
• target merging and requiredness checks are shared rather than duplicated;
• inspection does not check credential values or capacity;
• internal errors preserve the required identities;
• the ordinary preparation and execution tests still pass without weakened assertions; and
• no root public API or current-state documentation claims the feature yet.

Stage 2: Publish The Root Facade And Public Contract

Objective

Expose the minimal caller-owned inspection API through Engine, preserve public error and JSON compatibility, and protect the consumer-visible contract.

Implementation Prompt

Implement only Stage 2 of docs/roadmap/implementation.md after Stage 1 satisfies its completion gate. Re-read the fixed public API, error, credential, ownership, and test sections before editing.

1. Add ProfileInspection and its exact GoDoc to types.go immediately after ExecutionTarget. Do not add JSON tags or change ExecutionTarget.
2. Add fromDomainProfileInspection to convert.go using fromDomainExecutionTarget.
3. Add Engine.InspectProfile and its exact GoDoc to engine.go immediately before Prepare.
4. Update Engine GoDoc to include concurrent inspection.
5. Update the package GoDoc in doc.go for operation discovery, concurrency, ownership, and the non-stable JSON classification.
6. Add compact external-package contract coverage in public_contract_test.go, using existing fixtures and fakes where they remain clear.
7. Confirm that the existing errors.go mapping meets the plan; do not change it unless a required public identity test fails for a genuine mapping reason.
8. Run the focused validation below and repair regressions before ending the stage.

Do not add enumeration, request overrides, profile fingerprints, JSON stability, prompt inspection, environment credential checks, caching, or current-state prose documentation in this stage.

Focused Validation

Run:

gofmt -w doc.go types.go convert.go engine.go public_contract_test.go
go test .
go test ./internal/usecase
go test . -run 'Test(InspectProfile|.*Profile.*PublicError|.*Profile.*Contract)'
go vet .
go build .

If the repository's actual focused test names differ, use the implemented test names rather than weakening or skipping the intended assertions.

Completion Gate

Stage 2 is complete only when:

• a consumer can call Engine.InspectProfile through the root package;
• the result contains only the normalized ID, caller-owned effective target, and direct-key-required signal;
• no credential value can enter the result;
• environment, direct, and no-credential states are unambiguous;
• nil, blank, missing, malformed, unknown-backend, and cancellation errors have the required public identities;
• not-found remains distinct from profile-load failure;
• the effective target matches ordinary preparation absent request overrides;
• repeated calls and caller mutation cannot alter engine-owned state;
• no stable JSON or new error contract was introduced; and
• existing public preparation, execution, and stable JSON tests remain unchanged and passing.

Stage 3: Update Documentation And Validate The Repository

Objective

Make implemented profile inspection discoverable in its canonical documentation, reconcile temporary roadmap state, and complete full repository validation.

Implementation Prompt

Implement only Stage 3 of docs/roadmap/implementation.md after Stages 1 and 2 satisfy their completion gates.

1. Update docs/consumers/pkg-promptkit.md, docs/formats.md, docs/internal/runner.md, docs/internal/sources.md, and docs/internal/overview.md according to the documentation ownership section above.
2. Update the future catalog and both downstream wishlist dispositions so they no longer describe profile inspection as merely accepted work.
3. Check every changed Markdown link and confirm its file and heading target.
4. Run the full validation sequence below.
5. Only after every check passes, set the feature roadmap and this implementation plan to **Status:** Complete.
6. Re-run git diff --check after the status edits.

Do not add release notes, an example program, a new public package, or a duplicate API reference. Keep detailed contracts in GoDoc and task-oriented usage in the consumer guide.

Full Validation

Run from the repository root:

gofmt -w internal/domain/domain.go \
  internal/usecase/profile_inspection.go \
  internal/usecase/profile_inspection_test.go \
  internal/usecase/runner.go \
  doc.go types.go convert.go engine.go public_contract_test.go
gofmt -l $(git ls-files '*.go')
go test ./...
go test -race ./...
go vet ./...
go build ./...
go run ./examples/go-library/prepare
git diff --check
git status --short

The gofmt -l command must print no paths. The maintained example must remain offline and must not require a real credential or provider.

Inspect the final diff and confirm:

• only files required by this feature and pre-existing user changes are present;
• no credential values, private infrastructure details, workspace files, local replacements, generated binaries, or unrelated formatting changes were added;
• the public declarations and GoDoc own exact API behavior;
• current-state documentation describes only implemented behavior and links to canonical owners;
• roadmap documents contain future scope or completion status rather than a duplicate current API reference; and
• no release, commit, or tag was created.

Completion Gate

The implementation is complete only when:

• every Stage 1 and Stage 2 gate remains satisfied;
• the complete ordinary and race-enabled suites pass;
• vet, build, formatting, the maintained offline example, Markdown links, and whitespace checks pass;
• consumer, format, internal, future, and wishlist documentation are consistent with the implemented boundary;
• both roadmap statuses are Complete;
• the working tree contains no unintended files or changes; and
• the repository is ready for maintainer review without a commit or release having been created by this plan.

Open Questions

None. The accepted feature roadmap and the fixed decisions above fully specify the implementation boundary.