promptkit/docs/roadmap/implementation.md

Executable Preparation Handles Implementation Plan

Status: Complete.

Purpose

This document is the decision-complete implementation plan for executable preparation handles. It is written for a coding agent that will implement each stage in order.

The feature roadmap owns the motivation, consumer workflow, policy choices, compatibility requirements, non-goals, and target end state. This document owns the concrete design, file-level changes, implementation sequence, test ownership, documentation work, validation commands, and completion gates.

Implementation Rules

• Complete the stages in order. Keep the repository compiling and the focused tests passing at every stage boundary.
• Preserve unrelated working-tree changes. In particular, retain the accepted roadmap and the downstream wishlist and future-roadmap edits that may already be present.
• Follow every policy under docs/policy/, the task-specific reading guide in docs/development.md, and the accepted behavior in prepared-execution.md.
• Keep the supported API in the root promptkit package. Internal lifecycle, validation, and execution state must remain below Go's internal/ boundary.
• Preserve the observable behavior and ordering of existing Engine.Prepare and Engine.Run. Do not implement the new workflow by redefining either existing method in terms of the new public methods.
• Reuse preparation and post-preparation execution mechanics where their ordering is genuinely shared. Preserve Run's existing early admission boundary while ensuring PrepareExecution performs complete preparation without admission.
• Do not add durable serialization, retries, provider failover, background workers, a task queue, capacity reservation during preparation, or consumer-specific persistence and redaction policy.
• Keep direct API-key values in private execution state only. Every public details value, formatted handle, JSON value, result, hash, and error must remain credential-free.
• Keep tests lean but give durable coverage to the public compatibility, security, concurrency, one-attempt, source-snapshot, cancellation, and capacity invariants. Use real internal collaborators where they are fast and deterministic and fakes at model-generation and synchronization boundaries.
• Update exact GoDoc with exported declarations. Update current-state consumer and internal documentation only after the corresponding behavior exists.
• Do not add release notes, change a module version, create a release, or tag a commit as part of this work.

Fixed Design

Public API

Add an opaque root-package type and these methods:

type PreparedExecution struct {
	// Unexported Promptkit-owned state only.
}

func (e *Engine) PrepareExecution(
	ctx context.Context,
	req RunRequest,
) (*PreparedExecution, error)

func (p *PreparedExecution) Details() PreparedRun

func (p *PreparedExecution) Discard()

func (e *Engine) RunPrepared(
	ctx context.Context,
	prepared *PreparedExecution,
) (*RunResult, error)

Place the public handle and its handle-local methods in a new root file named prepared_execution.go. Keep the two engine operations in engine.go beside Prepare and Run so all engine workflows remain discoverable together.

PreparedExecution must contain no exported fields. It wraps one pointer to Promptkit-owned internal state; copying the public struct therefore preserves one shared lifecycle rather than creating another execution opportunity.

Implement both:

func (p *PreparedExecution) String() string
func (p *PreparedExecution) GoString() string

with a constant credential- and content-free representation such as promptkit.PreparedExecution{opaque}. These methods must not branch on, inspect, or format retained execution state. Do not add a custom JSON representation: with no exported fields, ordinary JSON encoding exposes no state, and JSON is explicitly outside the handle contract.

The exact GoDoc must own these behaviors:

• PrepareExecution performs complete preparation but no model call and no backend-capacity admission;
• the handle is bound to the creating engine and permits one RunPrepared invocation;
• Details returns a fresh caller-owned, credential-redacted copy on every call and remains usable after discard or execution;
• Discard is nil-safe and idempotent, invalidates an unclaimed handle, and is not an execution-cancellation mechanism;
• RunPrepared atomically consumes the one attempt before credential revalidation, admission, generation, or validation;
• lifecycle misuse matches ErrInvalidRequest, while a nil engine matches ErrInvalidConfig;
• an operational execution error returns no partial RunResult;
• the RunPrepared context does not inherit from the preparation context and governs the execution attempt; and
• handles are opaque in-process values, not serializable or restartable jobs.

For a nil or zero-value handle, Details returns a zero PreparedRun and Discard does nothing. A nil, zero-value, foreign-engine, discarded, claimed, or used handle passed to RunPrepared returns an error matching ErrInvalidRequest.

Internal Validation Snapshot

Extend internal/validate/validator.go with two internal interfaces:

// PreparedValidation validates artifacts against one frozen output contract.
type PreparedValidation interface {
	Validate(
		ctx context.Context,
		artifact *domain.Artifact,
	) (domain.ValidationResult, error)

	// SchemaDocument returns the root JSON Schema document used for provider
	// structured output, or nil for non-schema modes. The returned value is
	// internal immutable state and must not be mutated by callers.
	SchemaDocument() any
}

// ValidationPreparer freezes the validation resources for one output
// contract.
type ValidationPreparer interface {
	PrepareValidation(
		ctx context.Context,
		contract domain.OutputContract,
	) (PreparedValidation, error)
}

The exact names may be adjusted only to avoid an actual Go naming conflict; the separation of responsibilities and method shapes are fixed. validate.Validator and its existing Validate method remain unchanged so the existing Run behavior and internal test fakes do not acquire a new mandatory method.

StandardValidator and FSValidator implement ValidationPreparer. Their prepared-validation implementation:

• captures a value copy of the complete OutputContract;
• handles none, basic, and JSON validation without any source access;
• for JSON Schema, loads the root document, validates its dialect, registers it with a new compiler, compiles it during PrepareValidation, and retains the compiled schema;
• allows the compiler to resolve every transitive reference during that compilation using the existing contained filesystem or fs.FS loader;
• returns the same loaded root document through SchemaDocument, so the provider-facing structured-output constraint and later validation derive from one root snapshot; and
• validates generated instances only against retained in-memory state. Its Validate method must not reopen a path, call fs.ReadFile, resolve a symlink, or invoke a schema loader after preparation succeeds.

Reuse the existing JSON parsing, validation-result construction, schema dialect checks, contained-path rules, compiler configuration, and validation error wording. A prepared JSON Schema compilation or reference-loading failure is a preparation-time validation error. Existing direct calls to Validator.Validate, including the existing Run path, retain their current lazy compilation and source-loading behavior.

Do not expose the JSON Schema library's compiled type outside internal/validate. Do not place compiled schemas or validation interfaces in internal/domain.

Internal Prepared-Execution State

Add internal/usecase/prepared_execution.go. Define an internal PreparedExecution owned by one Runner. It contains:

• the immutable owner *Runner;
• a mutex;
• a small private lifecycle state with at least ready, claimed, and discarded states;
• a credential-redacted prepared-details snapshot that survives every lifecycle transition; and
• while ready, a pointer to a private payload containing the full domain.PreparedRun and its validate.PreparedValidation.

The payload is the only retained object containing RunRequest.APIKey. Construct the public-details snapshot as a deep copy of the prepared run and clear its EffectiveModelParams.APIKey. Deep-copy every mutable nested value, including messages, maps, target extra parameters, and the structured-output schema. Reuse the repository's JSON-compatible copy helpers rather than maintaining an ad hoc reflection copier.

Add these internal operations:

func (r *Runner) PrepareExecution(
	ctx context.Context,
	req domain.RunRequest,
) (*PreparedExecution, error)

func (p *PreparedExecution) Details() *domain.PreparedRun

func (p *PreparedExecution) Discard()

func (r *Runner) RunPrepared(
	ctx context.Context,
	prepared *PreparedExecution,
) (*domain.RunResult, error)

Internal exported names are acceptable here because the root facade must call them, but they remain inaccessible to downstream consumers through Go's internal rule.

PrepareExecution performs these operations in order:

1. call the existing request-copy boundary in the root facade before entering the runner;
2. run resolvePreparation, including the existing environment credential availability check;
3. prepare a frozen validation plan through validate.ValidationPreparer;
4. construct provider structured-output metadata from the plan's root schema document for JSON Schema mode;
5. load and hash artifacts, render the session and messages, and assemble the complete domain.PreparedRun; and
6. construct the ready internal handle with independent full and redacted snapshots.

Extract the artifact-loading, rendering, hashing, timing, and domain.PreparedRun assembly portion of completePreparation into a helper that accepts an already-resolved *domain.StructuredOutputSpec. completePreparation must continue to call the existing resolveStructuredOutput first and then use that helper. This preserves existing Prepare and Run schema behavior. PrepareExecution uses the frozen validation plan's schema document to build the same structured-output shape without calling resolveStructuredOutput.

When the runner has no validator, preserve the existing runner semantics: validation is skipped and a no-op prepared plan is sufficient. When it has a non-nil validator that does not implement ValidationPreparer, PrepareExecution fails with ErrValidation; do not fall back to a plan that would reopen sources later. The root engine's standard and fs.FS validators always implement the new interface.

Claim, Cleanup, And Engine Binding

RunPrepared checks engine ownership before attempting a claim. A foreign runner returns ErrInvalidRequest without changing the handle. The owning runner then locks the state and atomically:

1. accepts only the ready state;
2. changes it permanently to claimed;
3. detaches the private payload from the handle; and
4. unlocks before doing any blocking or collaborator work.

Every valid owning-engine invocation consumes the attempt at this point. Run ID creation, credential failure, pre-canceled context, capacity rejection, generation failure, operational validation failure, and success all leave the handle claimed and unusable.

The execution call holds the detached payload locally and defers cleanup on every exit. Cleanup must at minimum:

• overwrite the direct EffectiveModelParams.APIKey field with an empty string;
• drop the local prepared-run pointer;
• drop the frozen-validation reference; and
• leave the separate redacted details snapshot intact.

This is reference cleanup, not a promise of secure Go string-memory erasure.

Discard locks the same state. If ready, it changes the state to discarded, detaches the payload, unlocks, and performs the same cleanup. If claimed or already discarded, it is a no-op. Therefore a RunPrepared/Discard race has one winner: discard prevents a claim only if it acquires the ready state first; otherwise it does not cancel the running attempt.

Details locks only long enough to read the retained redacted snapshot and returns a fresh deep copy. It must not return a pointer, slice, map, schema, or extra-parameter tree shared with either the handle or another Details call.

Prepared Execution Mechanics

Refactor the post-preparation portion of Runner.Run into a private execution helper that accepts:

• a fully prepared domain.PreparedRun;
• a caller-supplied run ID and execution start time; and
• a validation function that accepts the artifact and repair-attempt count.

The helper owns model generation, artifact construction, content validation, internal repair, final result assembly, and result end timing. The live Runner.Run path supplies its existing validateOutput function. The prepared path supplies a wrapper around the retained validate.PreparedValidation. Both wrappers set the actual repair-attempt count on the returned validation result.

Keep admission outside this shared helper:

• Runner.Run retains its current sequence of run-ID creation, start time, target resolution, early admission, complete preparation, execution, and release.
• Runner.RunPrepared claims the handle, creates the run ID and execution start time, revalidates credential availability, admits the frozen backend, executes the shared helper, and releases admission.

Extract a small private admission helper only if it preserves the exact existing capacity wrapping and release behavior. The limited-backend lease must cover generation, validation, and all internal repair calls and must be released on every return.

Before admission, RunPrepared calls the existing validateAPIKey with the frozen environment-variable name, retained direct key, and frozen requirement flag. A missing environment credential returns an error wrapping ErrInvalidRequest and ErrAPIKeyEnvMissing. Do not read and retain the environment value; the model client continues to resolve it when generation begins.

The result produced by the shared helper uses the frozen preparation values for all provenance. For RunPrepared, StartTime, EndTime, and Duration begin after the successful claim and exclude preparation and consumer-held delay. For ordinary Run, retain the current timing boundary beginning before preparation.

Error Mapping And Compatibility

Do not add a new public sentinel. Internal lifecycle errors wrap usecase.ErrInvalidRequest, which the existing root mapping exposes as promptkit.ErrInvalidRequest.

PrepareExecution uses the same public categories as Prepare. RunPrepared may return:

• ErrInvalidConfig for a nil engine;
• ErrInvalidRequest for handle misuse and invalid credentials;
• ErrAPIKeyEnvMissing together with ErrInvalidRequest when the frozen environment name is no longer set and no direct key is retained;
• ErrCapacityExceeded for admission rejection;
• ErrLLMGenerate for model-generation failures; and
• ErrValidation for inability to validate or repair.

Preserve wrapped collaborator and context errors according to the existing mapping rules. A content validation rejection remains a successful RunResult with ValidationFailed, not an operational error.

Do not change stable JSON for PreparedRun or RunResult, public request and result ownership, provider request bodies or headers, backend selection, capacity policies, LLMClient, ArtifactReader, Prepare, or Run.

Test Ownership

Tests must be split by the narrowest stable owner.

internal/validate/standard_validator_test.go owns frozen schema mechanics. Add focused tests for both configured schema-source forms:

• a directory-backed StandardValidator test whose root schema references a second file; after PrepareValidation, replace or remove both files and prove that valid and invalid artifacts are judged by the original compiled schema; and
• an FSValidator test using a mutable in-memory fs.FS; mutate its root and referenced entries after PrepareValidation and prove the same invariant.

Each test must also confirm that SchemaDocument is the original root document used at preparation. Existing path-containment and dialect matrices remain the owners of those rules and must not be duplicated.

internal/usecase tests own orchestration details that cannot be observed cleanly at the facade:

• PrepareExecution completes source loading, artifact reading, rendering, and validation-plan creation without calling admission or generation;
• RunPrepared rechecks environment credential availability before admission;
• the prepared path supplies the frozen validation plan to initial validation and every repair attempt;
• admission is released on success and each error exit; and
• the ordinary Run admission-before-complete-preparation ordering remains unchanged after refactoring.

Use small test-controlled collaborators. Do not assert private enum values, mutex layout, helper call graphs, or cleanup implementation details.

Add prepared_execution_contract_test.go in external package promptkit_test for the public contract. Cover these distinct risks with the smallest coherent set of tests:

1. Frozen execution and independent details. Prepare a request, mutate caller-owned request values, mutable prompt/profile/input/schema sources, and one returned PreparedRun, then run the handle. Assert that the captured GenerateRequest, validation, result provenance, and a second Details call all retain the original prepared state. Assert that source collaborators are not reopened during execution.
2. Lifecycle and engine binding. Prove that a foreign engine is rejected without consuming the handle, the owner can then run it once, a second call through either the original or a copied public handle is rejected, and details remain available.
3. Concurrent claim. Race two owning-engine RunPrepared calls against a blocking fake client. Exactly one reaches generation and the other matches ErrInvalidRequest. Run this test under the race detector.
4. Discard and formatting security. Retain a distinctive direct API key, verify that Details, ordinary JSON encoding, %v, %+v, %#v, and lifecycle errors do not contain it or rendered content, discard the handle, and verify idempotence, execution rejection, and retained details.
5. Credential and capacity timing. Prove that preparation does not occupy admission; removing a required environment credential makes execution fail before generation; a capacity rejection consumes the handle; and the execution result's timing excludes a test-controlled delay between preparation and execution.

Reuse existing root contract fixtures and capacity fakes where practical. Never put a real-looking credential in a fixture. Do not test JSON equality for the opaque handle, exact lifecycle error strings, private state, run-ID format, or clock-duration precision.

Documentation Ownership

After implementation:

• exported declarations and GoDoc in prepared_execution.go, engine.go, and types.go own the exact public lifecycle, ownership, security, timing, cancellation, and error contracts;
• docs/consumers/pkg-promptkit.md explains when to choose Prepare, Run, or PrepareExecution plus RunPrepared, and shows one concise two-phase workflow using defer prepared.Discard();
• docs/internal/sources.md explains that prepared execution freezes all source-derived state, including transitive schema references, while ordinary Prepare remains inspection-only;
• docs/internal/llm.md explains that prepared generation uses the retained target and messages, rechecks environment credential availability, and does not reopen sources;
• docs/internal/capacity.md explains that PrepareExecution performs no admission and that RunPrepared acquires and holds the normal run lease across generation, validation, and internal repair; and
• docs/internal/overview.md updates the validator and use-case inventory to mention frozen validation plans and one-attempt prepared execution.

Keep these documents at their established abstraction levels. Link to exact Go declarations rather than reproducing field-by-field contracts. No README, format reference, integration contract, maintained example, architecture policy, release note, or release-procedure update is required because the feature changes none of their owned concerns.

Stage 1: Add Frozen Validation Plans

Status: Complete.

Objective

Create an internal validation artifact that eagerly captures every schema resource required by later validation, without changing existing validation or engine behavior.

Implementation Prompt

1. Re-read docs/development.md, all files under docs/policy/, prepared-execution.md, internal/validate/validator.go, internal/validate/standard_validator.go, and the focused validator tests.
2. Confirm the working tree and preserve all existing roadmap and wishlist changes.
3. Add PreparedValidation and ValidationPreparer to internal/validate/validator.go with the exact responsibilities in the fixed design. Leave Validator unchanged.
4. Implement PrepareValidation for StandardValidator and FSValidator. Reuse the existing root resolution, resource identifiers, contained loaders, dialect checks, compiler configuration, JSON parser, and validation-result builder.
5. Ensure the root document is registered from the already-loaded value and every transitive reference is loaded during compilation. Retain only the compiled schema, root document, and copied contract required for later validation.
6. Ensure the prepared validator checks its validation-time context and does not touch either source after successful preparation.
7. Add the two focused frozen-reference tests described under Test Ownership. Keep existing tests unchanged except for shared test setup that materially reduces duplication.

Focused Validation

Run:

gofmt -w internal/validate/validator.go \
  internal/validate/standard_validator.go \
  internal/validate/standard_validator_test.go
go test ./internal/validate
go test -race ./internal/validate
go vet ./internal/validate
git diff --check

Completion Gate

Stage 1 is complete when:

• both built-in validator forms produce prepared plans;
• prepared schema validation works after root and referenced sources change or disappear;
• provider schema metadata can use the exact root document captured by the plan;
• existing Validator.Validate behavior remains unchanged; and
• focused tests, race tests, vet, formatting, and whitespace checks pass.

Stage 2: Implement Internal Handle Lifecycle And Prepared Execution

Status: Complete.

Objective

Add complete preparation, one-attempt lifecycle management, cleanup, and source-free execution to the use-case layer while preserving existing Prepare and Run behavior.

Implementation Prompt

1. Re-read the completed Stage 1 declarations, internal/usecase/runner.go, its focused tests, internal/capacity, and the execution flow described in prepared-execution.md.
2. Add internal/usecase/prepared_execution.go with the internal handle, payload, lifecycle transitions, deep-copying, redaction, cleanup, and runner ownership described in the fixed design.
3. Extract the common final preparation helper that accepts resolved structured-output metadata. Keep completePreparation and existing resolveStructuredOutput behavior intact for Prepare and Run.
4. Implement Runner.PrepareExecution using validate.ValidationPreparer. Build JSON Schema structured-output metadata from the frozen plan's root document.
5. Refactor post-preparation generation, validation, repair, and result assembly into the shared private helper. Preserve ordinary Run timing, early admission, error ordering, and live validator behavior.
6. Implement Runner.RunPrepared: verify owner, claim once, create run ID and execution start time, recheck credentials, acquire admission, execute against the retained plan, release admission, and clean the detached payload on every exit.
7. Implement nil-safe, idempotent discard and fresh redacted details copying. Confirm the RunPrepared/Discard race is resolved solely by the shared mutex transition.
8. Add the internal orchestration tests described under Test Ownership, including regression coverage for ordinary Run's admission ordering.

Focused Validation

Run:

gofmt -w internal/usecase/runner.go \
  internal/usecase/prepared_execution.go \
  internal/usecase/*_test.go
go test ./internal/usecase
go test -race ./internal/usecase
go test ./internal/validate
go vet ./internal/usecase ./internal/validate
git diff --check

Completion Gate

Stage 2 is complete when:

• a runner can prepare and later execute without reopening any request source;
• one valid invocation consumes the handle on every outcome;
• foreign-runner rejection does not consume the handle;
• discard and claim are race-safe and cleanup drops secret-bearing state;
• prepared execution rechecks credentials and admits only at execution time;
• frozen validation is reused across initial output and repair attempts;
• existing Prepare and Run tests retain their observable ordering and behavior; and
• focused tests, race tests, vet, formatting, and whitespace checks pass.

Stage 3: Publish The Root Facade And Public Contract

Status: Complete.

Objective

Expose the accepted opaque handle workflow through the root package and protect its compatibility, security, and concurrency guarantees at the consumer boundary.

Implementation Prompt

1. Re-read the accepted public workflow, root conversion and error-mapping helpers, existing Prepare and Run GoDoc, and the external-package contract-test conventions.
2. Add root prepared_execution.go with the opaque wrapper, Details, Discard, String, and GoString.
3. Add Engine.PrepareExecution and Engine.RunPrepared to engine.go. Reuse toDomainRunRequest, fromDomainPreparedRun, fromDomainRunResult, and mapPublicError.
4. Add or reuse one public prepared-run deep-copy helper so every Details call returns fresh nested state. Do not expose the internal handle or retain a public direct credential.
5. Write complete GoDoc for the type and methods. Update RunRequest, PreparedRun, and RunResult GoDoc only where cross-references are needed to distinguish the new workflow; do not change their stable JSON or ownership contracts.
6. Add prepared_execution_contract_test.go and cover the five risk groups under Test Ownership. Reuse existing fixtures and fakes rather than duplicating backend, profile, schema, and capacity matrices.
7. Explicitly inspect formatted and JSON-encoded handles, lifecycle errors, details, captured model requests, and results for the test sentinel direct key.

Focused Validation

Run:

gofmt -w prepared_execution.go engine.go types.go convert.go \
  prepared_execution_contract_test.go
go test . -run 'PreparedExecution|RunPrepared|PrepareExecution'
go test -race . -run 'PreparedExecution|RunPrepared|PrepareExecution'
go test .
go vet .
go build .
git diff --check

Completion Gate

Stage 3 is complete when:

• the public API matches the accepted shape;
• details are fresh, stable, credential-redacted copies before and after every lifecycle outcome;
• default formatting and JSON expose no retained request or credential data;
• foreign, copied, concurrent, discarded, and reused handles obey the one-attempt contract;
• execution uses exactly the frozen request and validation state;
• preparation performs no generation or admission;
• result provenance matches details and execution timing excludes preparation and retention delay;
• public errors preserve the required errors.Is identities; and
• focused public, race, package, vet, build, formatting, and whitespace checks pass.

Stage 4: Update Documentation And Validate The Repository

Status: Complete.

Objective

Publish current-state guidance at the correct documentation owners, validate the complete repository, and mark the temporary roadmaps complete.

Implementation Prompt

1. Re-read the implemented declarations and tests before documenting them.
2. Update docs/consumers/pkg-promptkit.md, docs/internal/sources.md, docs/internal/llm.md, docs/internal/capacity.md, and docs/internal/overview.md according to Documentation Ownership.
3. Keep the consumer example concise, copyable, secret-free, and explicit about defer prepared.Discard(). Link to declarations for exact lifecycle and error semantics.
4. Follow every added or changed Markdown link and confirm its target exists. Confirm that repository-relative links in both roadmap documents and all changed current-state documents resolve.
5. Run the complete maintainer validation sequence below.
6. Inspect the final diff for accidental provider-wire changes, source reloading during prepared execution, stable-JSON changes, credentials, generated artifacts, local workspace files, module replacements, or unrelated edits.
7. After every completion gate passes, set every stage status, this plan's status, and the status in prepared-execution.md to Complete. Do not retire or remove the roadmaps in the implementation change; roadmap retirement follows implementation review.

Full Validation

Run the complete sequence from docs/development.md:

go test ./...
go test -race ./...
go run ./examples/go-library/prepare
go vet ./...
go build ./...
gofmt -l $(git ls-files '*.go')

Then run the documented Markdown-link and repository-hygiene checks from docs/development.md, followed by:

git diff --check
git status --short

Completion Gate

Stage 4 and the feature are complete when:

• exact public contracts and task-oriented guidance are documented at their canonical owners;
• internal source, validation, execution, credential, and capacity boundaries are accurately described without duplicating public GoDoc;
• all changed links resolve and no documentation claims unimplemented behavior;
• the complete ordinary and race test suites pass offline;
• the maintained preparation example, vet, build, formatting, link, hygiene, and whitespace checks pass;
• the final diff is limited to the feature, its tests, documentation, and already-present roadmap changes; and
• both roadmap documents accurately report completion.

Open Questions

None.