diff --git a/docs/roadmap/future.md b/docs/roadmap/future.md index 8787ed2..555a907 100644 --- a/docs/roadmap/future.md +++ b/docs/roadmap/future.md @@ -33,25 +33,9 @@ consumers. ## Ideas -### Executable preparation handles - -Allow a consumer to prepare one exact execution, inspect and retain its -caller-owned preparation details, and later execute that already-prepared work -without reloading or rerendering prompt, profile, schema, or input sources. -This would remove the duplicate `Prepare`-then-`Run` workaround reported by -[Notarius](notarius-promptkit-wishlist.md#priority-1-atomic-execution-with-prepared-details) -and -[Weatherreporter](weatherreporter-promptkit-wishlist.md#priority-1-executable-preparation-handles). - -- Keep the handle opaque, engine-bound, non-serializable, and initially - one-shot. -- Keep existing `Prepare` and `Run` workflows available. -- Return public preparation details under existing caller-ownership and - credential-redaction rules. -- Perform backend admission when execution begins rather than reserving - capacity while a consumer retains the prepared handle. -- Treat a separate atomic detailed-run method as possible later convenience - rather than a second initial execution model. +Executable preparation handles have been selected for active planning in the +[focused feature roadmap](prepared-execution.md). The remaining ideas are +still available for future selection. ### Prompt-independent profile inspection diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 2b8a8c6..c76f200 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,315 +1,683 @@ -# Local Backend Convenience Implementation Plan +# Executable Preparation Handles Implementation Plan -**Status:** Complete. +**Status:** Accepted. ## Purpose This document is the decision-complete implementation plan for -[local backend convenience](local-backend.md). It is written for a coding -agent that will implement each stage in order. +[executable preparation handles](prepared-execution.md). It is written for a +coding agent that will implement each stage in order. -The feature roadmap owns the motivation, consumer paths, policy choices, +The feature roadmap owns the motivation, consumer workflow, policy choices, compatibility requirements, non-goals, and target end state. This document -owns the exact proposed API, file-level changes, implementation sequence, test +owns the concrete design, file-level changes, implementation sequence, test ownership, documentation work, validation commands, and completion gates. ## Implementation Rules -- Complete the stages in order. Keep the root package compiling and its - focused tests passing at every stage boundary. -- Preserve unrelated working-tree changes. The feature roadmap may already be - uncommitted when implementation begins; retain it. +- Complete the stages in order. Keep the repository compiling and the focused + tests passing at every stage boundary. +- Preserve unrelated working-tree changes. In particular, retain the accepted + roadmap and the downstream wishlist and future-roadmap edits that may + already be present. - Follow every policy under `docs/policy/`, the task-specific reading guide in `docs/development.md`, and the accepted behavior in - `local-backend.md`. -- Keep the API in the root `promptkit` package. Do not add a public or internal - package for this feature. -- Implement the helper as a transparent constructor for the existing - `Backend` type. Do not add a second backend representation or bypass - `WithBackend`. -- Leave normalization, validation, copying, registry construction, capacity - defaulting, and duplicate detection in their existing owners. -- Do not add environment-variable discovery, package-global registration, - implicit local defaults, model selection, profile construction, or support - for non-OpenAI-compatible transports. -- Keep tests lean and behavior-focused. Do not duplicate the existing - registry and concurrency test matrices merely because the constructor - reaches those mechanisms. -- Update exact GoDoc with the exported declarations. Update current-state - consumer guidance only after the corresponding API is implemented. -- Do not update release notes, create a release, change a module version, or - tag a commit as part of this work. + `prepared-execution.md`. +- Keep the supported API in the root `promptkit` package. Internal lifecycle, + validation, and execution state must remain below Go's `internal/` + boundary. +- Preserve the observable behavior and ordering of existing `Engine.Prepare` + and `Engine.Run`. Do not implement the new workflow by redefining either + existing method in terms of the new public methods. +- Reuse preparation and post-preparation execution mechanics where their + ordering is genuinely shared. Preserve `Run`'s existing early admission + boundary while ensuring `PrepareExecution` performs complete preparation + without admission. +- Do not add durable serialization, retries, provider failover, background + workers, a task queue, capacity reservation during preparation, or + consumer-specific persistence and redaction policy. +- Keep direct API-key values in private execution state only. Every public + details value, formatted handle, JSON value, result, hash, and error must + remain credential-free. +- Keep tests lean but give durable coverage to the public compatibility, + security, concurrency, one-attempt, source-snapshot, cancellation, and + capacity invariants. Use real internal collaborators where they are fast and + deterministic and fakes at model-generation and synchronization boundaries. +- Update exact GoDoc with exported declarations. Update current-state + consumer and internal documentation only after the corresponding behavior + exists. +- Do not add release notes, change a module version, create a release, or tag + a commit as part of this work. ## Fixed Design -### Exported API +### Public API -Add these declarations to `backends.go` in package `promptkit`: +Add an opaque root-package type and these methods: ```go -// BackendLocal is the conventional ID used by LocalBackend. It is not a -// built-in or reserved backend and must be registered with WithBackend. -const BackendLocal = "local" +type PreparedExecution struct { + // Unexported Promptkit-owned state only. +} -// LocalBackend returns a Backend for a conventional local OpenAI-compatible -// endpoint. -func LocalBackend(endpoint string, concurrencyLimit int) Backend { - return Backend{ - ID: BackendLocal, - Endpoint: endpoint, - ConcurrencyLimit: concurrencyLimit, - } +func (e *Engine) PrepareExecution( + ctx context.Context, + req RunRequest, +) (*PreparedExecution, error) + +func (p *PreparedExecution) Details() PreparedRun + +func (p *PreparedExecution) Discard() + +func (e *Engine) RunPrepared( + ctx context.Context, + prepared *PreparedExecution, +) (*RunResult, error) +``` + +Place the public handle and its handle-local methods in a new root file named +`prepared_execution.go`. Keep the two engine operations in `engine.go` beside +`Prepare` and `Run` so all engine workflows remain discoverable together. + +`PreparedExecution` must contain no exported fields. It wraps one pointer to +Promptkit-owned internal state; copying the public struct therefore preserves +one shared lifecycle rather than creating another execution opportunity. + +Implement both: + +```go +func (p *PreparedExecution) String() string +func (p *PreparedExecution) GoString() string +``` + +with a constant credential- and content-free representation such as +`promptkit.PreparedExecution{opaque}`. These methods must not branch on, +inspect, or format retained execution state. Do not add a custom JSON +representation: with no exported fields, ordinary JSON encoding exposes no +state, and JSON is explicitly outside the handle contract. + +The exact GoDoc must own these behaviors: + +- `PrepareExecution` performs complete preparation but no model call and no + backend-capacity admission; +- the handle is bound to the creating engine and permits one + `RunPrepared` invocation; +- `Details` returns a fresh caller-owned, credential-redacted copy on every + call and remains usable after discard or execution; +- `Discard` is nil-safe and idempotent, invalidates an unclaimed handle, and + is not an execution-cancellation mechanism; +- `RunPrepared` atomically consumes the one attempt before credential + revalidation, admission, generation, or validation; +- lifecycle misuse matches `ErrInvalidRequest`, while a nil engine matches + `ErrInvalidConfig`; +- an operational execution error returns no partial `RunResult`; +- the `RunPrepared` context does not inherit from the preparation context and + governs the execution attempt; and +- handles are opaque in-process values, not serializable or restartable jobs. + +For a nil or zero-value handle, `Details` returns a zero `PreparedRun` and +`Discard` does nothing. A nil, zero-value, foreign-engine, discarded, +claimed, or used handle passed to `RunPrepared` returns an error matching +`ErrInvalidRequest`. + +### Internal Validation Snapshot + +Extend `internal/validate/validator.go` with two internal interfaces: + +```go +// PreparedValidation validates artifacts against one frozen output contract. +type PreparedValidation interface { + Validate( + ctx context.Context, + artifact *domain.Artifact, + ) (domain.ValidationResult, error) + + // SchemaDocument returns the root JSON Schema document used for provider + // structured output, or nil for non-schema modes. The returned value is + // internal immutable state and must not be mutated by callers. + SchemaDocument() any +} + +// ValidationPreparer freezes the validation resources for one output +// contract. +type ValidationPreparer interface { + PrepareValidation( + ctx context.Context, + contract domain.OutputContract, + ) (PreparedValidation, error) } ``` -The exact GoDoc may be wrapped or expanded for clarity, but it must own and -communicate all of these contract points: +The exact names may be adjusted only to avoid an actual Go naming conflict; +the separation of responsibilities and method shapes are fixed. +`validate.Validator` and its existing `Validate` method remain unchanged so +the existing `Run` behavior and internal test fakes do not acquire a new +mandatory method. -- `BackendLocal` is the case-sensitive conventional ID `"local"`; -- it is neither built in nor reserved; -- calling `LocalBackend` does not register anything; -- the returned value must be supplied through `WithBackend`; -- `endpoint` and `concurrencyLimit` are copied into the corresponding fields - without normalization or validation; -- `APIKeyEnv`, `ExtraParams`, and `QueueCapacity` retain their zero values; and -- normal `NewEngine` backend validation and concurrency semantics apply after - registration. +`StandardValidator` and `FSValidator` implement `ValidationPreparer`. Their +prepared-validation implementation: -Keep `BackendOpenRouter` unchanged. `BackendLocal` must not alias an internal -registry constant because the internal registry has no special local-backend -identity or behavior. +- captures a value copy of the complete `OutputContract`; +- handles none, basic, and JSON validation without any source access; +- for JSON Schema, loads the root document, validates its dialect, registers + it with a new compiler, compiles it during `PrepareValidation`, and retains + the compiled schema; +- allows the compiler to resolve every transitive reference during that + compilation using the existing contained filesystem or `fs.FS` loader; +- returns the same loaded root document through `SchemaDocument`, so the + provider-facing structured-output constraint and later validation derive + from one root snapshot; and +- validates generated instances only against retained in-memory state. Its + `Validate` method must not reopen a path, call `fs.ReadFile`, resolve a + symlink, or invoke a schema loader after preparation succeeds. -Place `BackendLocal` near `BackendOpenRouter`, and place `LocalBackend` after -the `Backend` declaration and before `WithBackend`. This keeps the conventional -IDs, configured value, convenience constructor, and registration option -discoverable in one file. +Reuse the existing JSON parsing, validation-result construction, schema +dialect checks, contained-path rules, compiler configuration, and validation +error wording. A prepared JSON Schema compilation or reference-loading failure +is a preparation-time validation error. Existing direct calls to +`Validator.Validate`, including the existing `Run` path, retain their current +lazy compilation and source-loading behavior. -### Constructor Semantics +Do not expose the JSON Schema library's compiled type outside +`internal/validate`. Do not place compiled schemas or validation interfaces in +`internal/domain`. -`LocalBackend` is a pure struct constructor. Its complete behavior is -equivalent to the keyed literal shown above. +### Internal Prepared-Execution State -In particular, the constructor must not: +Add `internal/usecase/prepared_execution.go`. Define an internal +`PreparedExecution` owned by one `Runner`. It contains: -- trim or parse the endpoint; -- reject blank endpoints or negative limits; -- choose a default limit; -- assign an API-key environment variable; -- allocate an empty `ExtraParams` map; -- assign a queue-capacity pointer; -- read process environment; -- mutate package-global or engine state; or -- call `WithBackend` itself. +- the immutable owner `*Runner`; +- a mutex; +- a small private lifecycle state with at least ready, claimed, and discarded + states; +- a credential-redacted prepared-details snapshot that survives every + lifecycle transition; and +- while ready, a pointer to a private payload containing the full + `domain.PreparedRun` and its `validate.PreparedValidation`. -Deferred validation is intentional. It keeps one validation path for all -`Backend` values: `WithBackend` copies the public value into engine options, -and `NewEngine` constructs the validated immutable registry. A positive limit -with a nil queue continues to select the existing default queue capacity of -1024; zero continues to mean unlimited; a negative value continues to make -`NewEngine` fail with `ErrInvalidConfig`. +The payload is the only retained object containing `RunRequest.APIKey`. +Construct the public-details snapshot as a deep copy of the prepared run and +clear its `EffectiveModelParams.APIKey`. Deep-copy every mutable nested value, +including messages, maps, target extra parameters, and the structured-output +schema. Reuse the repository's JSON-compatible copy helpers rather than +maintaining an ad hoc reflection copier. -The returned `Backend` remains an ordinary caller-owned value. Consumers can -modify it before passing it to `WithBackend`, although task-oriented -documentation should direct materially customized configurations to an -explicit keyed `Backend` literal. - -### Identity And Compatibility - -Do not add `"local"` to the internal built-in or reserved ID set. A consumer -must be able to register either: +Add these internal operations: ```go -promptkit.LocalBackend(endpoint, limit) +func (r *Runner) PrepareExecution( + ctx context.Context, + req domain.RunRequest, +) (*PreparedExecution, error) + +func (p *PreparedExecution) Details() *domain.PreparedRun + +func (p *PreparedExecution) Discard() + +func (r *Runner) RunPrepared( + ctx context.Context, + prepared *PreparedExecution, +) (*domain.RunResult, error) ``` -or: +Internal exported names are acceptable here because the root facade must call +them, but they remain inaccessible to downstream consumers through Go's +`internal` rule. -```go -promptkit.Backend{ - ID: promptkit.BackendLocal, - Endpoint: endpoint, -} -``` +`PrepareExecution` performs these operations in order: -through the existing `WithBackend` option. Both forms participate in ordinary -duplicate-ID detection. Existing consumers that already use the literal ID -`"local"` remain source- and behavior-compatible. +1. call the existing request-copy boundary in the root facade before entering + the runner; +2. run `resolvePreparation`, including the existing environment credential + availability check; +3. prepare a frozen validation plan through + `validate.ValidationPreparer`; +4. construct provider structured-output metadata from the plan's root schema + document for JSON Schema mode; +5. load and hash artifacts, render the session and messages, and assemble the + complete `domain.PreparedRun`; and +6. construct the ready internal handle with independent full and redacted + snapshots. -No existing `Backend`, `WithBackend`, profile, registry, execution-target, or -capacity semantics change. Do not modify `internal/backend`, -`internal/capacity`, `internal/domain`, `engine.go`, `profiles.go`, or -`types.go` for this feature. +Extract the artifact-loading, rendering, hashing, timing, and +`domain.PreparedRun` assembly portion of `completePreparation` into a helper +that accepts an already-resolved `*domain.StructuredOutputSpec`. +`completePreparation` must continue to call the existing +`resolveStructuredOutput` first and then use that helper. This preserves +existing `Prepare` and `Run` schema behavior. `PrepareExecution` uses the +frozen validation plan's schema document to build the same structured-output +shape without calling `resolveStructuredOutput`. + +When the runner has no validator, preserve the existing runner semantics: +validation is skipped and a no-op prepared plan is sufficient. When it has a +non-nil validator that does not implement `ValidationPreparer`, +`PrepareExecution` fails with `ErrValidation`; do not fall back to a plan that +would reopen sources later. The root engine's standard and `fs.FS` validators +always implement the new interface. + +### Claim, Cleanup, And Engine Binding + +`RunPrepared` checks engine ownership before attempting a claim. A foreign +runner returns `ErrInvalidRequest` without changing the handle. The owning +runner then locks the state and atomically: + +1. accepts only the ready state; +2. changes it permanently to claimed; +3. detaches the private payload from the handle; and +4. unlocks before doing any blocking or collaborator work. + +Every valid owning-engine invocation consumes the attempt at this point. Run +ID creation, credential failure, pre-canceled context, capacity rejection, +generation failure, operational validation failure, and success all leave the +handle claimed and unusable. + +The execution call holds the detached payload locally and defers cleanup on +every exit. Cleanup must at minimum: + +- overwrite the direct `EffectiveModelParams.APIKey` field with an empty + string; +- drop the local prepared-run pointer; +- drop the frozen-validation reference; and +- leave the separate redacted details snapshot intact. + +This is reference cleanup, not a promise of secure Go string-memory erasure. + +`Discard` locks the same state. If ready, it changes the state to discarded, +detaches the payload, unlocks, and performs the same cleanup. If claimed or +already discarded, it is a no-op. Therefore a `RunPrepared`/`Discard` race has +one winner: discard prevents a claim only if it acquires the ready state +first; otherwise it does not cancel the running attempt. + +`Details` locks only long enough to read the retained redacted snapshot and +returns a fresh deep copy. It must not return a pointer, slice, map, schema, or +extra-parameter tree shared with either the handle or another `Details` call. + +### Prepared Execution Mechanics + +Refactor the post-preparation portion of `Runner.Run` into a private execution +helper that accepts: + +- a fully prepared `domain.PreparedRun`; +- a caller-supplied run ID and execution start time; and +- a validation function that accepts the artifact and repair-attempt count. + +The helper owns model generation, artifact construction, content validation, +internal repair, final result assembly, and result end timing. The live +`Runner.Run` path supplies its existing `validateOutput` function. The +prepared path supplies a wrapper around the retained +`validate.PreparedValidation`. Both wrappers set the actual repair-attempt +count on the returned validation result. + +Keep admission outside this shared helper: + +- `Runner.Run` retains its current sequence of run-ID creation, start time, + target resolution, early admission, complete preparation, execution, and + release. +- `Runner.RunPrepared` claims the handle, creates the run ID and execution + start time, revalidates credential availability, admits the frozen backend, + executes the shared helper, and releases admission. + +Extract a small private admission helper only if it preserves the exact +existing capacity wrapping and release behavior. The limited-backend lease +must cover generation, validation, and all internal repair calls and must be +released on every return. + +Before admission, `RunPrepared` calls the existing `validateAPIKey` with the +frozen environment-variable name, retained direct key, and frozen requirement +flag. A missing environment credential returns an error wrapping +`ErrInvalidRequest` and `ErrAPIKeyEnvMissing`. Do not read and retain the +environment value; the model client continues to resolve it when generation +begins. + +The result produced by the shared helper uses the frozen preparation values +for all provenance. For `RunPrepared`, `StartTime`, `EndTime`, and `Duration` +begin after the successful claim and exclude preparation and consumer-held +delay. For ordinary `Run`, retain the current timing boundary beginning before +preparation. + +### Error Mapping And Compatibility + +Do not add a new public sentinel. Internal lifecycle errors wrap +`usecase.ErrInvalidRequest`, which the existing root mapping exposes as +`promptkit.ErrInvalidRequest`. + +`PrepareExecution` uses the same public categories as `Prepare`. +`RunPrepared` may return: + +- `ErrInvalidConfig` for a nil engine; +- `ErrInvalidRequest` for handle misuse and invalid credentials; +- `ErrAPIKeyEnvMissing` together with `ErrInvalidRequest` when the frozen + environment name is no longer set and no direct key is retained; +- `ErrCapacityExceeded` for admission rejection; +- `ErrLLMGenerate` for model-generation failures; and +- `ErrValidation` for inability to validate or repair. + +Preserve wrapped collaborator and context errors according to the existing +mapping rules. A content validation rejection remains a successful +`RunResult` with `ValidationFailed`, not an operational error. + +Do not change stable JSON for `PreparedRun` or `RunResult`, public request and +result ownership, provider request bodies or headers, backend selection, +capacity policies, `LLMClient`, `ArtifactReader`, `Prepare`, or `Run`. ### Test Ownership -The root external-package contract suite in `public_contract_test.go` owns the -new public behavior. Add one focused test named: +Tests must be split by the narrowest stable owner. -```go -func TestLocalBackendConstructsAndRegistersConventionalBackend(t *testing.T) -``` +`internal/validate/standard_validator_test.go` owns frozen schema mechanics. +Add focused tests for both configured schema-source forms: -The test must: +- a directory-backed `StandardValidator` test whose root schema references a + second file; after `PrepareValidation`, replace or remove both files and + prove that valid and invalid artifacts are judged by the original compiled + schema; and +- an `FSValidator` test using a mutable in-memory `fs.FS`; mutate its root and + referenced entries after `PrepareValidation` and prove the same invariant. -1. call `promptkit.LocalBackend` with a test endpoint and a positive, - test-owned concurrency limit; -2. compare the returned value with this complete expected value: +Each test must also confirm that `SchemaDocument` is the original root +document used at preparation. Existing path-containment and dialect matrices +remain the owners of those rules and must not be duplicated. - ```go - promptkit.Backend{ - ID: promptkit.BackendLocal, - Endpoint: endpoint, - ConcurrencyLimit: limit, - } - ``` +`internal/usecase` tests own orchestration details that cannot be observed +cleanly at the facade: - A whole-value comparison is appropriate here because the exact zero-value - fields are part of this small public constructor's contract. -3. register that returned value with `WithBackend`; -4. add an in-memory profile whose `BackendID` is - `promptkit.BackendLocal`; -5. construct an engine through the existing contract-test prompt fixture; -6. call `Prepare`; and -7. assert that the prepared result exposes `BackendLocal` as the selected - backend and the supplied endpoint as the effective endpoint. +- `PrepareExecution` completes source loading, artifact reading, rendering, + and validation-plan creation without calling admission or generation; +- `RunPrepared` rechecks environment credential availability before admission; +- the prepared path supplies the frozen validation plan to initial validation + and every repair attempt; +- admission is released on success and each error exit; and +- the ordinary `Run` admission-before-complete-preparation ordering remains + unchanged after refactoring. -This single test protects the realistic compatibility risks: accidental field -defaults, a changed conventional ID, failure to compose with `WithBackend`, -and accidental treatment of `"local"` as reserved. It also demonstrates that -the helper uses the existing backend/profile path. +Use small test-controlled collaborators. Do not assert private enum values, +mutex layout, helper call graphs, or cleanup implementation details. -Do not add separate tests for blank endpoints, malformed endpoints, negative -limits, queue defaulting, duplicate IDs, engine isolation, runtime capacity, -or caller mutation. Those mechanisms are unchanged and already have tests at -their owning boundaries. Do not add internal-package tests for this root -facade constructor. +Add `prepared_execution_contract_test.go` in external package +`promptkit_test` for the public contract. Cover these distinct risks with the +smallest coherent set of tests: -### Consumer Documentation +1. **Frozen execution and independent details.** Prepare a request, mutate + caller-owned request values, mutable prompt/profile/input/schema sources, + and one returned `PreparedRun`, then run the handle. Assert that the + captured `GenerateRequest`, validation, result provenance, and a second + `Details` call all retain the original prepared state. Assert that source + collaborators are not reopened during execution. +2. **Lifecycle and engine binding.** Prove that a foreign engine is rejected + without consuming the handle, the owner can then run it once, a second + call through either the original or a copied public handle is rejected, and + details remain available. +3. **Concurrent claim.** Race two owning-engine `RunPrepared` calls against a + blocking fake client. Exactly one reaches generation and the other matches + `ErrInvalidRequest`. Run this test under the race detector. +4. **Discard and formatting security.** Retain a distinctive direct API key, + verify that `Details`, ordinary JSON encoding, `%v`, `%+v`, `%#v`, and + lifecycle errors do not contain it or rendered content, discard the handle, + and verify idempotence, execution rejection, and retained details. +5. **Credential and capacity timing.** Prove that preparation does not occupy + admission; removing a required environment credential makes execution fail + before generation; a capacity rejection consumes the handle; and the + execution result's timing excludes a test-controlled delay between + preparation and execution. -Update `docs/consumers/pkg-promptkit.md` after the API exists. Keep Go -declarations and GoDoc as the exact API owner; the guide should help consumers -choose a workflow and link to `backends.go` for precise semantics. +Reuse existing root contract fixtures and capacity fakes where practical. +Never put a real-looking credential in a fixture. Do not test JSON equality +for the opaque handle, exact lifecycle error strings, private state, run-ID +format, or clock-duration precision. -Restructure the backend guidance to present these paths in increasing order of -configuration: +### Documentation Ownership -1. **Endpoint-only profile.** Show a small in-memory `Profile` with - `Endpoint` and `Model`. Explain that this is the simplest choice when only - one profile needs the endpoint and shared backend identity or capacity - policy is unnecessary. -2. **Local convenience constructor.** Show - `WithBackend(promptkit.LocalBackend("http://localhost:8000/v1", 2))` - together with a profile using - `BackendID: promptkit.BackendLocal`. Explain briefly that the helper is - explicit, is not pre-registered, does not read environment variables, and - leaves the queue capacity at the existing default for a positive limit. -3. **Complete backend value.** Preserve an advanced example using a keyed - `Backend` literal for needs such as `APIKeyEnv`, an explicit - `QueueCapacity`, extra parameters, a custom ID, or multiple local - endpoints. Use a custom ID other than `"local"` in that example so the - distinction from the conventional helper is clear. +After implementation: -Keep the existing backend-routing, selected-backend identity, concurrency, -credential, and error guidance unless a small wording adjustment is required -to make the new decision path coherent. Avoid repeating the complete field -contract or registry validation rules from GoDoc. +- exported declarations and GoDoc in `prepared_execution.go`, `engine.go`, and + `types.go` own the exact public lifecycle, ownership, security, timing, + cancellation, and error contracts; +- `docs/consumers/pkg-promptkit.md` explains when to choose `Prepare`, `Run`, + or `PrepareExecution` plus `RunPrepared`, and shows one concise two-phase + workflow using `defer prepared.Discard()`; +- `docs/internal/sources.md` explains that prepared execution freezes all + source-derived state, including transitive schema references, while + ordinary `Prepare` remains inspection-only; +- `docs/internal/llm.md` explains that prepared generation uses the retained + target and messages, rechecks environment credential availability, and + does not reopen sources; +- `docs/internal/capacity.md` explains that `PrepareExecution` performs no + admission and that `RunPrepared` acquires and holds the normal run lease + across generation, validation, and internal repair; and +- `docs/internal/overview.md` updates the validator and use-case inventory to + mention frozen validation plans and one-attempt prepared execution. -Do not add a new maintained example, README section, format-reference entry, -integration-contract change, internal-document change, or release note. The -consumer-guide snippets are sufficient for this small convenience API, and -none of those other documents owns the affected task or contract. +Keep these documents at their established abstraction levels. Link to exact +Go declarations rather than reproducing field-by-field contracts. No README, +format reference, integration contract, maintained example, architecture +policy, release note, or release-procedure update is required because the +feature changes none of their owned concerns. -## Stage 1: Add The Public Constructor And Contract Test +## Stage 1: Add Frozen Validation Plans -**Status:** Complete. +**Status:** Pending. ### Objective -Add the smallest public API that expresses the accepted local-backend -convention and protect its compatibility through the root public boundary. +Create an internal validation artifact that eagerly captures every schema +resource required by later validation, without changing existing validation +or engine behavior. ### Implementation Prompt 1. Re-read `docs/development.md`, all files under `docs/policy/`, - `local-backend.md`, `backends.go`, the backend-related portion of - `public_contract_test.go`, and the existing backend/concurrency GoDoc before - editing. -2. Confirm the working tree and preserve the uncommitted roadmaps and any - unrelated consumer changes. -3. Add the untyped exported string constant `BackendLocal = "local"` to - `backends.go` without changing `BackendOpenRouter`. -4. Add `LocalBackend(endpoint string, concurrencyLimit int) Backend` to - `backends.go` using the exact keyed-literal implementation in the fixed - design. -5. Write complete GoDoc for both declarations. Make their conventional, - explicit, non-built-in, non-reserved, and deferred-validation semantics - unambiguous. -6. Add - `TestLocalBackendConstructsAndRegistersConventionalBackend` to - `public_contract_test.go` exactly as specified under Test Ownership. Reuse - the existing contract prompt fixture rather than adding a fixture or test - helper. -7. Do not edit internal packages. If the constructor appears to require an - internal change, stop and reconcile the implementation with the fixed - transparent-constructor design instead. + `prepared-execution.md`, `internal/validate/validator.go`, + `internal/validate/standard_validator.go`, and the focused validator tests. +2. Confirm the working tree and preserve all existing roadmap and wishlist + changes. +3. Add `PreparedValidation` and `ValidationPreparer` to + `internal/validate/validator.go` with the exact responsibilities in the + fixed design. Leave `Validator` unchanged. +4. Implement `PrepareValidation` for `StandardValidator` and `FSValidator`. + Reuse the existing root resolution, resource identifiers, contained + loaders, dialect checks, compiler configuration, JSON parser, and + validation-result builder. +5. Ensure the root document is registered from the already-loaded value and + every transitive reference is loaded during compilation. Retain only the + compiled schema, root document, and copied contract required for later + validation. +6. Ensure the prepared validator checks its validation-time context and does + not touch either source after successful preparation. +7. Add the two focused frozen-reference tests described under Test Ownership. + Keep existing tests unchanged except for shared test setup that materially + reduces duplication. ### Focused Validation Run: ```sh -gofmt -w backends.go public_contract_test.go -go test . -run '^TestLocalBackendConstructsAndRegistersConventionalBackend$' +gofmt -w internal/validate/validator.go \ + internal/validate/standard_validator.go \ + internal/validate/standard_validator_test.go +go test ./internal/validate +go test -race ./internal/validate +go vet ./internal/validate +git diff --check +``` + +### Completion Gate + +Stage 1 is complete when: + +- both built-in validator forms produce prepared plans; +- prepared schema validation works after root and referenced sources change or + disappear; +- provider schema metadata can use the exact root document captured by the + plan; +- existing `Validator.Validate` behavior remains unchanged; and +- focused tests, race tests, vet, formatting, and whitespace checks pass. + +## Stage 2: Implement Internal Handle Lifecycle And Prepared Execution + +**Status:** Pending. + +### Objective + +Add complete preparation, one-attempt lifecycle management, cleanup, and +source-free execution to the use-case layer while preserving existing +`Prepare` and `Run` behavior. + +### Implementation Prompt + +1. Re-read the completed Stage 1 declarations, `internal/usecase/runner.go`, + its focused tests, `internal/capacity`, and the execution flow described in + `prepared-execution.md`. +2. Add `internal/usecase/prepared_execution.go` with the internal handle, + payload, lifecycle transitions, deep-copying, redaction, cleanup, and + runner ownership described in the fixed design. +3. Extract the common final preparation helper that accepts resolved + structured-output metadata. Keep `completePreparation` and existing + `resolveStructuredOutput` behavior intact for `Prepare` and `Run`. +4. Implement `Runner.PrepareExecution` using + `validate.ValidationPreparer`. Build JSON Schema structured-output metadata + from the frozen plan's root document. +5. Refactor post-preparation generation, validation, repair, and result + assembly into the shared private helper. Preserve ordinary `Run` timing, + early admission, error ordering, and live validator behavior. +6. Implement `Runner.RunPrepared`: verify owner, claim once, create run ID and + execution start time, recheck credentials, acquire admission, execute + against the retained plan, release admission, and clean the detached + payload on every exit. +7. Implement nil-safe, idempotent discard and fresh redacted details copying. + Confirm the `RunPrepared`/`Discard` race is resolved solely by the shared + mutex transition. +8. Add the internal orchestration tests described under Test Ownership, + including regression coverage for ordinary `Run`'s admission ordering. + +### Focused Validation + +Run: + +```sh +gofmt -w internal/usecase/runner.go \ + internal/usecase/prepared_execution.go \ + internal/usecase/*_test.go +go test ./internal/usecase +go test -race ./internal/usecase +go test ./internal/validate +go vet ./internal/usecase ./internal/validate +git diff --check +``` + +### Completion Gate + +Stage 2 is complete when: + +- a runner can prepare and later execute without reopening any request source; +- one valid invocation consumes the handle on every outcome; +- foreign-runner rejection does not consume the handle; +- discard and claim are race-safe and cleanup drops secret-bearing state; +- prepared execution rechecks credentials and admits only at execution time; +- frozen validation is reused across initial output and repair attempts; +- existing `Prepare` and `Run` tests retain their observable ordering and + behavior; and +- focused tests, race tests, vet, formatting, and whitespace checks pass. + +## Stage 3: Publish The Root Facade And Public Contract + +**Status:** Pending. + +### Objective + +Expose the accepted opaque handle workflow through the root package and +protect its compatibility, security, and concurrency guarantees at the +consumer boundary. + +### Implementation Prompt + +1. Re-read the accepted public workflow, root conversion and error-mapping + helpers, existing `Prepare` and `Run` GoDoc, and the external-package + contract-test conventions. +2. Add root `prepared_execution.go` with the opaque wrapper, `Details`, + `Discard`, `String`, and `GoString`. +3. Add `Engine.PrepareExecution` and `Engine.RunPrepared` to `engine.go`. + Reuse `toDomainRunRequest`, `fromDomainPreparedRun`, + `fromDomainRunResult`, and `mapPublicError`. +4. Add or reuse one public prepared-run deep-copy helper so every `Details` + call returns fresh nested state. Do not expose the internal handle or + retain a public direct credential. +5. Write complete GoDoc for the type and methods. Update `RunRequest`, + `PreparedRun`, and `RunResult` GoDoc only where cross-references are needed + to distinguish the new workflow; do not change their stable JSON or + ownership contracts. +6. Add `prepared_execution_contract_test.go` and cover the five risk groups + under Test Ownership. Reuse existing fixtures and fakes rather than + duplicating backend, profile, schema, and capacity matrices. +7. Explicitly inspect formatted and JSON-encoded handles, lifecycle errors, + details, captured model requests, and results for the test sentinel direct + key. + +### Focused Validation + +Run: + +```sh +gofmt -w prepared_execution.go engine.go types.go convert.go \ + prepared_execution_contract_test.go +go test . -run 'PreparedExecution|RunPrepared|PrepareExecution' +go test -race . -run 'PreparedExecution|RunPrepared|PrepareExecution' go test . go vet . go build . git diff --check ``` -Inspect the diff and confirm that this stage changes only `backends.go`, -`public_contract_test.go`, and the already-present roadmap files. - ### Completion Gate -Stage 1 is complete when: +Stage 3 is complete when: -- the exported constant and constructor match the fixed API; -- the constructor returns only the three specified non-zero fields; -- the helper registers through the ordinary `WithBackend` path; -- `"local"` remains a valid consumer registration rather than a reserved - built-in; -- the focused public-contract test passes; and -- the root package test, vet, build, formatting, and whitespace checks pass. +- the public API matches the accepted shape; +- details are fresh, stable, credential-redacted copies before and after every + lifecycle outcome; +- default formatting and JSON expose no retained request or credential data; +- foreign, copied, concurrent, discarded, and reused handles obey the + one-attempt contract; +- execution uses exactly the frozen request and validation state; +- preparation performs no generation or admission; +- result provenance matches details and execution timing excludes preparation + and retention delay; +- public errors preserve the required `errors.Is` identities; and +- focused public, race, package, vet, build, formatting, and whitespace checks + pass. -## Stage 2: Publish Consumer Guidance And Validate The Repository +## Stage 4: Update Documentation And Validate The Repository -**Status:** Complete. +**Status:** Pending. ### Objective -Make the simplest suitable local-endpoint configuration easy to discover, -confirm the complete change across the repository, and close the temporary -roadmaps. +Publish current-state guidance at the correct documentation owners, validate +the complete repository, and mark the temporary roadmaps complete. ### Implementation Prompt -1. Re-read the implemented declarations and GoDoc before describing them. -2. Update `docs/consumers/pkg-promptkit.md` according to the three-path - structure under Consumer Documentation. -3. Keep examples illustrative, minimal, secret-free, and consistent with the - implemented declarations. Link precise semantics to `backends.go` rather - than duplicating its field-by-field contract. +1. Re-read the implemented declarations and tests before documenting them. +2. Update `docs/consumers/pkg-promptkit.md`, + `docs/internal/sources.md`, `docs/internal/llm.md`, + `docs/internal/capacity.md`, and `docs/internal/overview.md` according to + Documentation Ownership. +3. Keep the consumer example concise, copyable, secret-free, and explicit + about `defer prepared.Discard()`. Link to declarations for exact lifecycle + and error semantics. 4. Follow every added or changed Markdown link and confirm its target exists. - Confirm that all local repository-relative links in - `local-backend.md`, this plan, and the changed consumer guide resolve. + Confirm that repository-relative links in both roadmap documents and all + changed current-state documents resolve. 5. Run the complete maintainer validation sequence below. -6. Inspect the final diff for accidental internal behavior changes, - generated artifacts, credentials, local workspace files, or module - replacements. -7. After every completion gate passes, set both stage statuses, this plan's - status, and the status in `local-backend.md` to `Complete`. Do not retire or - remove the roadmaps in the implementation change; roadmap retirement - follows implementation review. +6. Inspect the final diff for accidental provider-wire changes, source + reloading during prepared execution, stable-JSON changes, credentials, + generated artifacts, local workspace files, module replacements, or + unrelated edits. +7. After every completion gate passes, set every stage status, this plan's + status, and the status in `prepared-execution.md` to `Complete`. Do not + retire or remove the roadmaps in the implementation change; roadmap + retirement follows implementation review. ### Full Validation @@ -318,69 +686,37 @@ Run the complete sequence from `docs/development.md`: ```sh go test ./... go test -race ./... +go run ./examples/go-library/prepare go vet ./... go build ./... -go run ./examples/go-library/prepare gofmt -l $(git ls-files '*.go') -git diff --check ``` -The formatting command must produce no paths. Follow every added or changed -Markdown link and confirm its target and heading exist. - -Also inspect: +Then run the documented Markdown-link and repository-hygiene checks from +`docs/development.md`, followed by: ```sh +git diff --check git status --short -git diff --stat -git diff ``` -Confirm that: - -- production changes are limited to the root convenience API; -- no internal backend, registry, capacity, profile, or engine behavior - changed; -- `BackendLocal` is conventional and consumer-registerable, not built in or - reserved; -- `LocalBackend` performs no validation, normalization, environment lookup, - registration, allocation, or hidden default selection; -- the returned `Backend` leaves `QueueCapacity` nil so existing positive-limit - defaulting remains owned by the registry; -- existing endpoint-only profiles and complete custom backends remain - documented and supported; -- current-state documentation describes only the now-implemented API and - links to the canonical GoDoc for exact semantics; -- no `go.work`, `go.work.sum`, local module replacement, credential, - generated binary, or unrelated change was introduced; and -- the feature and implementation roadmaps contain no unresolved work marked - complete. - ### Completion Gate -Stage 2 is complete when every target-end-state item in -`local-backend.md` is implemented, the consumer guide clearly presents all -three configuration paths, all complete validation commands pass, all changed -links resolve, and the roadmap statuses accurately report completion. +Stage 4 and the feature are complete when: -## Implementation Handoff - -The implementation handoff should report: - -- the new `BackendLocal` and `LocalBackend` public API; -- that the helper remains explicit and composes with the ordinary backend - registry; -- the focused public-contract coverage added; -- the consumer-guide decision path added; -- the complete validation commands and results; and -- any unrelated working-tree changes that were preserved. - -Do not claim that a local backend is built in, pre-registered, configured from -environment variables, or assigned a default model or concurrency limit. +- exact public contracts and task-oriented guidance are documented at their + canonical owners; +- internal source, validation, execution, credential, and capacity boundaries + are accurately described without duplicating public GoDoc; +- all changed links resolve and no documentation claims unimplemented + behavior; +- the complete ordinary and race test suites pass offline; +- the maintained preparation example, vet, build, formatting, link, hygiene, + and whitespace checks pass; +- the final diff is limited to the feature, its tests, documentation, and + already-present roadmap changes; and +- both roadmap documents accurately report completion. ## Open Questions -None. The feature roadmap and this plan fix the exported API, constructor -semantics, identity treatment, compatibility behavior, test boundary, -documentation ownership, implementation sequence, validation gates, and -non-goals required for implementation. +None. diff --git a/docs/roadmap/notarius-promptkit-wishlist.md b/docs/roadmap/notarius-promptkit-wishlist.md index 493f4e1..e8b2871 100644 --- a/docs/roadmap/notarius-promptkit-wishlist.md +++ b/docs/roadmap/notarius-promptkit-wishlist.md @@ -18,10 +18,9 @@ not prescriptive names or finalized Go contracts. ## Priority 1: Atomic Execution With Prepared Details **Disposition:** Covered by the accepted -[executable preparation handles](future.md#executable-preparation-handles) -catalog entry. The shared two-phase capability should provide the required -single-preparation consistency; a separate `RunDetailed` method is not -cataloged initially. +[executable preparation handles](prepared-execution.md) roadmap. The shared +two-phase capability should provide the required single-preparation +consistency; a separate `RunDetailed` method is not cataloged initially. ### Downstream need diff --git a/docs/roadmap/prepared-execution.md b/docs/roadmap/prepared-execution.md new file mode 100644 index 0000000..b92f626 --- /dev/null +++ b/docs/roadmap/prepared-execution.md @@ -0,0 +1,280 @@ +# Executable Preparation Handles + +**Status:** Accepted. + +## Purpose + +Allow a consumer to prepare one exact Promptkit execution, inspect and retain +its credential-redacted public preparation details, and later execute that +already-prepared work without resolving or rendering the request again. + +This provides a supported preflight-before-generation boundary for +[Weatherreporter](weatherreporter-promptkit-wishlist.md#priority-1-executable-preparation-handles) +and removes the duplicate `Prepare`-then-`Run` workaround described by +[Notarius](notarius-promptkit-wishlist.md#priority-1-atomic-execution-with-prepared-details). + +## Motivation + +`Engine.Prepare` currently returns the provenance and rendered details that +consumers need for debugging, persistence, and preflight checks. `Engine.Run` +then performs its own preparation before generation. A consumer that needs +both values must therefore prepare the same logical request twice. + +That workaround duplicates source loading, input hashing, schema work, and +template rendering. It also permits filesystem-backed prompts, profiles, +schemas, or inputs to change between the public preparation and the +preparation that actually produces the result. + +Promptkit already owns the complete preparation and execution pipeline. An +opt-in prepared-execution handle should expose the missing boundary without +putting rendered content into every `RunResult` or moving persistence policy +into the library. + +## Consumer Workflow + +The target public workflow is: + +```go +prepared, err := engine.PrepareExecution(ctx, request) +if err != nil { + // Handle preparation failure. + return +} +defer prepared.Discard() + +details := prepared.Details() +// Persist or inspect a consumer-selected safe subset of details. + +result, err := engine.RunPrepared(ctx, prepared) +``` + +The target public surface is: + +```go +type PreparedExecution struct { + // Opaque Promptkit-owned state. +} + +func (e *Engine) PrepareExecution( + ctx context.Context, + req RunRequest, +) (*PreparedExecution, error) + +func (p *PreparedExecution) Details() PreparedRun + +func (p *PreparedExecution) Discard() + +func (e *Engine) RunPrepared( + ctx context.Context, + prepared *PreparedExecution, +) (*RunResult, error) +``` + +The declarations and GoDoc will own the exact implemented contract. The +important public shape is an opaque handle, caller-owned `PreparedRun` +details, an explicit discard operation, and execution through the engine that +created the handle. + +## Prepared Snapshot + +`PrepareExecution` performs complete preparation without model generation or +backend-capacity admission. It applies the same request validation, source +precedence, backend and profile resolution, credential requirement checks, +output-contract resolution, artifact loading, hashing, schema loading, +session resolution, and rendering behavior as `Prepare`. + +A successful handle freezes all source-derived state required for later +execution, including: + +- the selected prompt definition, profile, and backend identity; +- the complete effective execution target and request-field presence; +- rendered messages and effective session ID; +- prompt, rendered-prompt, and input hashes; +- the effective output contract and provider-facing structured-output + constraint; and +- private validation state sufficient to validate generated output without + reopening schema files or `fs.FS` resources, including required schema + references. + +After `PrepareExecution` succeeds, changes to prompt, profile, schema, input, +or request-owned data cannot change what `RunPrepared` sends to the model or +how it validates the generated output. + +The handle retains an internal snapshot independent from values returned by +`Details`. Mutating a returned `PreparedRun`, its maps, slices, messages, or +schema values does not affect later execution. Each `Details` call returns a +fresh caller-owned copy under the existing `PreparedRun` ownership and stable +JSON rules. + +## Handle Lifecycle + +A `PreparedExecution` is: + +- created only by a successful `PrepareExecution` call; +- bound to the exact `Engine` that created it; +- valid for one `RunPrepared` invocation; +- safe for repeated `Details` calls; +- intentionally opaque and without a supported JSON representation; and +- in-process state rather than a durable or restartable job. + +`RunPrepared` atomically claims a valid handle before beginning the execution +attempt. A second or concurrent invocation fails without starting another +execution, including when the first invocation ended in cancellation, capacity +rejection, generation failure, or operational validation failure. Copies of +the public handle share the same one-attempt state and cannot bypass this rule. + +A nil, zero-value, foreign-engine, discarded, already-claimed, or already-used +handle is invalid. `RunPrepared` reports these lifecycle errors through the +ordinary public invalid-request category. A failed foreign-engine invocation +does not consume a handle that remains valid for its owning engine. + +`Discard` idempotently makes an unclaimed handle unavailable for execution and +drops Promptkit's references to secret-bearing or execution-only state. +`RunPrepared` performs the same cleanup automatically after claiming a handle. +Credential-redacted public preparation details remain available after discard, +success, or failure so consumers can retain diagnostics. Promptkit does not +promise secure erasure of Go string memory. + +Lifecycle transitions are concurrency-safe. When `RunPrepared` and `Discard` +race, exactly one claims the ready handle. `Discard` is not an execution +cancellation mechanism and does not interrupt an attempt that has already +claimed the handle; consumers cancel that attempt through its context. + +## Credentials And Sensitive Data + +`Details` has the same security contract as `PreparedRun`: it can contain +rendered messages, schemas, identifiers, and hashes, but never a resolved API +key value. Consumers remain responsible for selecting, redacting, storing, and +retaining any persisted preparation material. + +A direct `RunRequest.APIKey` is retained only in opaque execution state until +the handle is run or discarded. It is never added to details, hashes, JSON, +`String`, or `GoString` output. + +An environment-variable name is frozen as part of the effective target, but +its credential value is not captured for the lifetime of the handle. +`PrepareExecution` applies the existing preparation-time availability check. +`RunPrepared` rechecks availability before admission, and the selected model +client uses the environment value visible during execution. This preserves +current secret ownership and avoids retaining an environment credential while +a consumer persists preflight material. + +The opaque handle must not expose retained request data or credentials through +default formatting, JSON, or error messages. + +## Admission, Cancellation, And Execution + +`PrepareExecution` never reserves backend admission or an active-generation +permit. Its context governs preparation only; cancellation after it returns +does not invalidate the handle. + +`RunPrepared` uses its own context for credential revalidation, backend +admission, active-generation waiting, model generation, output validation, and +any internal repair. Admission occurs when `RunPrepared` begins so a consumer +cannot occupy bounded capacity while inspecting or persisting preparation +details. + +For a limited backend, the admission lease covers the complete prepared +execution attempt after admission: generation, validation, internal repair, +and every success or failure exit. Actual generation continues to use the +backend's FIFO active-generation permit. Existing capacity error identity, +cancellation behavior, and release guarantees remain in force. + +Because a prepared handle is one-attempt, cancellation or capacity rejection +does not make it reusable. Retry and backoff policy remains with the consumer, +which may create a new prepared handle when another attempt is appropriate. + +## Results And Failures + +On success, `RunPrepared` returns the existing caller-owned `RunResult`. Its +source-derived provenance must match `Details`, including prompt identity and +hash, rendered-prompt hash, session ID, selected profile and backend, +effective target, and input hashes. + +`RunResult.StartTime`, `EndTime`, and `Duration` describe the +`RunPrepared` execution attempt. They exclude preparation time and any delay +while the consumer retained the handle. Preparation timing remains in +`PreparedRun`. + +Preparation failure returns no handle. After successful preparation, +`RunPrepared` retains the existing rule that an operational failure returns no +partial `RunResult`; the consumer already has independent preparation details. +A completed content-validation failure remains a successful result with +`ValidationFailed`. + +`PrepareExecution` preserves the public error categories of `Prepare`. +`RunPrepared` preserves applicable invalid-request, credential, capacity, +generation, validation, collaborator, and cancellation identities without +reintroducing source-loading or rendering failures from frozen state. + +## Compatibility And Existing Workflows + +This feature is additive: + +- `Prepare` remains the simple preparation-only operation; +- `Run` remains the simple prepare-and-execute operation with its current + early-admission and error-ordering behavior; +- `PreparedRun` and `RunResult` retain their existing stable JSON + representations; +- model-client and artifact-reader extension interfaces remain unchanged; and +- backend routing, concurrency limits, queue capacities, and provider wire + behavior remain unchanged. + +The new workflow may share internal machinery with `Prepare` and `Run`, but it +must not change their observable behavior merely to simplify implementation. + +## Documentation + +The completed documentation set has these ownership boundaries: + +- exported declarations and GoDoc own the exact handle, method, lifecycle, + ownership, concurrency, credential, error, and cancellation contracts; +- the promptkit consumer guide explains when to use `Prepare`, `Run`, or the + two-phase prepared-execution workflow; and +- internal runner, source-validation, capacity, and model-client documentation + describe the implemented collaborator boundaries without duplicating public + contracts. + +No release document is part of the feature implementation itself. Release +guidance is prepared only when the resulting public API is selected for +publication. + +## Non-Goals + +This work does not include: + +- serializable, durable, resumable, or cross-process execution handles; +- reuse for multiple consumer-initiated executions; +- concurrent execution of one handle; +- capacity reservation during preparation; +- a background task queue, priorities, worker lifecycle, or job status; +- retry, backoff, or provider failover policy; +- a separate `RunDetailed` convenience method; +- adding preparation details or rendered messages to every `RunResult`; +- prompt or profile inspection APIs; +- structured capacity or generation errors; +- freezing environment-variable credential values for the handle lifetime; +- snapshotting provider state or mutable behavior inside an injected + `LLMClient`; +- allowing target, validation, session, variable, or input overrides after + preparation; or +- changing existing `Prepare`, `Run`, file-format, provider-request, or stable + JSON contracts. + +## Target End State + +After this work: + +- consumers can perform and persist preflight before starting provider work; +- one prepared handle executes exactly the source-derived prompt, target, + schema, inputs, session, and messages described by its public details; +- execution never reloads or rerenders consumer sources; +- direct credentials remain confined to opaque, explicitly discardable state; +- environment credentials are not retained across the preflight boundary; +- backend capacity is reserved only when execution begins; +- one handle can start at most one execution attempt, including any internal + repair calls owned by that attempt; +- preparation details remain available after execution success or failure; +- existing simple `Prepare` and `Run` consumers remain unaffected; and +- Promptkit continues to own reusable execution mechanics without taking on + downstream persistence, redaction, retry, or job-management policy. diff --git a/docs/roadmap/weatherreporter-promptkit-wishlist.md b/docs/roadmap/weatherreporter-promptkit-wishlist.md index 58cb58d..7b69ffd 100644 --- a/docs/roadmap/weatherreporter-promptkit-wishlist.md +++ b/docs/roadmap/weatherreporter-promptkit-wishlist.md @@ -20,7 +20,7 @@ overlapping features from another downstream consumer's perspective. ## Priority 1: Executable Preparation Handles **Disposition:** Accepted into the -[future catalog](future.md#executable-preparation-handles). +[executable preparation handles](prepared-execution.md) feature roadmap. ### Downstream need