promptkit/docs/internal/runner.md

Internal Runner

Purpose

This document describes Promptkit's implemented internal orchestration. The architecture policy owns dependency and consumer boundaries. The source and validation document owns configuration, artifact, rendering, and validation behavior, while the model-client document owns generation behavior and failure categories.

The runner remains under internal/usecase and is assembled by internal/engine. Its concrete type is not part of the public API. The framework format reference owns prompt, profile, schema, and override semantics consumed by the runner.

Collaborators

Runner coordinates narrow internal interfaces for captured prompt definitions and resolved profiles, artifacts, rendering, model generation, and validation. The internal engine supplies one engine-local run admitter and a generation coordinator using the same capacity manager. Validation plans and provider-facing schema metadata come from the validator's preparation interface. The internal engine supplies one default output repairer through the explicit runner constructor, using the same coordinator as initial generation. The no-repair runner constructor remains available for focused internal callers and tests.

Each invocation carries its state in request, prepared-run, and result values. The runner has no durable run or session store.

Shared Prompt Selection

Ordinary preparation selects the full prompt definition and construction-time hash. It retains its early request-ID check before direct-session normalization. Referenced message content and static template syntax were captured when the engine was built.

InspectPrompt selects captured metadata through an optional repository capability, falling back to the full definition and existing hash resolution for internal repositories without it. It returns owned metadata without rendering templates or touching profile, artifact, schema, renderer, validator, admission, or model collaborators. The root Engine.InspectPrompt GoDoc owns the public operation's exact contract.

InspectRequirements uses the same metadata selection to resolve the request-profile/default identity, then obtains the captured requirement snapshot by the selected prompt ID. The analyzer receives only those identity fields rather than an execution definition. It analyzes the snapshot with the supplied caller variables and normalized direct-session bypass; it does not resolve target readiness or enter the preparation pipeline. Missing snapshot capability is a prompt-load failure rather than a source fallback. The root Engine.InspectRequirements GoDoc owns the public operation's exact contract.

Shared Profile Selection

The runner uses one profile-selection and target-resolution boundary for ordinary preparation and exact profile inspection. Preparation first selects a request profile or a prompt default; inspection begins with its required explicit profile ID. Both use the captured, presence-aware target constructed from the final effective profile and backend configuration. Request overrides apply afterward; preparation checks endpoint and model completeness only after those overrides. Inspection can report a partial profile without requiring a runnable target. Neither path checks credential availability or performs prompt, artifact, schema, rendering, admission, or model-client work. The root Engine.InspectProfile GoDoc owns the public operation's exact contract.

Smoke-Test Diagnostics

The separate SmokeTester in internal/usecase selects from the same captured configuration snapshot and backend/profile collaborators as the runner. Explicit profile selection takes precedence over prompt-default selection and a backend diagnostic default; HTTP-only selection does not resolve a default inference profile. Exact public selection rules belong to the root Engine.SmokeTest GoDoc.

Diagnostics do not call Run, Prepare, or the prompt preparation pipeline. They do not render application messages, load application artifacts, use application schemas, or inspect session templates. Private text and structured cases and the structured case's schema remain internal to the diagnostic operation; they are not entries in prompt/profile listings or inspection. InspectConfiguration remains a separate, explicit operator-oriented view of effective settings and source history. A diagnostic report may include selected effective settings and identifiers, so consumers still apply their disclosure policy.

The engine assembles the smoke tester with the runner's generation coordinator and capacity manager. Inference holds the ordinary logical admission lease and uses the same active permits, retries, timeout, and request evidence path as generation. Reachability and authentication probes bypass generation capacity and do not require a model. An engine with an injected LLMClient can run inference diagnostics but has no built-in HTTP probe collaborator; HTTP-only diagnostics report that limitation instead of falling back to another client. The public GoDoc owns the caller-visible timeout, cancellation, report, and transport semantics.

Shared Preparation Pipeline

Prepare and Run share one private preparation pipeline split at the point where a run can be assigned to its selected backend pool. The resolution phase performs only the work needed to validate routing and admission:

1. validate the required prompt selection and normalize any direct session ID;
2. select the captured prompt definition and hash;
3. select the request profile or the prompt's default profile;
4. retain the profile's selected backend identity, when present;
5. apply explicit request overrides to its captured backend-completed target;
6. validate endpoint, model, numeric overrides, and credential requirements;
7. resolve and validate the effective output contract without loading its schema; and
8. retain the definition, source identities, effective settings, output contract, and preparation start time in invocation-local state.

The completion phase consumes that state without reloading the prompt, profile, or backend:

1. create one operation-local validation plan and derive structured-output schema metadata from it when required;
2. load and hash input artifacts;
3. render messages and the prompt-defined session;
4. apply any direct session ID, then append already-normalized request messages after the rendered definition messages;
5. hash the effective rendered prompt; and
6. construct the prepared value and preparation timing.

Prepare runs both phases consecutively and never performs capacity admission. Run performs backend admission between the phases. This structure preserves one execution-precedence and error-ordering implementation while allowing a full backend pool to reject work before expensive schema, artifact, and rendering operations. Prepare discards the plan after returning its public metadata. Run retains the plan through initial and repaired-output validation and discards it when the operation ends. Prepared execution transfers the completed preparation's owned message, input-hash, and copied-schema values into its private payload, while deep-copying mutable provider extras and creating an independent details snapshot. Claim completion or discard clears the execution payload without removing the details snapshot.

The root facade's Engine.ValidateRequest uses the same complete preparation path as Prepare and discards the resulting internal prepared value without converting it into a public snapshot. It does not add a runner operation or validator component; request conversion and public error mapping remain at the facade boundary.

Pointer-based numeric overrides preserve an explicit zero. Invalid negative or out-of-range values fail as invalid requests. Endpoint overrides do not change the selected backend identity. Non-empty extra-parameter maps replace whole lower-precedence maps. A direct API key takes precedence over environment lookup; otherwise request, explicit profile, and backend environment-variable names apply in that order. A profile requiring a direct key suppresses an inherited backend environment name unless the request supplies its own name. Secret values remain excluded from serialized metadata.

Reasoning overrides are tri-state: nil inherits the profile, a pointer to a nonblank string trims and replaces it, and a pointer to a blank string clears it. A nonblank direct session is normalized before prompt selection, bypasses the prompt session template, and is applied after ordinary message rendering. A blank direct value retains prompt-template behavior. The runner clears the template only on a value copy of the definition, so the definition hash always describes captured effective content while the rendered-prompt hash includes the effective direct or rendered session and the complete effective message sequence. The rendered-prompt hash uses a versioned, length-framed SHA-256 encoding of that session, every role and content value, and cache-control presence and values; its hexadecimal value is opaque.

The configuration snapshot is read-only after engine construction. Concurrent Prepare and Run calls copy their resolved targets and keep all invocation state local.

Run Flow

Run records its start time, performs the shared resolution phase, and asks its RunAdmitter to reserve capacity for the effective backend ID. A nil admitter is an internal unlimited fallback. After successful admission, Run immediately defers the returned release function, performs the completion phase, makes one initial generation call, builds the named output artifact, and validates that artifact with the plan compiled during completion. Invalid generated content remains a validation result; an inability to generate or perform validation is an operational error.

When the public facade installs an invocation-local observation recorder, ordinary Run reuses its run ID and carries it to the generation coordinator through context. Prompt and profile identities are enriched only after their respective resolutions succeed, before admission. The facade starts observation before request conversion and emits its terminal event after public error mapping and after the runner's deferred admission release. Preparation and prepared execution follows the same terminal mapping after a successful claim; invalid or discarded handles do not acquire an observer.

Validation preparation and execution honor cancellation at every Promptkit-controlled boundary and do not publish a partial plan or result. Schema reads are bounded and context-checked between chunks; JSON decoding, schema compilation, and schema execution are checked immediately before and after their synchronous calls. Promptkit does not move arbitrary filesystem or JSON Schema work to background goroutines, so an already-blocked dependency method must return before cancellation can take precedence over its outcome.

The admission lease covers completion-phase preparation, initial generation, validation, every repair, and every exit. It bounds accepted work without serializing preparation or validation behind the active-generation limit. The coordinator separately acquires a FIFO active permit only around each actual model attempt. It releases the permit before a transient retry wait.

After a failed basic, JSON, or JSON Schema validation with a positive frozen budget, the installed repairer can make a bounded corrective call. Each request starts with a fresh copy of the complete effective message sequence (the configured prefix followed by the request suffix), includes only the latest nonempty candidate as an assistant message, and appends one corrective user message. Empty candidates omit that assistant message. The correction carries validation diagnostics as JSON data bounded to 64 KiB; the full diagnostics remain in the validation result.

Repair receives the effective execution target, explicit numeric-presence bits, credential, backend identity, session ID, structured-output specification, and frozen transient retry budget. The same request constructor supplies those common fields to initial and repair generation. The default repairer uses the same coordinator as initial generation, so each repair reacquires the selected backend's active permit while remaining inside its original admission lease. Repair never performs a second bounded admission, and repaired outputs use the operation's existing validation plan. The runner stops at the first valid candidate, sums completed generation usage, reports calls actually started, and returns the final failed validation result on exhaustion. A repair generation failure follows the ordinary generation-error category rather than becoming a validation error.

A successful result includes the output artifact and raw output, validation state, effective session ID, prompt and rendered-prompt hashes, selected profile and backend, effective settings, input hashes, token usage, a generated run identifier, and UTC timing. The same effective session reaches initial generation and any repair attempt through the rendered prompt. The same effective target and presence metadata, including backend identity and direct credential during execution, reaches generation and every repair attempt. Result usage is the field-wise sum of all five usage values from the initial response and every completed repair response. Final raw output, artifact, and validation state still come from the last candidate. A repair error returns no partial run result or partial result usage, although the invocation-local observer retains completed-attempt usage and reports it in the terminal event.

Failure Categories

Package errors distinguish invalid requests, required profile selection, credential failures, and prompt, profile, artifact, rendering, generation, and validation failures. Wrapping preserves the package identities mapped by the public facade and retains collaborator identities where they are part of the internal contract.

Admission capacity exhaustion retains the internal capacity identity. At the use-case boundary, the runner attaches the selected backend ID in an internal typed error, and the root facade copies that value into the public CapacityError without parsing diagnostic text. It is not recategorized as an invalid request or generation failure, and no partial result is returned. A context already done at admission retains its context identity directly. Cancellation while waiting for an active generation permit prevents client invocation when it wins the grant race; the model-client boundary then preserves the context error through the generation-failure category. Deferred release restores the admission lease on preparation, generation, validation, repair, and cancellation failures.

Other context cancellation propagates through the invoked collaborator and is classified by the owning operation. In particular, cancellation observed by validation retains the context identity through the validation error category. An overlong direct session is an invalid request before prompt selection, while an invalid or overlong prompt session template remains a prompt-render failure. An unknown selected backend, or a selected backend with no configured resolver, is classified as a profile-load failure.

Test Ownership And Changes

The runner tests own preparation order, selection and override precedence, the two-phase boundary, early admission, lease lifetime and release, direct-session resolution, schema-before-generation behavior, hashing, generation and validation outcomes, backend propagation, bounded repair progression, initial/repair request parity, cumulative usage, shared initial/repair capacity, credentials and redaction, error categories, artifact metadata, and timing. The capacity subsystem document identifies the focused pool and waiter tests. The generation document owns attempt and retry coordination tests.

Changes to orchestration should continue to use the existing package interfaces, keep request state local to an invocation, and preserve the shared resolution and completion pipeline. Source, renderer, validator, or model-client contract changes belong first in their owning package and document. The smoke selection tests, diagnostic orchestration tests, and private inference-case tests own diagnostic selection, report assembly, and inference findings.