promptkit/docs/internal/runner.md

Internal Runner

Purpose

This document describes Promptkit's implemented internal orchestration. The architecture policy owns dependency and consumer boundaries. The source and validation document owns repository, artifact, rendering, and validation behavior, while the model-client document owns generation behavior and failure categories.

The runner remains under internal/usecase and is assembled by the root Promptkit engine. Its concrete type is not part of the public API. The framework format reference owns prompt, profile, schema, and override semantics consumed by the runner.

Collaborators

Runner coordinates narrow internal interfaces for prompt definitions, profiles, backend resolution, artifacts, rendering, model generation, and validation. The root engine supplies one immutable registry containing the built-in backend and validated consumer additions, one engine-local run admitter, and a model client wrapped by the same capacity manager. Schema documents are loaded through the validator's optional schema-loader interface. An output repairer can be injected internally, but the ordinary runner constructor does not enable one.

Each invocation carries its state in request, prepared-run, and result values. The runner has no durable run or session store.

Shared Prompt Selection

The runner uses one prompt-selection and hashing boundary for ordinary preparation and exact prompt inspection. Preparation retains its early request-ID check before direct-session normalization; both operations then use the configured prompt repository to select one definition, load referenced message content, and calculate the same prompt hash.

Inspection stops after that structural lookup. It does not parse templates or touch profile, artifact, schema, renderer, validator, admission, or model collaborators. The root Engine.InspectPrompt GoDoc owns the public operation's exact contract.

Shared Profile Selection

The runner uses one profile-selection and target-resolution boundary for ordinary preparation and exact profile inspection. Preparation first selects a request profile or a prompt default; inspection begins with its required explicit profile ID. Both then apply the ordinary source precedence, resolve a named backend, and construct the effective target from framework, backend, and profile values.

Inspection stops after the resulting endpoint and model are structurally validated. It does not check credential availability or perform prompt, artifact, schema, rendering, admission, or model-client work. The root Engine.InspectProfile GoDoc owns the public operation's exact contract.

Shared Preparation Pipeline

Prepare and Run share one private preparation pipeline split at the point where a run can be assigned to its selected backend pool. The resolution phase performs only the work needed to validate routing and admission:

1. validate the required prompt selection and normalize any direct session ID;
2. load the prompt definition and hash the original definition;
3. select the request profile or the prompt's default profile;
4. resolve the profile's backend ID, when present;
5. resolve application-neutral defaults, backend defaults, profile values, and explicit request overrides in that order;
6. validate endpoint, model, numeric overrides, and credential requirements;
7. resolve the effective output contract without loading its schema; and
8. retain the definition, source identities, effective settings, output contract, and preparation start time in invocation-local state.

The completion phase consumes that state without reloading the prompt, profile, or backend:

1. load structured-output schema metadata when required;
2. load and hash input artifacts;
3. render messages and the prompt-defined session;
4. apply any direct session ID;
5. hash the effective rendered prompt; and
6. construct the prepared value and preparation timing.

Prepare runs both phases consecutively and never performs capacity admission. Run performs backend admission between the phases. This structure preserves one execution-precedence and error-ordering implementation while allowing a full backend pool to reject work before expensive schema, artifact, and rendering operations.

Pointer-based numeric overrides preserve an explicit zero. Invalid negative or out-of-range values fail as invalid requests. Endpoint overrides do not change the selected backend identity. Non-empty extra-parameter maps replace whole lower-precedence maps. A direct API key takes precedence over environment lookup; otherwise request, profile, and backend environment-variable names apply in that order. A profile requiring a direct key clears an inherited backend environment name unless the request supplies its own name. Secret values remain excluded from serialized metadata.

Reasoning overrides are tri-state: nil inherits the profile, a pointer to a nonblank string trims and replaces it, and a pointer to a blank string clears it. A nonblank direct session is normalized before source loading, bypasses the prompt session template, and is applied after ordinary message rendering. A blank direct value retains prompt-template behavior. The runner clears the template only on a value copy of the definition, so the definition hash always describes the original source while the rendered-prompt hash includes the effective direct or rendered session.

The registry is read-only after engine construction. Concurrent Prepare and Run calls resolve independent defensive backend values and keep all invocation state local.

Run Flow

Run records its start time, performs the shared resolution phase, and asks its RunAdmitter to reserve capacity for the effective backend ID. A nil admitter is an internal unlimited fallback. After successful admission, Run immediately defers the returned release function, performs the completion phase, makes one initial generation call, builds the named output artifact, and validates that artifact. Invalid generated content remains a validation result; an inability to perform validation is an operational error.

The admission lease covers completion-phase preparation, initial generation, validation, every repair, and every exit. It bounds accepted work without serializing preparation or validation behind the active-generation limit. The wrapped model client separately acquires a FIFO active permit only around each actual generation call.

When an internal repairer is present, a JSON or JSON Schema content failure can trigger bounded repair attempts. Repair receives the effective execution target and session ID, validation errors, prior output, and structured-output specification. The default repairer uses the same wrapped client as initial generation, so each repair reacquires the selected backend's active permit while remaining inside its original admission lease. Repair never performs a second bounded admission. This capability remains internal and is not a public option.

A successful result includes the output artifact and raw output, validation state, effective session ID, prompt and rendered-prompt hashes, selected profile and backend, effective settings, input hashes, token usage, a generated run identifier, and UTC timing. The same effective session reaches initial generation and any repair attempt through the rendered prompt. The same effective target, including backend identity, reaches generation and any repair attempt.

Failure Categories

Package errors distinguish invalid requests, required profile selection, credential failures, and prompt, profile, artifact, rendering, generation, and validation failures. Wrapping preserves the package identities mapped by the public facade and retains collaborator identities where they are part of the internal contract.

Admission capacity exhaustion retains the internal capacity identity and adds the selected backend ID as context. It is not recategorized as an invalid request or generation failure, and no partial result is returned. A context already done at admission retains its context identity directly. Cancellation while waiting for an active generation permit prevents client invocation when it wins the grant race; the model-client boundary then preserves the context error through the generation-failure category. Deferred release restores the admission lease on preparation, generation, validation, repair, and cancellation failures.

Other context cancellation propagates through the invoked collaborator and is classified by the owning operation. An overlong direct session is an invalid request before source loading, while an invalid or overlong prompt session template remains a prompt-render failure. An unknown selected backend, or a selected backend with no configured resolver, is classified as a profile-load failure.

Test Ownership And Changes

The runner tests own preparation order, selection and override precedence, the two-phase boundary, early admission, lease lifetime and release, direct-session resolution, schema-before-generation behavior, hashing, generation and validation outcomes, backend propagation, bounded repair, shared initial/repair capacity, credentials and redaction, error categories, artifact metadata, usage, and timing. The capacity subsystem document identifies the focused pool, waiter, and wrapped-client tests.

Changes to orchestration should continue to use the existing package interfaces, keep request state local to an invocation, and preserve the shared resolution and completion pipeline. Source, renderer, validator, or model-client contract changes belong first in their owning package and document.