promptkit/docs/internal/runner.md

Internal Runner

Purpose

This document describes Promptkit's implemented internal orchestration. The architecture policy owns dependency and consumer boundaries. The source and validation document owns repository, artifact, rendering, and validation behavior, while the model-client document owns generation behavior and failure categories.

The runner remains under internal/usecase and is assembled by the root Promptkit engine. Its concrete type is not part of the public API. The framework format reference owns prompt, profile, schema, and override semantics consumed by the runner.

Collaborators

Runner coordinates narrow internal interfaces for prompt definitions, profiles, backend resolution, artifacts, rendering, model generation, and validation. The root engine supplies an immutable built-in backend registry. Schema documents are loaded through the validator's optional schema-loader interface. An output repairer can be injected internally, but the ordinary runner constructor does not enable one.

Each invocation carries its state in request, prepared-run, and result values. The runner has no durable run or session store.

Preparation Flow

Prepare performs the reusable pre-generation workflow:

1. validate the prompt selection and load the prompt definition;
2. hash the loaded definition;
3. select the request profile or the prompt's default profile;
4. resolve the profile's backend ID, when present;
5. resolve application-neutral defaults, backend defaults, profile values, and explicit request overrides in that order;
6. validate endpoint, model, numeric overrides, and credential requirements;
7. resolve the output contract and load a structured-output schema when required;
8. load and hash input artifacts;
9. render and hash the prompt; and
10. return the effective settings, source identities, messages, hashes, and preparation timing.

Pointer-based numeric overrides preserve an explicit zero. Invalid negative or out-of-range values fail as invalid requests. Endpoint overrides do not change the selected backend identity. Non-empty extra-parameter maps replace whole lower-precedence maps. A direct API key takes precedence over environment lookup; otherwise request, profile, and backend environment-variable names apply in that order. A profile requiring a direct key clears an inherited backend environment name unless the request supplies its own name. Secret values remain excluded from serialized metadata.

Run Flow

Run calls Prepare rather than maintaining a second preparation path. It performs one initial generation call, builds the named output artifact, and validates that artifact. Invalid generated content remains a validation result; an inability to perform validation is an operational error.

When an internal repairer is present, a JSON or JSON Schema content failure can trigger bounded repair attempts. Repair receives the effective execution target, validation errors, prior output, and structured-output specification. This capability remains internal and is not a public option.

A successful result includes the output artifact and raw output, validation state, prompt and rendered-prompt hashes, selected profile and backend, effective settings, input hashes, token usage, a generated run identifier, and UTC timing. The same effective target, including backend identity, reaches generation and any repair attempt.

Failure Categories

Package errors distinguish invalid requests, required profile selection, credential failures, and prompt, profile, artifact, rendering, generation, and validation failures. Wrapping preserves the package identities mapped by the public facade and retains collaborator identities where they are part of the internal contract. Context cancellation propagates through the invoked collaborator and is classified by the owning operation. An unknown selected backend, or a selected backend with no configured resolver, is classified as a profile-load failure.

Test Ownership And Changes

The runner tests own preparation order, selection and override precedence, schema-before-generation behavior, hashing, generation and validation outcomes, backend propagation, bounded repair, credentials and redaction, error categories, artifact metadata, usage, and timing.

Changes to orchestration should continue to use the existing package interfaces, keep request state local to an invocation, and preserve Run's use of Prepare. Source, renderer, validator, or model-client contract changes belong first in their owning package and document.