102 lines
4.8 KiB
Markdown
102 lines
4.8 KiB
Markdown
# Internal Runner
|
|
|
|
## Purpose
|
|
|
|
This document describes Promptkit's implemented internal orchestration. The
|
|
[architecture policy](../policy/architecture.md) owns dependency and consumer
|
|
boundaries. The [source and validation document](sources.md) owns repository,
|
|
artifact, rendering, and validation behavior, while the
|
|
[model-client document](llm.md) owns generation behavior and failure
|
|
categories.
|
|
|
|
The runner remains under `internal/usecase` and is assembled by the root
|
|
Promptkit engine. Its concrete type is not part of the public API.
|
|
The [framework format reference](../formats.md) owns prompt, profile, schema,
|
|
and override semantics consumed by the runner.
|
|
|
|
## Collaborators
|
|
|
|
`Runner` coordinates narrow internal interfaces for prompt definitions,
|
|
profiles, backend resolution, artifacts, rendering, model generation, and
|
|
validation. The root engine supplies one immutable registry containing the
|
|
built-in backend and validated consumer additions. Schema documents are loaded
|
|
through the validator's optional schema-loader interface. An output repairer
|
|
can be injected internally, but the ordinary runner constructor does not
|
|
enable one.
|
|
|
|
Each invocation carries its state in request, prepared-run, and result values.
|
|
The runner has no durable run or session store.
|
|
|
|
## Preparation Flow
|
|
|
|
`Prepare` performs the reusable pre-generation workflow:
|
|
|
|
1. validate the prompt selection and load the prompt definition;
|
|
2. hash the loaded definition;
|
|
3. select the request profile or the prompt's default profile;
|
|
4. resolve the profile's backend ID, when present;
|
|
5. resolve application-neutral defaults, backend defaults, profile values,
|
|
and explicit request overrides in that order;
|
|
6. validate endpoint, model, numeric overrides, and credential requirements;
|
|
7. resolve the output contract and load a structured-output schema when
|
|
required;
|
|
8. load and hash input artifacts;
|
|
9. render and hash the prompt; and
|
|
10. return the effective settings, source identities, messages, hashes, and
|
|
preparation timing.
|
|
|
|
Pointer-based numeric overrides preserve an explicit zero. Invalid negative or
|
|
out-of-range values fail as invalid requests. Endpoint overrides do not change
|
|
the selected backend identity. Non-empty extra-parameter maps replace whole
|
|
lower-precedence maps. A direct API key takes precedence over environment
|
|
lookup; otherwise request, profile, and backend environment-variable names
|
|
apply in that order. A profile requiring a direct key clears an inherited
|
|
backend environment name unless the request supplies its own name. Secret
|
|
values remain excluded from serialized metadata.
|
|
|
|
The registry is read-only after engine construction. Concurrent `Prepare` and
|
|
`Run` calls resolve independent defensive backend values and keep all
|
|
invocation state local.
|
|
|
|
## Run Flow
|
|
|
|
`Run` calls `Prepare` rather than maintaining a second preparation path. It
|
|
performs one initial generation call, builds the named output artifact, and
|
|
validates that artifact. Invalid generated content remains a validation result;
|
|
an inability to perform validation is an operational error.
|
|
|
|
When an internal repairer is present, a JSON or JSON Schema content failure can
|
|
trigger bounded repair attempts. Repair receives the effective execution
|
|
target, validation errors, prior output, and structured-output specification.
|
|
This capability remains internal and is not a public option.
|
|
|
|
A successful result includes the output artifact and raw output, validation
|
|
state, prompt and rendered-prompt hashes, selected profile and backend,
|
|
effective settings, input hashes, token usage, a generated run identifier, and
|
|
UTC timing. The same effective target, including backend identity, reaches
|
|
generation and any repair attempt.
|
|
|
|
## Failure Categories
|
|
|
|
Package errors distinguish invalid requests, required profile selection,
|
|
credential failures, and prompt, profile, artifact, rendering, generation, and
|
|
validation failures. Wrapping preserves the package identities mapped by the
|
|
public facade and retains collaborator identities where they are part of the
|
|
internal contract. Context cancellation propagates through the invoked
|
|
collaborator and is classified by the owning operation.
|
|
An unknown selected backend, or a selected backend with no configured resolver,
|
|
is classified as a profile-load failure.
|
|
|
|
## Test Ownership And Changes
|
|
|
|
The [runner tests](../../internal/usecase/runner_test.go) own preparation order,
|
|
selection and override precedence, schema-before-generation behavior, hashing,
|
|
generation and validation outcomes, backend propagation, bounded repair,
|
|
credentials and redaction, error categories, artifact metadata, usage, and
|
|
timing.
|
|
|
|
Changes to orchestration should continue to use the existing package
|
|
interfaces, keep request state local to an invocation, and preserve `Run`'s use
|
|
of `Prepare`. Source, renderer, validator, or model-client contract changes
|
|
belong first in their owning package and document.
|