147 lines
4.3 KiB
Markdown
147 lines
4.3 KiB
Markdown
# Runner Internals
|
|
|
|
## Purpose
|
|
|
|
`internal/usecase.Runner` is the core use case orchestrator for prompt preparation and execution.
|
|
|
|
It owns request validation, prompt/profile resolution, runtime-parameter merge, artifact loading, prompt rendering, structured-output setup, LLM invocation, output validation, and result metadata.
|
|
|
|
## Inputs And Outputs
|
|
|
|
Primary input type:
|
|
|
|
- `domain.RunRequest`
|
|
|
|
Primary output types:
|
|
|
|
- `domain.PreparedRun` from `Prepare`
|
|
- `domain.RunResult` from `Run`
|
|
|
|
LLM boundary types:
|
|
|
|
- `domain.GenerateRequest`
|
|
- `domain.GenerateResponse`
|
|
|
|
## Boundaries
|
|
|
|
`Runner` coordinates the following interfaces:
|
|
|
|
- `promptdef.Repository`
|
|
- `profile.Repository`
|
|
- `artifact.Reader`
|
|
- `prompt.Renderer`
|
|
- `llm.Client`
|
|
- `validate.Validator`
|
|
- optional `usecase.OutputRepairer`
|
|
|
|
Transport concerns (CLI flags, HTTP DTO parsing, status-code mapping) stay outside runner.
|
|
|
|
## Config Fields Used
|
|
|
|
`Runner` does not read app config files directly.
|
|
|
|
It receives fully constructed repositories/readers/validators from adapters. Effective behavior depends on adapter wiring, including:
|
|
|
|
- prompt/profile directories
|
|
- schema base directory
|
|
- selected profile/runtime overrides in request
|
|
|
|
## External Adapters Used
|
|
|
|
`Runner` works with adapter implementations via interfaces. Current wiring from CLI/HTTP uses:
|
|
|
|
- filesystem prompt/profile repositories
|
|
- composite artifact reader
|
|
- Go-template prompt renderer
|
|
- OpenAI-compatible LLM client
|
|
- standard validator
|
|
|
|
## State And Resume Behavior
|
|
|
|
`Runner` is stateless across requests.
|
|
|
|
- No durable run-state storage.
|
|
- No built-in resume/skip checkpoints.
|
|
- Each `Run`/`Prepare` executes from request inputs and current repositories.
|
|
|
|
## Failure Behavior
|
|
|
|
Key error classes surfaced from `Runner`:
|
|
|
|
- `ErrInvalidRequest`: invalid prompt/profile/request/runtime/API-key-env prerequisites.
|
|
- `ErrProfileLoad`: prompt or profile load failures.
|
|
- `ErrArtifactLoad`: artifact read failures.
|
|
- `ErrPromptRender`: template render failures.
|
|
- `ErrLLMGenerate`: model request failures.
|
|
- `ErrValidation`: validation runtime failures (including schema load/compile failures).
|
|
|
|
Validation content failures are not run errors:
|
|
|
|
- `Run` can succeed with `Validation.Status == failed`.
|
|
- CLI maps this to exit code `2`.
|
|
- HTTP returns `200` with failed validation details.
|
|
|
|
## Prepare Flow
|
|
|
|
`Prepare` performs:
|
|
|
|
1. validate request basics (prompt ID present).
|
|
2. load prompt definition by ID/version.
|
|
3. select profile ID:
|
|
- explicit request profile ID
|
|
- prompt `default_profile`
|
|
- otherwise request error
|
|
4. load execution profile.
|
|
5. merge effective runtime target:
|
|
- built-in execution defaults
|
|
- selected profile values
|
|
- request overrides
|
|
6. verify required `api_key_env` environment variable (name only; value is not returned).
|
|
7. resolve output contract and structured-output schema payload when `json_schema` mode is active.
|
|
8. read input artifacts.
|
|
9. render prompt messages.
|
|
10. compute prompt/input/render hashes and return `PreparedRun`.
|
|
|
|
`Prepare` does not call the LLM.
|
|
|
|
## Run Flow
|
|
|
|
`Run` performs:
|
|
|
|
1. generate run ID.
|
|
2. call `Prepare`.
|
|
3. call LLM with prepared messages/effective target/structured-output spec.
|
|
4. build output artifact content type from output format.
|
|
5. validate output.
|
|
6. optionally attempt bounded repair when repairer is injected and contract allows it.
|
|
7. return `RunResult` with artifact, raw output, validation, hashes, profile/model metadata, usage, and timestamps.
|
|
|
|
## Repair Hook Boundary
|
|
|
|
Repair attempts occur only when all are true:
|
|
|
|
- repairer is injected
|
|
- `repair_attempts > 0`
|
|
- validation status is `failed`
|
|
- validation mode is `json` or `json_schema`
|
|
|
|
Current production wiring boundary:
|
|
|
|
- CLI and HTTP adapters call `usecase.NewRunner(...)` (no repairer argument).
|
|
- Therefore normal CLI/HTTP execution does not perform repair attempts today.
|
|
|
|
## Tests To Inspect Before Changing
|
|
|
|
- `internal/usecase/runner_test.go`
|
|
- `internal/usecase/integration_test.go`
|
|
- `internal/adapter/cli/run_test.go`
|
|
- `internal/adapter/http/handler_test.go`
|
|
|
|
## Architectural Invariants
|
|
|
|
- `Run` reuses `Prepare`; prepare logic is not duplicated.
|
|
- Effective API-key environment-variable name may appear; resolved secret value must not.
|
|
- Structured-output schema document must load before LLM call for `json_schema` mode.
|
|
- Repair loops are bounded by `repair_attempts` and repairer presence.
|
|
- Runner stays transport-agnostic.
|