Resolve profiles through built-in backends
This commit is contained in:
@@ -136,7 +136,7 @@ A profile supplies model execution settings:
|
||||
|
||||
```yaml
|
||||
id: local-summary
|
||||
endpoint: http://localhost:8000/v1
|
||||
backend: openrouter
|
||||
model: example-model
|
||||
temperature: 0.2
|
||||
max_tokens: 500
|
||||
@@ -144,7 +144,6 @@ top_p: 0.95
|
||||
timeout_seconds: 90
|
||||
service_tier: flex
|
||||
reasoning_effort: medium
|
||||
api_key_env: EXAMPLE_API_KEY
|
||||
extra_params:
|
||||
provider_option: enabled
|
||||
```
|
||||
@@ -152,7 +151,8 @@ extra_params:
|
||||
| Field | Required | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `id` | yes | Non-empty profile identifier. IDs must be unique within one source. |
|
||||
| `endpoint` | yes | Non-empty OpenAI-compatible base URL, including an API version path when required. |
|
||||
| `backend` | unless `endpoint` is present | Backend registry ID. It is trimmed and registry membership is checked when the profile is prepared. |
|
||||
| `endpoint` | unless `backend` is present | Non-empty OpenAI-compatible base URL, including an API version path when required. When both connection fields are present, this overrides the backend endpoint without changing backend identity. |
|
||||
| `model` | yes | Non-empty provider model name. |
|
||||
| `temperature` | no | Number from 0 through 2. |
|
||||
| `max_tokens` | no | Integer zero or greater. |
|
||||
@@ -166,6 +166,9 @@ extra_params:
|
||||
Raw `api_key` is prohibited in profile YAML. Store only an environment
|
||||
variable name in `api_key_env`.
|
||||
|
||||
Promptkit does not infer a backend from a model or endpoint. Endpoint-only
|
||||
profiles remain supported and have no effective backend ID.
|
||||
|
||||
`extra_params` accepts null, booleans, finite numbers, strings, arrays, and
|
||||
objects with string keys. Keys must be non-empty. With the built-in client,
|
||||
they also cannot collide with the standard fields listed in the
|
||||
@@ -176,8 +179,9 @@ they also cannot collide with the standard fields listed in the
|
||||
Execution settings resolve in this order:
|
||||
|
||||
1. framework defaults;
|
||||
2. the selected profile; and
|
||||
3. request `ExecutionTargetOverride` values.
|
||||
2. the selected backend, when the profile names one;
|
||||
3. the selected profile; and
|
||||
4. request `ExecutionTargetOverride` values.
|
||||
|
||||
The framework defaults are:
|
||||
|
||||
@@ -194,8 +198,10 @@ explicit zero is preserved. In particular, an explicit request
|
||||
`timeout_seconds` of zero disables the per-generation deadline while leaving
|
||||
the caller context and transport timeout intact.
|
||||
|
||||
Non-empty request strings replace profile strings. A non-empty request
|
||||
`ExtraParams` map replaces the profile map rather than merging keys.
|
||||
Non-empty profile strings replace backend defaults, and non-empty request
|
||||
strings replace both. Backend identity is retained when either layer overrides
|
||||
the endpoint. A non-empty `extra_params` map at each layer replaces the entire
|
||||
lower-precedence map rather than merging keys.
|
||||
The [outbound integration contract](integrations/openai-compatible-chat.md)
|
||||
defines how the effective settings are serialized.
|
||||
|
||||
@@ -217,9 +223,11 @@ invalid matching profile is an error and does not fall back. In-memory
|
||||
|
||||
## Built-In Profile Catalog
|
||||
|
||||
Built-ins use the OpenRouter-compatible endpoint and
|
||||
`OPENROUTER_API_KEY`. A custom or in-memory profile with the same ID takes
|
||||
precedence.
|
||||
Every built-in selects the `openrouter` backend. The engine's built-in backend
|
||||
registry supplies `https://openrouter.ai/api/v1` and the environment-variable
|
||||
name `OPENROUTER_API_KEY`, so individual profiles contain only model and
|
||||
generation settings. A custom or in-memory profile with the same profile ID
|
||||
takes precedence.
|
||||
|
||||
| Provider | ID | Model |
|
||||
| --- | --- | --- |
|
||||
@@ -270,6 +278,10 @@ prompt, profile, schema, or example files:
|
||||
- a request can provide a direct `APIKey` or override `APIKeyEnv`; and
|
||||
- a direct request key takes precedence over environment lookup.
|
||||
|
||||
After a direct request key, the credential-source precedence is request
|
||||
`APIKeyEnv`, profile `api_key_env`, then the backend default. An in-memory
|
||||
profile with `APIKeyRequired` clears an inherited backend environment name and
|
||||
requires a direct key unless the request explicitly supplies `APIKeyEnv`.
|
||||
Promptkit validates required credential availability during preparation.
|
||||
Direct keys are excluded from JSON results and redacted by public string
|
||||
formatters. Environment-variable names may appear in prepared metadata, but
|
||||
|
||||
@@ -17,10 +17,11 @@ and override semantics consumed by the runner.
|
||||
## Collaborators
|
||||
|
||||
`Runner` coordinates narrow internal interfaces for prompt definitions,
|
||||
profiles, artifacts, rendering, model generation, and validation. Schema
|
||||
documents are loaded through the validator's optional schema-loader interface.
|
||||
An output repairer can be injected internally, but the ordinary runner
|
||||
constructor does not enable one.
|
||||
profiles, backend resolution, artifacts, rendering, model generation, and
|
||||
validation. The root engine supplies an immutable built-in backend registry.
|
||||
Schema documents are loaded through the validator's optional schema-loader
|
||||
interface. An output repairer can be injected internally, but the ordinary
|
||||
runner constructor does not enable one.
|
||||
|
||||
Each invocation carries its state in request, prepared-run, and result values.
|
||||
The runner has no durable run or session store.
|
||||
@@ -32,20 +33,25 @@ The runner has no durable run or session store.
|
||||
1. validate the prompt selection and load the prompt definition;
|
||||
2. hash the loaded definition;
|
||||
3. select the request profile or the prompt's default profile;
|
||||
4. resolve application-neutral defaults, profile values, and explicit request
|
||||
overrides in that order;
|
||||
5. validate endpoint, model, numeric overrides, and credential requirements;
|
||||
6. resolve the output contract and load a structured-output schema when
|
||||
4. resolve the profile's backend ID, when present;
|
||||
5. resolve application-neutral defaults, backend defaults, profile values,
|
||||
and explicit request overrides in that order;
|
||||
6. validate endpoint, model, numeric overrides, and credential requirements;
|
||||
7. resolve the output contract and load a structured-output schema when
|
||||
required;
|
||||
7. load and hash input artifacts;
|
||||
8. render and hash the prompt; and
|
||||
9. return the effective settings, source identities, messages, hashes, and
|
||||
8. load and hash input artifacts;
|
||||
9. render and hash the prompt; and
|
||||
10. return the effective settings, source identities, messages, hashes, and
|
||||
preparation timing.
|
||||
|
||||
Pointer-based numeric overrides preserve an explicit zero. Invalid negative or
|
||||
out-of-range values fail as invalid requests. A direct API key takes
|
||||
precedence over environment lookup for execution; secret values remain
|
||||
excluded from serialized metadata.
|
||||
out-of-range values fail as invalid requests. Endpoint overrides do not change
|
||||
the selected backend identity. Non-empty extra-parameter maps replace whole
|
||||
lower-precedence maps. A direct API key takes precedence over environment
|
||||
lookup; otherwise request, profile, and backend environment-variable names
|
||||
apply in that order. A profile requiring a direct key clears an inherited
|
||||
backend environment name unless the request supplies its own name. Secret
|
||||
values remain excluded from serialized metadata.
|
||||
|
||||
## Run Flow
|
||||
|
||||
@@ -60,8 +66,10 @@ target, validation errors, prior output, and structured-output specification.
|
||||
This capability remains internal and is not a public option.
|
||||
|
||||
A successful result includes the output artifact and raw output, validation
|
||||
state, prompt and rendered-prompt hashes, selected profile, effective settings,
|
||||
input hashes, token usage, a generated run identifier, and UTC timing.
|
||||
state, prompt and rendered-prompt hashes, selected profile and backend,
|
||||
effective settings, input hashes, token usage, a generated run identifier, and
|
||||
UTC timing. The same effective target, including backend identity, reaches
|
||||
generation and any repair attempt.
|
||||
|
||||
## Failure Categories
|
||||
|
||||
@@ -71,13 +79,16 @@ validation failures. Wrapping preserves the package identities mapped by the
|
||||
public facade and retains collaborator identities where they are part of the
|
||||
internal contract. Context cancellation propagates through the invoked
|
||||
collaborator and is classified by the owning operation.
|
||||
An unknown selected backend, or a selected backend with no configured resolver,
|
||||
is classified as a profile-load failure.
|
||||
|
||||
## Test Ownership And Changes
|
||||
|
||||
The [runner tests](../../internal/usecase/runner_test.go) own preparation order,
|
||||
selection and override precedence, schema-before-generation behavior, hashing,
|
||||
generation and validation outcomes, bounded repair, credentials and redaction,
|
||||
error categories, artifact metadata, usage, and timing.
|
||||
generation and validation outcomes, backend propagation, bounded repair,
|
||||
credentials and redaction, error categories, artifact metadata, usage, and
|
||||
timing.
|
||||
|
||||
Changes to orchestration should continue to use the existing package
|
||||
interfaces, keep request state local to an invocation, and preserve `Run`'s use
|
||||
|
||||
@@ -24,13 +24,19 @@ duplicate detection, and source containment:
|
||||
|
||||
`internal/profile` loads and validates execution profiles from an
|
||||
operating-system filesystem or an `fs.FS`. It supports a primary repository
|
||||
with fallback only when the primary reports that a profile is absent.
|
||||
with fallback only when the primary reports that a profile is absent. Strict
|
||||
YAML decoding recognizes the optional `backend` field, trims its value, and
|
||||
requires a model plus at least one non-blank backend or endpoint. Loading does
|
||||
not check registry membership because the available registry belongs to the
|
||||
assembled engine; the runner checks membership during preparation.
|
||||
|
||||
`internal/profile/builtin` embeds the maintained built-in profile catalog and
|
||||
can place a caller-selected repository ahead of that catalog. Profile behavior
|
||||
is owned by the
|
||||
can place a caller-selected repository ahead of that catalog. Every embedded
|
||||
profile selects `openrouter` and inherits its endpoint and credential
|
||||
environment-variable name from the built-in backend registry rather than
|
||||
repeating those values. Profile behavior is owned by the
|
||||
[profile repository tests](../../internal/profile/repository_test.go), while
|
||||
catalog completeness, duplicate IDs, and overlay behavior are owned by the
|
||||
catalog completeness, backend-selection invariant, duplicate IDs, and overlay behavior are owned by the
|
||||
[built-in repository tests](../../internal/profile/builtin/repository_test.go).
|
||||
|
||||
## Ordinary Artifacts
|
||||
|
||||
@@ -299,6 +299,8 @@ immutable and the final repository race suite exercises concurrent reads.
|
||||
|
||||
## Stage 2 — Profile Selection And Effective Resolution
|
||||
|
||||
**Status:** Complete.
|
||||
|
||||
### Goal
|
||||
|
||||
Make the built-in OpenRouter backend selectable by file and in-memory profiles,
|
||||
|
||||
Reference in New Issue
Block a user