eric/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Prepare the backend registry implementation roadmap

Browse Source
This commit is contained in:
Eric Rakestraw
2026-07-29 16:50:36 +00:00
parent b462153483
commit d0010689f3
4 changed files with 684 additions and 243 deletions
Download Patch File Download Diff File Expand all files Collapse all files

180
docs/roadmap/backends.md Normal file
View File

@@ -0,0 +1,180 @@
# Extensible LLM Backend Registry
## Purpose
This roadmap defines the scope and target end state for formal
OpenAI-compatible backend support in Promptkit. It establishes the behavioral
boundary and policy choices for the work.
This document is planning material, not a description of current behavior.
Current public contracts remain owned by Go declarations and GoDoc, framework
file formats by the [format reference](../formats.md), and outbound HTTP
behavior by the
[OpenAI-compatible integration contract](../integrations/openai-compatible-chat.md).
## Motivation
Execution profiles currently carry backend connection and authentication
details alongside model and generation settings. This repeats values such as
the OpenRouter endpoint and `OPENROUTER_API_KEY` across profiles, leaves the
built-in catalog implicitly tied to one backend, and gives consumers no
explicit extension point for naming other OpenAI-compatible services.
Promptkit should distinguish:
- a **backend**, which identifies reusable connection, authentication, and
limited backend-wide request defaults; and
- a **profile**, which selects a model and its reusable generation settings.
That distinction should let consumers configure OpenRouter, OpenAI,
local-network services, or other OpenAI-compatible deployments without
duplicating backend defaults in every profile or replacing Promptkit's model
client.
## Scope
The work introduces an engine-scoped registry of OpenAI-compatible backend
definitions.
Each backend definition will provide:
- a stable, non-empty backend ID;
- a default OpenAI-compatible base endpoint;
- an optional environment-variable name for its API key; and
- limited JSON-compatible request defaults when they are genuinely
backend-wide rather than model-specific.
Promptkit will provide a small built-in backend catalog, initially containing
OpenRouter. Downstream consumers will be able to register additional backend
IDs during engine construction. Registration will be explicit and local to an
engine; it will not mutate package-global state.
Profiles will be able to name a backend. A profile that selects a backend may
override its default endpoint and other supported defaults while continuing to
own its model and generation settings. Existing endpoint-based custom profiles
will remain supported without requiring a backend ID.
The built-in model profile catalog will use the built-in OpenRouter backend
instead of repeating OpenRouter connection and credential defaults in every
profile.
## Resolution And Precedence
Effective execution settings will resolve in this order:
1. application-neutral framework defaults;
2. selected backend defaults, when a backend is named;
3. selected profile values; and
4. explicit per-run overrides.
A profile endpoint will override its selected backend's endpoint. A per-run
endpoint override will continue to take precedence over both.
Credential selection will preserve direct request credentials as the highest
precedence. An explicit per-run environment-variable override will take
precedence over profile credential configuration, which will take precedence
over the backend's default environment-variable name. Backend definitions and
profiles will contain credential-source metadata only, never resolved secret
values.
Backend request defaults, profile request values, and per-run request
overrides will follow one deterministic replacement rule. Resolution must not
introduce implicit deep merging whose result depends on map iteration or
incidental representation.
## Registration And Validation
Backend registration will be deterministic and validated during engine
construction.
- Built-in backend IDs are reserved and cannot be replaced by consumers.
- Consumer registrations may add only new IDs.
- Duplicate consumer IDs are invalid, including duplicates introduced through
repeated configuration.
- Backend IDs, endpoints, credential-source metadata, and request defaults
must be validated before an engine is returned.
- Selecting an unknown backend is an error associated with the profile or
request boundary that selected it.
- A profile without a backend must continue to provide the connection details
required by the current endpoint-based path.
- A backend may omit credential requirements so unauthenticated local-network
services remain supported.
Failures will retain error identities appropriate to the existing engine
configuration, profile-loading, and per-run validation boundaries.
## Public And Extension Boundaries
The effective backend ID will be observable in prepared and completed run
metadata and will be available to injected model clients. This gives consumers
and extensions a stable routing identity without requiring them to infer a
backend from an endpoint URL.
The existing public model-client injection boundary will remain supported.
Because the initial registry supports only OpenAI-compatible backends,
Promptkit does not need backend-specific transport factories or multiple
protocol implementations in this scope. The built-in client will continue to
send the resolved execution target to the selected OpenAI-compatible endpoint.
Backend definitions are configuration values, not live service objects. They
will not own mutable connections, credentials, health state, or process
lifecycle.
## Compatibility
The target end state preserves the existing ways consumers configure and run
Promptkit:
- Profiles with explicit endpoints and no backend ID continue to work.
- Profile endpoint overrides remain supported.
- Direct request API keys and environment-variable overrides retain their
precedence.
- Consumers may continue injecting a custom model client.
- Model IDs and generation settings remain profile concerns.
- Prompt selection, rendering, validation, artifacts, and synchronous
`Prepare` and `Run` behavior remain unchanged except for exposing the
resolved backend identity.
## Non-Goals
This scope does not include:
- non-OpenAI-compatible protocols or provider-specific SDKs;
- backend-specific concurrency limits or admission queues;
- asynchronous task submission or durable jobs;
- model discovery, live model catalogs, or provider capability probing;
- retries, rate limiting, failover, load balancing, or health checks;
- automatic backend selection based on model names or endpoint inspection;
- global mutable registration;
- consumer replacement of built-in backend IDs;
- credential discovery beyond configured direct values and environment
variables; or
- provider-specific application, deployment, or security policy.
The dependent concurrency and bounded-queue work remains a separate future
unit after backend identity and resolution are stable.
## Target End State
This roadmap reaches its target end state when:
- Promptkit has an immutable, engine-scoped registry of validated
OpenAI-compatible backend definitions;
- OpenRouter is available as a built-in backend with its endpoint and
credential environment-variable default;
- consumers can register additional unique backend IDs without modifying
Promptkit;
- profiles can select a backend and optionally override its endpoint;
- existing endpoint-only profiles remain valid;
- backend, profile, and per-run values resolve through documented,
deterministic precedence;
- secret values remain outside backend and profile definitions;
- the effective backend identity is available in preparation, execution, and
injected-client values;
- the built-in model profiles use the OpenRouter backend rather than duplicate
its connection defaults;
- the built-in OpenAI-compatible client and injected clients continue to work
through the existing generation boundary; and
- current-state GoDoc, format, consumer, integration, architecture, and
internal documentation describe the implemented behavior without relying on
this roadmap.

222
docs/roadmap/documentation.md
View File

@@ -1,222 +0,0 @@
# Documentation Hardening Roadmap
## Purpose
This roadmap coordinates a focused pass over Promptkit's documentation and
public contract. The work should close the gaps identified by the documentation
audit, strengthen canonical ownership, and leave consumers and contributors
with guidance that is accurate, navigable, and proportionate to their needs.
This is planning material, not a description of implemented behavior. Follow
the [documentation policy](../policy/documentation.md) throughout the work and
update current-state documents only when their claims are supported by the
implementation and tests.
## Scope And Principles
The work covers public GoDoc, consumer guidance, maintained examples, format
and integration references, architecture ownership, roadmap lifecycle, and
documentation validation.
- Resolve ambiguous behavior before documenting it as a contract.
- Keep exact exported API semantics in Go declarations and GoDoc.
- Keep task-oriented guidance in consumer documents and complete runnable
artifacts in `examples/`.
- Put security-relevant consumer responsibilities at the public boundary, not
only in internal contributor documents.
- Remove duplicated ownership instead of synchronizing parallel references.
- Make documentation checks reproducible where practical.
- Preserve existing behavior unless a stage explicitly selects and tests an
API change.
The roadmap does not add new Promptkit features, redesign framework formats,
or implement ideas from [the future feature catalog](future.md). If resolving
an ambiguity requires a behavioral change, treat that change as a separately
reviewable implementation unit and update its canonical documentation in the
same unit.
## Stage 1: Resolve Public Contract Questions
Before expanding prose, decide the intended contract for exported behavior
that is currently ambiguous.
- [x] Decide whether `RunRequest.Metadata` has a supported observable purpose.
Define its propagation and ownership, or remove or deprecate it through an
intentional public API change.
- [x] Decide and test whether one `Engine` supports concurrent `Prepare` and
`Run` calls.
- [x] Decide how repeated options of the same category behave, including
prompt, profile, schema, model-client, and artifact-reader options.
- [x] Define which public values have supported JSON representations.
- [x] Define JSON time units and omission behavior, including the relationship
between prepared-run and run-result durations.
- [x] Decide whether run IDs and exposed hashes have stable formats or must be
treated as opaque values.
- [x] Confirm the intended transport-timeout default and its zero or negative
configuration semantics.
- [x] Confirm the supported JSON Schema dialect and reference boundaries,
including whether remote references are allowed.
The selected exported API contracts are implemented and tested. Their durable
definitions now belong to the root package declarations and GoDoc.
One format-level decision remains here until Stage 5 moves it to the framework
format reference: JSON Schema uses Draft 2020-12, with that dialect selected
when `$schema` is omitted. Same-document fragments and relative references
contained by a directory or `fs.FS` schema root are supported. A single-file
source supports only references contained in that document. Absolute,
escaping, and remote references are rejected.
**Gate:** Each question has an explicit answer backed by existing behavior or
by an accepted implementation change and proportionate tests. No later stage
should invent a contract merely to fill a documentation gap.
## Stage 2: Make GoDoc The Canonical Public Contract
Strengthen the root package declarations so `go doc` is sufficient to
understand exact public behavior without relying on internal documents.
- [x] Add useful field-level GoDoc to configuration, request, profile,
execution-target, result, artifact, validation, structured-output, and model
client values.
- [x] Document required fields and nil, empty, and zero-value semantics.
- [x] Document override, replacement, profile-precedence, and copy-ownership
behavior where it belongs to the exported API.
- [x] Document credential inputs, redaction, and the values intentionally
excluded from serialization.
- [x] Give each public error sentinel an accurate comment and document the
supported `errors.Is` relationships.
- [x] Document engine concurrency and option-composition behavior selected in
Stage 1.
- [x] Document serialization, time, run-ID, and hash semantics selected in
Stage 1.
- [x] Review constructor, option, extension-interface, `Prepare`, and `Run`
GoDoc for complete failure and cancellation expectations.
Update the [consumer guide](../consumers/pkg-promptkit.md) to summarize and link
to these contracts instead of maintaining exhaustive copies of exported names
or exact semantics.
**Gate:** `go doc -all .` presents a coherent public contract, exported
declarations have accurate comments, and contract tests protect every newly
documented behavior whose compatibility risk warrants durable coverage.
## Stage 3: Improve Consumer Safety And Executable Guidance
Move consumer-relevant security boundaries to the places where consumers will
encounter them and add one representative execution workflow.
- [x] Explain in public GoDoc and the consumer guide that the default file
artifact reader accepts unrestricted caller-selected paths.
- [x] Make clear that Promptkit does not impose an application root, inbound
request-size policy, or untrusted-input security boundary.
- [x] Explain that rendered messages, artifact bodies, raw model output, and
validation details may be sensitive even when credentials are redacted.
- [x] Clarify the responsibilities of injected artifact readers and model
clients for cancellation, copying, logging, and secret handling.
- [x] Add a maintained offline `Run` example using an injected deterministic
model client, without credentials, live network access, or paid calls.
- [x] Link the consumer guide to the execution example and keep embedded
snippets smaller than the maintained artifact.
- [x] Decide whether the existing preparation example should remain separate
or share reusable fixtures without obscuring either workflow.
The preparation and execution examples remain separate, self-contained
workflows. Each keeps its own small prompt fixture so consumers can copy or run
one example without depending on the other.
**Gate:** Both preparation and execution have complete, secret-free,
deterministic consumer examples, and the consumer guide exposes the important
filesystem and data-sensitivity boundaries without leaking internal mechanics.
## Stage 4: Restore Canonical Ownership
Remove parallel definitions and make navigation follow the ownership model in
the documentation policy.
- [ ] Reduce the [architecture policy](../policy/architecture.md) to durable
boundaries, layers, dependency direction, invariants, and non-goals.
- [ ] Keep the exact implemented package and component inventory solely in the
[internal component overview](../internal/overview.md).
- [ ] Review the consumer guide's public error and option lists so they remain
task-oriented summaries rather than duplicate API references.
- [ ] Review internal documents for public-contract statements that should be
links to GoDoc or the format and integration owners.
- [ ] Reconcile the documentation policy's temporary-roadmap lifecycle with
the continuing idea-catalog role of `docs/roadmap/future.md`.
- [ ] Rephrase or link roadmap statements that depend on exact current API
behavior, particularly the runtime reasoning entry.
- [ ] Confirm that every document states its audience or purpose and links to
the canonical owner of adjacent topics.
**Gate:** Every authoritative fact has one clear owner, package inventory
changes no longer require edits to the architecture policy, and roadmaps
cannot be mistaken for current-state references.
## Stage 5: Refine Format And Integration References
Close compatibility gaps in the documents that own file formats and outbound
wire behavior.
- [ ] State or canonically link the exact session-ID limit enforced by the
OpenAI-compatible client.
- [ ] State the configured and default transport-timeout behavior without
referring to an unnamed internal default.
- [ ] Document the JSON Schema dialect and local, contained, and remote
reference behavior selected in Stage 1.
- [ ] Clarify structured-output naming and strictness when those values are
part of the public or integration contract.
- [ ] Add a caveat that built-in profiles are maintained configurations, not a
guarantee of continuing third-party model availability.
- [ ] Recheck every prompt, profile, schema, credential, request-body, response,
timeout, and precedence statement against its owning implementation and
tests.
**Gate:** A consumer can determine the supported file and wire compatibility
boundaries without consulting internal source code or relying on unspecified
defaults.
## Stage 6: Make Documentation Validation Reproducible
Align contributor and release procedures around a small, consistent set of
checks.
- [ ] Use one robust command for checking every tracked Go file with `gofmt`.
- [ ] Provide a repository-local or clearly documented command that validates
local Markdown targets and heading fragments.
- [ ] Decide how published external links are checked without making ordinary
validation depend on mutable network services.
- [ ] Reconcile the validation descriptions in the
[development guide](../development.md),
[testing policy](../policy/testing.md), and
[release procedure](../release.md) so one document owns each requirement.
- [ ] Ensure example validation covers every maintained example added by this
roadmap.
- [ ] Keep documentation-only validation proportionate while requiring full Go
validation when commands, examples, generated output, or checked behavior
changes.
**Gate:** A maintainer can run the documented formatting, link, example, Go,
and repository-hygiene checks exactly as written, with no hidden manual
procedure for local documentation.
## Completion Criteria
The roadmap is complete when:
- all Stage 1 contract questions are resolved;
- GoDoc is the authoritative and sufficient exported API reference;
- consumer guidance covers unrestricted file access and sensitive generated
data;
- maintained offline examples cover both `Prepare` and `Run`;
- architecture, inventory, consumer, internal, format, integration, and
roadmap documents follow their assigned ownership boundaries;
- schema, session, timeout, structured-output, and built-in-profile
compatibility statements are explicit;
- documentation validation is reproducible and consistent across contributor
and release workflows; and
- the complete maintainer validation passes.
After completion, move any durable decisions to GoDoc, policy, format,
integration, or ADR owners as appropriate. Remove this roadmap after incoming
links are updated; do not retain it as a second current-state reference.

28
docs/roadmap/future.md
View File

@@ -33,29 +33,15 @@ consumers.
## Ideas ## Ideas
### Extensible LLM backend registry
Introduce a registry that separates backend-specific connection,
authentication, and limited request defaults from model execution profiles.
Initial support would cover OpenAI-compatible backends and include a small
built-in catalog, potentially starting with OpenRouter. A profile could select
a backend while optionally overriding its default endpoint, and each backend
could name an optional environment variable for its API key without storing
the credential itself. Downstream consumers could register additional,
uniquely named backends, such as OpenAI or unauthenticated local-network
services, but could not replace built-in IDs. Model selection and generation
settings would remain profile concerns, and custom model clients would remain
available for behavior outside the registry's supported protocol.
### Backend-specific concurrency management ### Backend-specific concurrency management
Extend the proposed LLM backend registry with optional per-backend concurrency Extend the selected
limits and bounded, buffered admission queues. Promptkit could then route [LLM backend registry](backends.md) with optional per-backend concurrency limits
simultaneous generation requests according to backend capacity while and bounded, buffered admission queues. Promptkit could then route simultaneous
containing accidental runaway submission. Downstream consumers would continue generation requests according to backend capacity while containing accidental
invoking synchronous `Run` calls, including concurrently from multiple runaway submission. Downstream consumers would continue invoking synchronous
goroutines, and each admitted call would wait for and return its ordinary `Run` calls, including concurrently from multiple goroutines, and each
result. admitted call would wait for and return its ordinary result.
- Scope limits to an engine instance rather than hidden process-global state. - Scope limits to an engine instance rather than hidden process-global state.
- Give different backend IDs independent capacity pools. A profile endpoint - Give different backend IDs independent capacity pools. A profile endpoint

497
docs/roadmap/implementation.md Normal file
View File

@@ -0,0 +1,497 @@
# Extensible LLM Backend Registry Implementation Plan
## Purpose
This document is the decision-complete implementation plan for the
[extensible LLM backend registry](backends.md). It is written for a coding
agent that will implement each stage in order.
This is planning material, not a description of current behavior. During
implementation, update current-state GoDoc and documentation only as the
corresponding behavior lands. The feature roadmap owns the intended end state
and policy choices; this document owns API shape, package changes, sequencing,
tests, and completion gates.
## Implementation Rules
- Complete the stages in order. Do not begin a later stage while an earlier
stage's completion gate is unmet.
- Preserve unrelated working-tree changes and do not broaden the feature into
concurrency control, queues, retries, provider discovery, or
non-OpenAI-compatible transports.
- Keep the registry immutable and scoped to one `Engine`. Do not add
package-global registration or mutation after construction.
- Keep secrets out of backend definitions, profiles, prepared values, results,
hashes, logs, and JSON. Backends store only an environment-variable name.
- Preserve endpoint-only profiles and custom `LLMClient` injection.
- Follow the architecture, documentation, and testing policies under
`docs/policy/`. In particular, keep the public API in the root package,
place implementation packages under `internal/`, update the component
inventory when the new package lands, and assign each behavior to one
durable test owner.
- Use deterministic, offline tests. No stage may require a real provider,
credential, or network service.
## Fixed Design
### Public API
Add the following root-package declarations:
```go
const BackendOpenRouter = "openrouter"
type Backend struct {
ID string
Endpoint string
APIKeyEnv string
ExtraParams map[string]any
}
func WithBackend(backend Backend) Option
```
`Backend` and `WithBackend` have these contracts:
- `Backend` configures one OpenAI-compatible backend and has no stable JSON
representation.
- `ID` is trimmed, non-empty, case-sensitive, and is the stable registry key.
- `Endpoint` is trimmed and must be an absolute `http` or `https` URL with a
host. Paths are allowed; user information, query strings, and fragments are
rejected because this value is a base endpoint rather than a complete
request URL.
- `APIKeyEnv` is optional. When present, it is trimmed and must match the
portable environment-variable form `[A-Za-z_][A-Za-z0-9_]*`.
- `ExtraParams` follows the existing `Profile.ExtraParams` JSON-value rules:
non-empty string keys, finite numbers, JSON-compatible scalar and container
values, no cycles, and no collisions with `model`, `session_id`, `messages`,
`temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, or
`response_format`. It is deeply copied during `NewEngine`.
- An empty `ExtraParams` map means that the backend supplies no request
defaults.
- Each `WithBackend` call adds one registration. Calls with different IDs
accumulate in option order; a repeated ID is an error rather than a
replacement. This is an explicit additive exception to the last-option-wins
categories documented on `Option`.
- `BackendOpenRouter` is reserved. Consumers cannot replace it.
- There is no public enumerate, lookup, remove, replace, or post-construction
registration API in this scope.
Extend the existing public values as follows:
| Value | Addition | Contract |
| --- | --- | --- |
| `Profile` | `BackendID string` | Optional backend selection. `Endpoint` is required only when `BackendID` is blank. |
| `OpenAICompatibleProfileConfig` | `BackendID string` | Copied to `Profile.BackendID`. |
| `ExecutionTarget` | `BackendID string \`json:"backend_id,omitempty"\`` | Effective routing identity supplied to injected clients; empty for endpoint-only profiles. |
| `PreparedRun` | `SelectedBackendID string \`json:"selected_backend_id,omitempty"\`` | Equals the effective target backend ID. |
| `RunResult` | `SelectedBackendID string \`json:"selected_backend_id,omitempty"\`` | Carries the prepared backend identity through execution. |
Do not add a backend field to `RunRequest` or `ExecutionTargetOverride`.
Backend selection remains a profile concern. Endpoint overrides do not change
the selected backend identity.
Trim `Profile.BackendID` and YAML `backend` values at their respective
conversion/load boundaries. Treat an all-whitespace value as absent and expose
only the trimmed value in effective targets and metadata.
These exported struct fields are intentionally additive. Adding fields can
break downstream unkeyed composite literals even though keyed literals and
ordinary field access remain source-compatible. Accept that narrow risk; use
and document keyed literals, and do not add parallel wrapper types or a second
configuration path solely to preserve unkeyed literals.
### Profile Format
Add the optional strict-YAML field:
```yaml
backend: openrouter
```
The profile connection rule becomes:
- `model` remains required;
- at least one of `backend` or `endpoint` is required;
- when both are present, the profile endpoint overrides the backend endpoint;
and
- a profile with only an endpoint continues through the legacy path and has an
empty effective backend ID.
Do not infer a backend from a model name or endpoint. Do not validate whether a
backend ID exists while decoding a profile; registry membership is
engine-scoped and is checked when the selected profile is prepared.
### Registry And Internal Boundaries
Add `internal/backend` as the cohesive owner of:
- the immutable backend registry;
- backend-definition validation and defensive copying;
- the built-in OpenRouter definition; and
- the internal not-found error used when a selected ID is absent.
Add an internal domain backend value containing `ID`, `Endpoint`, `APIKeyEnv`,
and `ExtraParams`. The registry constructor accepts consumer additions, installs
built-ins first, rejects all collisions, and returns a fully constructed
read-only registry. Lookup returns a defensive value so callers cannot mutate
registry-owned maps.
Use these internal declarations:
```go
var ErrBackendNotFound = errors.New("backend not found")
func NewRegistry(additions []domain.Backend) (*Registry, error)
func (r *Registry) GetBackend(id string) (domain.Backend, error)
```
`internal/usecase` owns the narrow resolver interface it consumes:
```go
type BackendResolver interface {
GetBackend(string) (domain.Backend, error)
}
```
The concrete registry implements this interface. Supply it explicitly to
`Runner` through both runner constructors and from the root engine assembly.
A nil resolver must never panic; if an internally constructed runner selects a
backend without a resolver, preparation fails as a profile-load failure.
The built-in definition is exactly:
| Field | Value |
| --- | --- |
| ID | `openrouter` |
| Endpoint | `https://openrouter.ai/api/v1` |
| API key environment variable | `OPENROUTER_API_KEY` |
| Extra parameters | none |
Define `OpenRouterID` as a constant in `internal/backend` and define the public
`BackendOpenRouter` constant from it so there is one canonical literal. This
root-to-internal implementation import does not expose an internal type in a
public signature.
### Resolution
Resolve an execution target in this order:
1. framework defaults;
2. the selected backend, if any;
3. the selected profile; and
4. the request's `ExecutionTargetOverride`.
The layers behave as follows:
| Setting | Backend layer | Profile layer | Request layer |
| --- | --- | --- | --- |
| Backend ID | Supplies selected ID | Selects the backend | Cannot change it |
| Endpoint | Supplies default | Non-empty value replaces backend | Non-empty value replaces profile/backend |
| Model and generation fields | Not supplied | Existing profile behavior | Existing override behavior |
| API-key environment name | Supplies default | Non-empty `api_key_env` replaces it | Non-empty `APIKeyEnv` replaces it |
| Extra parameters | Non-empty map replaces prior map | Non-empty map replaces complete backend map | Non-empty map replaces complete profile/backend map |
Never deep-merge `ExtraParams` maps. A non-empty higher-precedence map replaces
the complete lower-precedence map. Preserve the existing distinction between a
nil or empty override map and a supplied non-empty replacement.
Credential handling remains:
1. a non-blank `RunRequest.APIKey` wins and suppresses environment lookup;
2. a non-blank request `APIKeyEnv` wins over profile and backend metadata;
3. file-profile `api_key_env` wins over the backend default;
4. backend `APIKeyEnv` is used when no higher layer supplies a credential
source; and
5. a profile with `APIKeyRequired: true` clears an inherited backend
environment name and requires a direct key unless the request explicitly
supplies its own `APIKeyEnv`.
Only the environment-variable name appears in effective targets. Preserve the
current just-in-time environment lookup and missing-variable behavior.
### Errors
Do not add a public backend-specific error sentinel.
| Failure | Required public identity |
| --- | --- |
| Invalid consumer backend definition | `ErrInvalidConfig` from `NewEngine` |
| Consumer ID duplicates another consumer ID | `ErrInvalidConfig` |
| Consumer ID collides with `BackendOpenRouter` | `ErrInvalidConfig` |
| Selected profile names an unknown backend | `ErrProfileLoad`, not `ErrProfileNotFound` or `ErrInvalidRequest` |
| File profile lacks both backend and endpoint | `ErrProfileLoad` |
| In-memory profile lacks both backend and endpoint | `ErrInvalidConfig` |
| Effective credential environment variable is unset | Existing `ErrAPIKeyEnvMissing` and `ErrInvalidRequest` identities |
| Other invalid per-run effective settings | Existing `ErrInvalidRequest` |
Error text must include the offending backend ID or field where useful, but
exact prose is not a compatibility contract. Preserve wrapped internal causes
where the existing error boundary permits it.
### Metadata And Transport
- Set `ExecutionTarget.BackendID`, `PreparedRun.SelectedBackendID`, and
`RunResult.SelectedBackendID` from the normalized selected profile backend.
- Endpoint-only profiles leave all three values empty.
- Pass `BackendID` through `GenerateRequest.Target` and repair targets so an
injected client can route or observe it.
- The built-in OpenAI-compatible client must not serialize `backend_id` as a
provider request field. It continues to use the effective endpoint,
credential, typed generation fields, and extra parameters.
- Backend selection does not change rendered-prompt hashes, prompt hashes,
session IDs, validation, artifacts, timing, or synchronous `Prepare` and
`Run` behavior.
## Stage 1 — Registry Foundation
### Goal
Introduce the internal domain and immutable registry without changing the
public configuration or profile format.
### Work
1. Add the internal backend definition to `internal/domain`.
2. Create `internal/backend` with:
- a registry constructor that installs OpenRouter and accepts additional
internal definitions;
- validation for IDs, endpoints, environment-variable names, JSON values,
and reserved request fields;
- deterministic built-in and consumer collision handling;
- immutable lookup with defensive map copying; and
- an internal recognizable not-found error.
3. Reuse or extract one validation rule for OpenAI-compatible reserved request
fields so registry validation and `internal/llm` payload construction cannot
drift. Do not maintain duplicate reserved-key lists.
4. Keep JSON-value copying type-preserving. Do not use a marshal/unmarshal
round trip that silently changes integer or container types.
5. Update `docs/policy/architecture.md` and `docs/internal/overview.md` only
after the package exists, describing `internal/backend` as an implemented
immutable configuration registry rather than a runtime service manager.
### Tests
`internal/backend` owns focused package tests for:
- the exact OpenRouter built-in values;
- successful unique consumer additions;
- built-in and consumer duplicate rejection;
- blank and whitespace-normalized IDs;
- invalid or non-HTTP(S) endpoints;
- invalid environment-variable names;
- invalid, cyclic, non-finite, empty-key, and reserved-key extra parameters;
- not-found lookup; and
- mutation isolation of input maps and returned values.
Use one representative table per validation family rather than one test per
branch. A race-specific registry test is unnecessary if the registry is
immutable and the final repository race suite exercises concurrent reads.
### Completion Gate
- No public declaration or profile behavior has changed.
- The registry has no mutating method after construction.
- The built-in definition contains no credential value.
- Focused backend tests and the complete repository validation pass.
## Stage 2 — Profile Selection And Effective Resolution
### Goal
Make the built-in OpenRouter backend selectable by file and in-memory profiles,
migrate the built-in profiles, and expose effective backend identity.
### Work
1. Add `BackendID` to internal execution profiles and targets and add the YAML
key `backend`.
2. Change file and public in-memory profile validation to require `model` plus
at least one of `backend` or `endpoint`. Preserve all existing numeric and
extra-parameter validation.
3. Add `BackendID` to `Profile` and
`OpenAICompatibleProfileConfig`, and add `BackendOpenRouter`.
4. Add backend identity to the public and internal execution target, prepared
run, and run result exactly as specified under **Public API**.
5. Add the `BackendResolver` dependency immediately after the profile
repository in both runner constructors and update all internal call sites
and test helpers explicitly.
6. Resolve the selected backend after profile loading and before effective
target resolution. Map lookup failure to `ErrProfileLoad`.
7. Implement the exact endpoint, credential, and whole-map precedence rules
under **Resolution**. Keep backend ID independent from endpoint overrides.
8. Construct the built-in-only registry in `NewEngine` and inject it into the
runner.
9. Replace `endpoint` and `api_key_env` in every
`internal/profile/builtin/assets/*.yaml` profile with
`backend: openrouter`. Retain each profile's model and generation settings.
10. Update conversions and copying so backend IDs survive every public/internal
boundary and registry-owned maps remain isolated.
11. Update the GoDoc for all changed public declarations.
12. Update `docs/formats.md`, `docs/internal/sources.md`, and
`docs/internal/runner.md` with the implemented profile field, conditional
endpoint rule, built-in selection, and effective resolution behavior.
### Tests
Assign test ownership as follows:
- `internal/profile` tests own strict YAML decoding and the conditional
backend-or-endpoint validation matrix.
- `internal/profile/builtin` tests own the invariant that every built-in
profile selects `openrouter` and no longer repeats its endpoint or
credential environment name.
- `internal/usecase` tests own backend/profile/request precedence, unknown-ID
error wrapping, endpoint override identity, credential precedence including
`APIKeyRequired`, and whole-map replacement.
- Root external-package contract tests own the new fields, JSON names and
omission behavior, public copying, and public error identities.
- One assembled engine test must prove a built-in profile prepares with the
expected backend ID, endpoint, and environment-variable name without making
an HTTP call.
Do not duplicate every profile-parser validation case at the engine boundary.
### Completion Gate
- All built-in and endpoint-only profiles prepare successfully.
- A selected unknown backend matches public `ErrProfileLoad`.
- Backend identity reaches prepared values, results, generation targets, and
repair targets.
- No backend ID or secret is added to the outbound provider payload.
- Current-state format and internal documentation match the implemented
built-in behavior.
- The complete repository validation passes.
## Stage 3 — Consumer Registration
### Goal
Expose the engine-scoped extension point for additional unique
OpenAI-compatible backend IDs.
### Work
1. Add `Backend` and `WithBackend` exactly as specified under **Public API**,
preferably in a cohesive root `backends.go` file.
2. Convert, validate, normalize, and deeply copy public backend values during
`NewEngine`. Store pending additions in `engineOptions`; do not mutate a
registry from option application.
3. After all options have applied, construct one immutable registry from the
built-in and pending consumer definitions. Map every construction failure
to `ErrInvalidConfig`.
4. Make unique `WithBackend` calls additive. Reject duplicate IDs even when
they arrive through separate calls, and document this exception on
`Option` and `WithBackend`.
5. Make registered backends available equally to directory, `fs.FS`, single
file, in-memory, prompt-default, and explicit request profile selection.
6. Preserve the last-valid-option-wins behavior of every existing option
category.
7. Update `docs/consumers/pkg-promptkit.md` with one minimal custom backend and
profile example. Keep exact field-by-field API detail in GoDoc and link to
it rather than duplicating it.
### Tests
Root public contract tests own:
- one custom backend used by an in-memory profile;
- one custom backend used by a file-backed profile;
- multiple unique `WithBackend` calls accumulating;
- duplicate consumer and built-in ID failures matching `ErrInvalidConfig`;
- invalid public values and nested-map mutation isolation;
- endpoint-only compatibility with no `WithBackend` option;
- profile endpoint and request endpoint overrides retaining the custom backend
ID; and
- an injected `LLMClient` observing the effective custom backend ID and
settings.
Use an injected client or `httptest.Server`; never contact the configured
external endpoint. Keep lower-level registry validation cases in
`internal/backend`.
### Completion Gate
- Consumers can add only new IDs and cannot mutate or replace built-ins.
- Registration is engine-local and two engines can use different definitions
for the same consumer ID without interference.
- Existing option-category contract tests still pass unchanged except for the
documented additive backend case.
- The consumer guide and GoDoc describe the implemented extension point.
- The complete repository validation passes.
## Stage 4 — Contract Hardening And Documentation Completion
### Goal
Audit the completed feature across public, profile, use-case, and transport
boundaries and make the current-state documentation self-sufficient.
### Work
1. Review the preceding tests as a suite. Remove redundant cases and retain a
lean precedence matrix that would catch layer-order, credential, copying,
compatibility, and error-identity regressions.
2. Add or adjust one built-in-client transport test proving that effective
backend defaults reach the configured endpoint while `backend_id` is not
serialized. Use `httptest.Server` and synthetic credentials.
3. Confirm public stable-JSON round trips for `ExecutionTarget`,
`PreparedRun`, and `RunResult`, including empty backend omission for legacy
profiles.
4. Confirm concurrent `Prepare` and `Run` calls can read the immutable registry
under the race detector. Do not add concurrency limiting or a queue.
5. Update all affected current-state owners:
- public GoDoc for the exact API and errors;
- `docs/formats.md` for profile YAML and built-ins;
- `docs/consumers/pkg-promptkit.md` for construction and use;
- `docs/integrations/openai-compatible-chat.md` for resolved endpoint,
credentials, request defaults, and the non-serialized routing identity;
- `docs/internal/overview.md`, `docs/internal/sources.md`,
`docs/internal/runner.md`, and `docs/internal/llm.md` for package
responsibilities and data flow; and
- `docs/policy/architecture.md` only to keep its implemented component
inventory and dependency description accurate.
6. Search current-state documentation for obsolete statements that every
profile requires an endpoint or that every built-in repeats OpenRouter
connection settings. Update the canonical owner and replace duplicates with
links.
7. Keep future concurrency and queue behavior only in
[the future feature catalog](future.md); do not imply that this feature
implements either capability.
### Final Validation
Run the complete maintainer sequence from the repository root:
```sh
go test ./...
go test -race ./...
go vet ./...
go build ./...
go run ./examples/go-library/prepare
gofmt -l $(git ls-files '*.go')
git diff --check
```
The formatting command must produce no paths. Follow every changed Markdown
link and confirm its target exists. Inspect the final diff for raw credentials,
global mutable state, generated workspace files, local `replace` directives,
and unrelated changes.
### Completion Gate
- Every target-end-state item in [the feature roadmap](backends.md) is
implemented and protected at its owning boundary.
- The public API is limited to the fixed declarations in this plan.
- Endpoint-only profiles and injected clients remain compatible.
- Built-in profiles obtain OpenRouter endpoint and credential metadata only
through the registry.
- Current-state documentation no longer relies on either roadmap to explain
implemented behavior.
- All final validation commands pass offline.
## Open Questions
None. The API shape, validation, precedence, compatibility behavior, error
identities, package ownership, test ownership, and staging required for
implementation are fixed by this plan.