Prepare the backend registry implementation roadmap
This commit is contained in:
180
docs/roadmap/backends.md
Normal file
180
docs/roadmap/backends.md
Normal file
@@ -0,0 +1,180 @@
|
||||
# Extensible LLM Backend Registry
|
||||
|
||||
## Purpose
|
||||
|
||||
This roadmap defines the scope and target end state for formal
|
||||
OpenAI-compatible backend support in Promptkit. It establishes the behavioral
|
||||
boundary and policy choices for the work.
|
||||
|
||||
This document is planning material, not a description of current behavior.
|
||||
Current public contracts remain owned by Go declarations and GoDoc, framework
|
||||
file formats by the [format reference](../formats.md), and outbound HTTP
|
||||
behavior by the
|
||||
[OpenAI-compatible integration contract](../integrations/openai-compatible-chat.md).
|
||||
|
||||
## Motivation
|
||||
|
||||
Execution profiles currently carry backend connection and authentication
|
||||
details alongside model and generation settings. This repeats values such as
|
||||
the OpenRouter endpoint and `OPENROUTER_API_KEY` across profiles, leaves the
|
||||
built-in catalog implicitly tied to one backend, and gives consumers no
|
||||
explicit extension point for naming other OpenAI-compatible services.
|
||||
|
||||
Promptkit should distinguish:
|
||||
|
||||
- a **backend**, which identifies reusable connection, authentication, and
|
||||
limited backend-wide request defaults; and
|
||||
- a **profile**, which selects a model and its reusable generation settings.
|
||||
|
||||
That distinction should let consumers configure OpenRouter, OpenAI,
|
||||
local-network services, or other OpenAI-compatible deployments without
|
||||
duplicating backend defaults in every profile or replacing Promptkit's model
|
||||
client.
|
||||
|
||||
## Scope
|
||||
|
||||
The work introduces an engine-scoped registry of OpenAI-compatible backend
|
||||
definitions.
|
||||
|
||||
Each backend definition will provide:
|
||||
|
||||
- a stable, non-empty backend ID;
|
||||
- a default OpenAI-compatible base endpoint;
|
||||
- an optional environment-variable name for its API key; and
|
||||
- limited JSON-compatible request defaults when they are genuinely
|
||||
backend-wide rather than model-specific.
|
||||
|
||||
Promptkit will provide a small built-in backend catalog, initially containing
|
||||
OpenRouter. Downstream consumers will be able to register additional backend
|
||||
IDs during engine construction. Registration will be explicit and local to an
|
||||
engine; it will not mutate package-global state.
|
||||
|
||||
Profiles will be able to name a backend. A profile that selects a backend may
|
||||
override its default endpoint and other supported defaults while continuing to
|
||||
own its model and generation settings. Existing endpoint-based custom profiles
|
||||
will remain supported without requiring a backend ID.
|
||||
|
||||
The built-in model profile catalog will use the built-in OpenRouter backend
|
||||
instead of repeating OpenRouter connection and credential defaults in every
|
||||
profile.
|
||||
|
||||
## Resolution And Precedence
|
||||
|
||||
Effective execution settings will resolve in this order:
|
||||
|
||||
1. application-neutral framework defaults;
|
||||
2. selected backend defaults, when a backend is named;
|
||||
3. selected profile values; and
|
||||
4. explicit per-run overrides.
|
||||
|
||||
A profile endpoint will override its selected backend's endpoint. A per-run
|
||||
endpoint override will continue to take precedence over both.
|
||||
|
||||
Credential selection will preserve direct request credentials as the highest
|
||||
precedence. An explicit per-run environment-variable override will take
|
||||
precedence over profile credential configuration, which will take precedence
|
||||
over the backend's default environment-variable name. Backend definitions and
|
||||
profiles will contain credential-source metadata only, never resolved secret
|
||||
values.
|
||||
|
||||
Backend request defaults, profile request values, and per-run request
|
||||
overrides will follow one deterministic replacement rule. Resolution must not
|
||||
introduce implicit deep merging whose result depends on map iteration or
|
||||
incidental representation.
|
||||
|
||||
## Registration And Validation
|
||||
|
||||
Backend registration will be deterministic and validated during engine
|
||||
construction.
|
||||
|
||||
- Built-in backend IDs are reserved and cannot be replaced by consumers.
|
||||
- Consumer registrations may add only new IDs.
|
||||
- Duplicate consumer IDs are invalid, including duplicates introduced through
|
||||
repeated configuration.
|
||||
- Backend IDs, endpoints, credential-source metadata, and request defaults
|
||||
must be validated before an engine is returned.
|
||||
- Selecting an unknown backend is an error associated with the profile or
|
||||
request boundary that selected it.
|
||||
- A profile without a backend must continue to provide the connection details
|
||||
required by the current endpoint-based path.
|
||||
- A backend may omit credential requirements so unauthenticated local-network
|
||||
services remain supported.
|
||||
|
||||
Failures will retain error identities appropriate to the existing engine
|
||||
configuration, profile-loading, and per-run validation boundaries.
|
||||
|
||||
## Public And Extension Boundaries
|
||||
|
||||
The effective backend ID will be observable in prepared and completed run
|
||||
metadata and will be available to injected model clients. This gives consumers
|
||||
and extensions a stable routing identity without requiring them to infer a
|
||||
backend from an endpoint URL.
|
||||
|
||||
The existing public model-client injection boundary will remain supported.
|
||||
Because the initial registry supports only OpenAI-compatible backends,
|
||||
Promptkit does not need backend-specific transport factories or multiple
|
||||
protocol implementations in this scope. The built-in client will continue to
|
||||
send the resolved execution target to the selected OpenAI-compatible endpoint.
|
||||
|
||||
Backend definitions are configuration values, not live service objects. They
|
||||
will not own mutable connections, credentials, health state, or process
|
||||
lifecycle.
|
||||
|
||||
## Compatibility
|
||||
|
||||
The target end state preserves the existing ways consumers configure and run
|
||||
Promptkit:
|
||||
|
||||
- Profiles with explicit endpoints and no backend ID continue to work.
|
||||
- Profile endpoint overrides remain supported.
|
||||
- Direct request API keys and environment-variable overrides retain their
|
||||
precedence.
|
||||
- Consumers may continue injecting a custom model client.
|
||||
- Model IDs and generation settings remain profile concerns.
|
||||
- Prompt selection, rendering, validation, artifacts, and synchronous
|
||||
`Prepare` and `Run` behavior remain unchanged except for exposing the
|
||||
resolved backend identity.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
This scope does not include:
|
||||
|
||||
- non-OpenAI-compatible protocols or provider-specific SDKs;
|
||||
- backend-specific concurrency limits or admission queues;
|
||||
- asynchronous task submission or durable jobs;
|
||||
- model discovery, live model catalogs, or provider capability probing;
|
||||
- retries, rate limiting, failover, load balancing, or health checks;
|
||||
- automatic backend selection based on model names or endpoint inspection;
|
||||
- global mutable registration;
|
||||
- consumer replacement of built-in backend IDs;
|
||||
- credential discovery beyond configured direct values and environment
|
||||
variables; or
|
||||
- provider-specific application, deployment, or security policy.
|
||||
|
||||
The dependent concurrency and bounded-queue work remains a separate future
|
||||
unit after backend identity and resolution are stable.
|
||||
|
||||
## Target End State
|
||||
|
||||
This roadmap reaches its target end state when:
|
||||
|
||||
- Promptkit has an immutable, engine-scoped registry of validated
|
||||
OpenAI-compatible backend definitions;
|
||||
- OpenRouter is available as a built-in backend with its endpoint and
|
||||
credential environment-variable default;
|
||||
- consumers can register additional unique backend IDs without modifying
|
||||
Promptkit;
|
||||
- profiles can select a backend and optionally override its endpoint;
|
||||
- existing endpoint-only profiles remain valid;
|
||||
- backend, profile, and per-run values resolve through documented,
|
||||
deterministic precedence;
|
||||
- secret values remain outside backend and profile definitions;
|
||||
- the effective backend identity is available in preparation, execution, and
|
||||
injected-client values;
|
||||
- the built-in model profiles use the OpenRouter backend rather than duplicate
|
||||
its connection defaults;
|
||||
- the built-in OpenAI-compatible client and injected clients continue to work
|
||||
through the existing generation boundary; and
|
||||
- current-state GoDoc, format, consumer, integration, architecture, and
|
||||
internal documentation describe the implemented behavior without relying on
|
||||
this roadmap.
|
||||
@@ -1,222 +0,0 @@
|
||||
# Documentation Hardening Roadmap
|
||||
|
||||
## Purpose
|
||||
|
||||
This roadmap coordinates a focused pass over Promptkit's documentation and
|
||||
public contract. The work should close the gaps identified by the documentation
|
||||
audit, strengthen canonical ownership, and leave consumers and contributors
|
||||
with guidance that is accurate, navigable, and proportionate to their needs.
|
||||
|
||||
This is planning material, not a description of implemented behavior. Follow
|
||||
the [documentation policy](../policy/documentation.md) throughout the work and
|
||||
update current-state documents only when their claims are supported by the
|
||||
implementation and tests.
|
||||
|
||||
## Scope And Principles
|
||||
|
||||
The work covers public GoDoc, consumer guidance, maintained examples, format
|
||||
and integration references, architecture ownership, roadmap lifecycle, and
|
||||
documentation validation.
|
||||
|
||||
- Resolve ambiguous behavior before documenting it as a contract.
|
||||
- Keep exact exported API semantics in Go declarations and GoDoc.
|
||||
- Keep task-oriented guidance in consumer documents and complete runnable
|
||||
artifacts in `examples/`.
|
||||
- Put security-relevant consumer responsibilities at the public boundary, not
|
||||
only in internal contributor documents.
|
||||
- Remove duplicated ownership instead of synchronizing parallel references.
|
||||
- Make documentation checks reproducible where practical.
|
||||
- Preserve existing behavior unless a stage explicitly selects and tests an
|
||||
API change.
|
||||
|
||||
The roadmap does not add new Promptkit features, redesign framework formats,
|
||||
or implement ideas from [the future feature catalog](future.md). If resolving
|
||||
an ambiguity requires a behavioral change, treat that change as a separately
|
||||
reviewable implementation unit and update its canonical documentation in the
|
||||
same unit.
|
||||
|
||||
## Stage 1: Resolve Public Contract Questions
|
||||
|
||||
Before expanding prose, decide the intended contract for exported behavior
|
||||
that is currently ambiguous.
|
||||
|
||||
- [x] Decide whether `RunRequest.Metadata` has a supported observable purpose.
|
||||
Define its propagation and ownership, or remove or deprecate it through an
|
||||
intentional public API change.
|
||||
- [x] Decide and test whether one `Engine` supports concurrent `Prepare` and
|
||||
`Run` calls.
|
||||
- [x] Decide how repeated options of the same category behave, including
|
||||
prompt, profile, schema, model-client, and artifact-reader options.
|
||||
- [x] Define which public values have supported JSON representations.
|
||||
- [x] Define JSON time units and omission behavior, including the relationship
|
||||
between prepared-run and run-result durations.
|
||||
- [x] Decide whether run IDs and exposed hashes have stable formats or must be
|
||||
treated as opaque values.
|
||||
- [x] Confirm the intended transport-timeout default and its zero or negative
|
||||
configuration semantics.
|
||||
- [x] Confirm the supported JSON Schema dialect and reference boundaries,
|
||||
including whether remote references are allowed.
|
||||
|
||||
The selected exported API contracts are implemented and tested. Their durable
|
||||
definitions now belong to the root package declarations and GoDoc.
|
||||
|
||||
One format-level decision remains here until Stage 5 moves it to the framework
|
||||
format reference: JSON Schema uses Draft 2020-12, with that dialect selected
|
||||
when `$schema` is omitted. Same-document fragments and relative references
|
||||
contained by a directory or `fs.FS` schema root are supported. A single-file
|
||||
source supports only references contained in that document. Absolute,
|
||||
escaping, and remote references are rejected.
|
||||
|
||||
**Gate:** Each question has an explicit answer backed by existing behavior or
|
||||
by an accepted implementation change and proportionate tests. No later stage
|
||||
should invent a contract merely to fill a documentation gap.
|
||||
|
||||
## Stage 2: Make GoDoc The Canonical Public Contract
|
||||
|
||||
Strengthen the root package declarations so `go doc` is sufficient to
|
||||
understand exact public behavior without relying on internal documents.
|
||||
|
||||
- [x] Add useful field-level GoDoc to configuration, request, profile,
|
||||
execution-target, result, artifact, validation, structured-output, and model
|
||||
client values.
|
||||
- [x] Document required fields and nil, empty, and zero-value semantics.
|
||||
- [x] Document override, replacement, profile-precedence, and copy-ownership
|
||||
behavior where it belongs to the exported API.
|
||||
- [x] Document credential inputs, redaction, and the values intentionally
|
||||
excluded from serialization.
|
||||
- [x] Give each public error sentinel an accurate comment and document the
|
||||
supported `errors.Is` relationships.
|
||||
- [x] Document engine concurrency and option-composition behavior selected in
|
||||
Stage 1.
|
||||
- [x] Document serialization, time, run-ID, and hash semantics selected in
|
||||
Stage 1.
|
||||
- [x] Review constructor, option, extension-interface, `Prepare`, and `Run`
|
||||
GoDoc for complete failure and cancellation expectations.
|
||||
|
||||
Update the [consumer guide](../consumers/pkg-promptkit.md) to summarize and link
|
||||
to these contracts instead of maintaining exhaustive copies of exported names
|
||||
or exact semantics.
|
||||
|
||||
**Gate:** `go doc -all .` presents a coherent public contract, exported
|
||||
declarations have accurate comments, and contract tests protect every newly
|
||||
documented behavior whose compatibility risk warrants durable coverage.
|
||||
|
||||
## Stage 3: Improve Consumer Safety And Executable Guidance
|
||||
|
||||
Move consumer-relevant security boundaries to the places where consumers will
|
||||
encounter them and add one representative execution workflow.
|
||||
|
||||
- [x] Explain in public GoDoc and the consumer guide that the default file
|
||||
artifact reader accepts unrestricted caller-selected paths.
|
||||
- [x] Make clear that Promptkit does not impose an application root, inbound
|
||||
request-size policy, or untrusted-input security boundary.
|
||||
- [x] Explain that rendered messages, artifact bodies, raw model output, and
|
||||
validation details may be sensitive even when credentials are redacted.
|
||||
- [x] Clarify the responsibilities of injected artifact readers and model
|
||||
clients for cancellation, copying, logging, and secret handling.
|
||||
- [x] Add a maintained offline `Run` example using an injected deterministic
|
||||
model client, without credentials, live network access, or paid calls.
|
||||
- [x] Link the consumer guide to the execution example and keep embedded
|
||||
snippets smaller than the maintained artifact.
|
||||
- [x] Decide whether the existing preparation example should remain separate
|
||||
or share reusable fixtures without obscuring either workflow.
|
||||
|
||||
The preparation and execution examples remain separate, self-contained
|
||||
workflows. Each keeps its own small prompt fixture so consumers can copy or run
|
||||
one example without depending on the other.
|
||||
|
||||
**Gate:** Both preparation and execution have complete, secret-free,
|
||||
deterministic consumer examples, and the consumer guide exposes the important
|
||||
filesystem and data-sensitivity boundaries without leaking internal mechanics.
|
||||
|
||||
## Stage 4: Restore Canonical Ownership
|
||||
|
||||
Remove parallel definitions and make navigation follow the ownership model in
|
||||
the documentation policy.
|
||||
|
||||
- [ ] Reduce the [architecture policy](../policy/architecture.md) to durable
|
||||
boundaries, layers, dependency direction, invariants, and non-goals.
|
||||
- [ ] Keep the exact implemented package and component inventory solely in the
|
||||
[internal component overview](../internal/overview.md).
|
||||
- [ ] Review the consumer guide's public error and option lists so they remain
|
||||
task-oriented summaries rather than duplicate API references.
|
||||
- [ ] Review internal documents for public-contract statements that should be
|
||||
links to GoDoc or the format and integration owners.
|
||||
- [ ] Reconcile the documentation policy's temporary-roadmap lifecycle with
|
||||
the continuing idea-catalog role of `docs/roadmap/future.md`.
|
||||
- [ ] Rephrase or link roadmap statements that depend on exact current API
|
||||
behavior, particularly the runtime reasoning entry.
|
||||
- [ ] Confirm that every document states its audience or purpose and links to
|
||||
the canonical owner of adjacent topics.
|
||||
|
||||
**Gate:** Every authoritative fact has one clear owner, package inventory
|
||||
changes no longer require edits to the architecture policy, and roadmaps
|
||||
cannot be mistaken for current-state references.
|
||||
|
||||
## Stage 5: Refine Format And Integration References
|
||||
|
||||
Close compatibility gaps in the documents that own file formats and outbound
|
||||
wire behavior.
|
||||
|
||||
- [ ] State or canonically link the exact session-ID limit enforced by the
|
||||
OpenAI-compatible client.
|
||||
- [ ] State the configured and default transport-timeout behavior without
|
||||
referring to an unnamed internal default.
|
||||
- [ ] Document the JSON Schema dialect and local, contained, and remote
|
||||
reference behavior selected in Stage 1.
|
||||
- [ ] Clarify structured-output naming and strictness when those values are
|
||||
part of the public or integration contract.
|
||||
- [ ] Add a caveat that built-in profiles are maintained configurations, not a
|
||||
guarantee of continuing third-party model availability.
|
||||
- [ ] Recheck every prompt, profile, schema, credential, request-body, response,
|
||||
timeout, and precedence statement against its owning implementation and
|
||||
tests.
|
||||
|
||||
**Gate:** A consumer can determine the supported file and wire compatibility
|
||||
boundaries without consulting internal source code or relying on unspecified
|
||||
defaults.
|
||||
|
||||
## Stage 6: Make Documentation Validation Reproducible
|
||||
|
||||
Align contributor and release procedures around a small, consistent set of
|
||||
checks.
|
||||
|
||||
- [ ] Use one robust command for checking every tracked Go file with `gofmt`.
|
||||
- [ ] Provide a repository-local or clearly documented command that validates
|
||||
local Markdown targets and heading fragments.
|
||||
- [ ] Decide how published external links are checked without making ordinary
|
||||
validation depend on mutable network services.
|
||||
- [ ] Reconcile the validation descriptions in the
|
||||
[development guide](../development.md),
|
||||
[testing policy](../policy/testing.md), and
|
||||
[release procedure](../release.md) so one document owns each requirement.
|
||||
- [ ] Ensure example validation covers every maintained example added by this
|
||||
roadmap.
|
||||
- [ ] Keep documentation-only validation proportionate while requiring full Go
|
||||
validation when commands, examples, generated output, or checked behavior
|
||||
changes.
|
||||
|
||||
**Gate:** A maintainer can run the documented formatting, link, example, Go,
|
||||
and repository-hygiene checks exactly as written, with no hidden manual
|
||||
procedure for local documentation.
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
The roadmap is complete when:
|
||||
|
||||
- all Stage 1 contract questions are resolved;
|
||||
- GoDoc is the authoritative and sufficient exported API reference;
|
||||
- consumer guidance covers unrestricted file access and sensitive generated
|
||||
data;
|
||||
- maintained offline examples cover both `Prepare` and `Run`;
|
||||
- architecture, inventory, consumer, internal, format, integration, and
|
||||
roadmap documents follow their assigned ownership boundaries;
|
||||
- schema, session, timeout, structured-output, and built-in-profile
|
||||
compatibility statements are explicit;
|
||||
- documentation validation is reproducible and consistent across contributor
|
||||
and release workflows; and
|
||||
- the complete maintainer validation passes.
|
||||
|
||||
After completion, move any durable decisions to GoDoc, policy, format,
|
||||
integration, or ADR owners as appropriate. Remove this roadmap after incoming
|
||||
links are updated; do not retain it as a second current-state reference.
|
||||
@@ -33,29 +33,15 @@ consumers.
|
||||
|
||||
## Ideas
|
||||
|
||||
### Extensible LLM backend registry
|
||||
|
||||
Introduce a registry that separates backend-specific connection,
|
||||
authentication, and limited request defaults from model execution profiles.
|
||||
Initial support would cover OpenAI-compatible backends and include a small
|
||||
built-in catalog, potentially starting with OpenRouter. A profile could select
|
||||
a backend while optionally overriding its default endpoint, and each backend
|
||||
could name an optional environment variable for its API key without storing
|
||||
the credential itself. Downstream consumers could register additional,
|
||||
uniquely named backends, such as OpenAI or unauthenticated local-network
|
||||
services, but could not replace built-in IDs. Model selection and generation
|
||||
settings would remain profile concerns, and custom model clients would remain
|
||||
available for behavior outside the registry's supported protocol.
|
||||
|
||||
### Backend-specific concurrency management
|
||||
|
||||
Extend the proposed LLM backend registry with optional per-backend concurrency
|
||||
limits and bounded, buffered admission queues. Promptkit could then route
|
||||
simultaneous generation requests according to backend capacity while
|
||||
containing accidental runaway submission. Downstream consumers would continue
|
||||
invoking synchronous `Run` calls, including concurrently from multiple
|
||||
goroutines, and each admitted call would wait for and return its ordinary
|
||||
result.
|
||||
Extend the selected
|
||||
[LLM backend registry](backends.md) with optional per-backend concurrency limits
|
||||
and bounded, buffered admission queues. Promptkit could then route simultaneous
|
||||
generation requests according to backend capacity while containing accidental
|
||||
runaway submission. Downstream consumers would continue invoking synchronous
|
||||
`Run` calls, including concurrently from multiple goroutines, and each
|
||||
admitted call would wait for and return its ordinary result.
|
||||
|
||||
- Scope limits to an engine instance rather than hidden process-global state.
|
||||
- Give different backend IDs independent capacity pools. A profile endpoint
|
||||
|
||||
497
docs/roadmap/implementation.md
Normal file
497
docs/roadmap/implementation.md
Normal file
@@ -0,0 +1,497 @@
|
||||
# Extensible LLM Backend Registry Implementation Plan
|
||||
|
||||
## Purpose
|
||||
|
||||
This document is the decision-complete implementation plan for the
|
||||
[extensible LLM backend registry](backends.md). It is written for a coding
|
||||
agent that will implement each stage in order.
|
||||
|
||||
This is planning material, not a description of current behavior. During
|
||||
implementation, update current-state GoDoc and documentation only as the
|
||||
corresponding behavior lands. The feature roadmap owns the intended end state
|
||||
and policy choices; this document owns API shape, package changes, sequencing,
|
||||
tests, and completion gates.
|
||||
|
||||
## Implementation Rules
|
||||
|
||||
- Complete the stages in order. Do not begin a later stage while an earlier
|
||||
stage's completion gate is unmet.
|
||||
- Preserve unrelated working-tree changes and do not broaden the feature into
|
||||
concurrency control, queues, retries, provider discovery, or
|
||||
non-OpenAI-compatible transports.
|
||||
- Keep the registry immutable and scoped to one `Engine`. Do not add
|
||||
package-global registration or mutation after construction.
|
||||
- Keep secrets out of backend definitions, profiles, prepared values, results,
|
||||
hashes, logs, and JSON. Backends store only an environment-variable name.
|
||||
- Preserve endpoint-only profiles and custom `LLMClient` injection.
|
||||
- Follow the architecture, documentation, and testing policies under
|
||||
`docs/policy/`. In particular, keep the public API in the root package,
|
||||
place implementation packages under `internal/`, update the component
|
||||
inventory when the new package lands, and assign each behavior to one
|
||||
durable test owner.
|
||||
- Use deterministic, offline tests. No stage may require a real provider,
|
||||
credential, or network service.
|
||||
|
||||
## Fixed Design
|
||||
|
||||
### Public API
|
||||
|
||||
Add the following root-package declarations:
|
||||
|
||||
```go
|
||||
const BackendOpenRouter = "openrouter"
|
||||
|
||||
type Backend struct {
|
||||
ID string
|
||||
Endpoint string
|
||||
APIKeyEnv string
|
||||
ExtraParams map[string]any
|
||||
}
|
||||
|
||||
func WithBackend(backend Backend) Option
|
||||
```
|
||||
|
||||
`Backend` and `WithBackend` have these contracts:
|
||||
|
||||
- `Backend` configures one OpenAI-compatible backend and has no stable JSON
|
||||
representation.
|
||||
- `ID` is trimmed, non-empty, case-sensitive, and is the stable registry key.
|
||||
- `Endpoint` is trimmed and must be an absolute `http` or `https` URL with a
|
||||
host. Paths are allowed; user information, query strings, and fragments are
|
||||
rejected because this value is a base endpoint rather than a complete
|
||||
request URL.
|
||||
- `APIKeyEnv` is optional. When present, it is trimmed and must match the
|
||||
portable environment-variable form `[A-Za-z_][A-Za-z0-9_]*`.
|
||||
- `ExtraParams` follows the existing `Profile.ExtraParams` JSON-value rules:
|
||||
non-empty string keys, finite numbers, JSON-compatible scalar and container
|
||||
values, no cycles, and no collisions with `model`, `session_id`, `messages`,
|
||||
`temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, or
|
||||
`response_format`. It is deeply copied during `NewEngine`.
|
||||
- An empty `ExtraParams` map means that the backend supplies no request
|
||||
defaults.
|
||||
- Each `WithBackend` call adds one registration. Calls with different IDs
|
||||
accumulate in option order; a repeated ID is an error rather than a
|
||||
replacement. This is an explicit additive exception to the last-option-wins
|
||||
categories documented on `Option`.
|
||||
- `BackendOpenRouter` is reserved. Consumers cannot replace it.
|
||||
- There is no public enumerate, lookup, remove, replace, or post-construction
|
||||
registration API in this scope.
|
||||
|
||||
Extend the existing public values as follows:
|
||||
|
||||
| Value | Addition | Contract |
|
||||
| --- | --- | --- |
|
||||
| `Profile` | `BackendID string` | Optional backend selection. `Endpoint` is required only when `BackendID` is blank. |
|
||||
| `OpenAICompatibleProfileConfig` | `BackendID string` | Copied to `Profile.BackendID`. |
|
||||
| `ExecutionTarget` | `BackendID string \`json:"backend_id,omitempty"\`` | Effective routing identity supplied to injected clients; empty for endpoint-only profiles. |
|
||||
| `PreparedRun` | `SelectedBackendID string \`json:"selected_backend_id,omitempty"\`` | Equals the effective target backend ID. |
|
||||
| `RunResult` | `SelectedBackendID string \`json:"selected_backend_id,omitempty"\`` | Carries the prepared backend identity through execution. |
|
||||
|
||||
Do not add a backend field to `RunRequest` or `ExecutionTargetOverride`.
|
||||
Backend selection remains a profile concern. Endpoint overrides do not change
|
||||
the selected backend identity.
|
||||
|
||||
Trim `Profile.BackendID` and YAML `backend` values at their respective
|
||||
conversion/load boundaries. Treat an all-whitespace value as absent and expose
|
||||
only the trimmed value in effective targets and metadata.
|
||||
|
||||
These exported struct fields are intentionally additive. Adding fields can
|
||||
break downstream unkeyed composite literals even though keyed literals and
|
||||
ordinary field access remain source-compatible. Accept that narrow risk; use
|
||||
and document keyed literals, and do not add parallel wrapper types or a second
|
||||
configuration path solely to preserve unkeyed literals.
|
||||
|
||||
### Profile Format
|
||||
|
||||
Add the optional strict-YAML field:
|
||||
|
||||
```yaml
|
||||
backend: openrouter
|
||||
```
|
||||
|
||||
The profile connection rule becomes:
|
||||
|
||||
- `model` remains required;
|
||||
- at least one of `backend` or `endpoint` is required;
|
||||
- when both are present, the profile endpoint overrides the backend endpoint;
|
||||
and
|
||||
- a profile with only an endpoint continues through the legacy path and has an
|
||||
empty effective backend ID.
|
||||
|
||||
Do not infer a backend from a model name or endpoint. Do not validate whether a
|
||||
backend ID exists while decoding a profile; registry membership is
|
||||
engine-scoped and is checked when the selected profile is prepared.
|
||||
|
||||
### Registry And Internal Boundaries
|
||||
|
||||
Add `internal/backend` as the cohesive owner of:
|
||||
|
||||
- the immutable backend registry;
|
||||
- backend-definition validation and defensive copying;
|
||||
- the built-in OpenRouter definition; and
|
||||
- the internal not-found error used when a selected ID is absent.
|
||||
|
||||
Add an internal domain backend value containing `ID`, `Endpoint`, `APIKeyEnv`,
|
||||
and `ExtraParams`. The registry constructor accepts consumer additions, installs
|
||||
built-ins first, rejects all collisions, and returns a fully constructed
|
||||
read-only registry. Lookup returns a defensive value so callers cannot mutate
|
||||
registry-owned maps.
|
||||
|
||||
Use these internal declarations:
|
||||
|
||||
```go
|
||||
var ErrBackendNotFound = errors.New("backend not found")
|
||||
|
||||
func NewRegistry(additions []domain.Backend) (*Registry, error)
|
||||
func (r *Registry) GetBackend(id string) (domain.Backend, error)
|
||||
```
|
||||
|
||||
`internal/usecase` owns the narrow resolver interface it consumes:
|
||||
|
||||
```go
|
||||
type BackendResolver interface {
|
||||
GetBackend(string) (domain.Backend, error)
|
||||
}
|
||||
```
|
||||
|
||||
The concrete registry implements this interface. Supply it explicitly to
|
||||
`Runner` through both runner constructors and from the root engine assembly.
|
||||
A nil resolver must never panic; if an internally constructed runner selects a
|
||||
backend without a resolver, preparation fails as a profile-load failure.
|
||||
|
||||
The built-in definition is exactly:
|
||||
|
||||
| Field | Value |
|
||||
| --- | --- |
|
||||
| ID | `openrouter` |
|
||||
| Endpoint | `https://openrouter.ai/api/v1` |
|
||||
| API key environment variable | `OPENROUTER_API_KEY` |
|
||||
| Extra parameters | none |
|
||||
|
||||
Define `OpenRouterID` as a constant in `internal/backend` and define the public
|
||||
`BackendOpenRouter` constant from it so there is one canonical literal. This
|
||||
root-to-internal implementation import does not expose an internal type in a
|
||||
public signature.
|
||||
|
||||
### Resolution
|
||||
|
||||
Resolve an execution target in this order:
|
||||
|
||||
1. framework defaults;
|
||||
2. the selected backend, if any;
|
||||
3. the selected profile; and
|
||||
4. the request's `ExecutionTargetOverride`.
|
||||
|
||||
The layers behave as follows:
|
||||
|
||||
| Setting | Backend layer | Profile layer | Request layer |
|
||||
| --- | --- | --- | --- |
|
||||
| Backend ID | Supplies selected ID | Selects the backend | Cannot change it |
|
||||
| Endpoint | Supplies default | Non-empty value replaces backend | Non-empty value replaces profile/backend |
|
||||
| Model and generation fields | Not supplied | Existing profile behavior | Existing override behavior |
|
||||
| API-key environment name | Supplies default | Non-empty `api_key_env` replaces it | Non-empty `APIKeyEnv` replaces it |
|
||||
| Extra parameters | Non-empty map replaces prior map | Non-empty map replaces complete backend map | Non-empty map replaces complete profile/backend map |
|
||||
|
||||
Never deep-merge `ExtraParams` maps. A non-empty higher-precedence map replaces
|
||||
the complete lower-precedence map. Preserve the existing distinction between a
|
||||
nil or empty override map and a supplied non-empty replacement.
|
||||
|
||||
Credential handling remains:
|
||||
|
||||
1. a non-blank `RunRequest.APIKey` wins and suppresses environment lookup;
|
||||
2. a non-blank request `APIKeyEnv` wins over profile and backend metadata;
|
||||
3. file-profile `api_key_env` wins over the backend default;
|
||||
4. backend `APIKeyEnv` is used when no higher layer supplies a credential
|
||||
source; and
|
||||
5. a profile with `APIKeyRequired: true` clears an inherited backend
|
||||
environment name and requires a direct key unless the request explicitly
|
||||
supplies its own `APIKeyEnv`.
|
||||
|
||||
Only the environment-variable name appears in effective targets. Preserve the
|
||||
current just-in-time environment lookup and missing-variable behavior.
|
||||
|
||||
### Errors
|
||||
|
||||
Do not add a public backend-specific error sentinel.
|
||||
|
||||
| Failure | Required public identity |
|
||||
| --- | --- |
|
||||
| Invalid consumer backend definition | `ErrInvalidConfig` from `NewEngine` |
|
||||
| Consumer ID duplicates another consumer ID | `ErrInvalidConfig` |
|
||||
| Consumer ID collides with `BackendOpenRouter` | `ErrInvalidConfig` |
|
||||
| Selected profile names an unknown backend | `ErrProfileLoad`, not `ErrProfileNotFound` or `ErrInvalidRequest` |
|
||||
| File profile lacks both backend and endpoint | `ErrProfileLoad` |
|
||||
| In-memory profile lacks both backend and endpoint | `ErrInvalidConfig` |
|
||||
| Effective credential environment variable is unset | Existing `ErrAPIKeyEnvMissing` and `ErrInvalidRequest` identities |
|
||||
| Other invalid per-run effective settings | Existing `ErrInvalidRequest` |
|
||||
|
||||
Error text must include the offending backend ID or field where useful, but
|
||||
exact prose is not a compatibility contract. Preserve wrapped internal causes
|
||||
where the existing error boundary permits it.
|
||||
|
||||
### Metadata And Transport
|
||||
|
||||
- Set `ExecutionTarget.BackendID`, `PreparedRun.SelectedBackendID`, and
|
||||
`RunResult.SelectedBackendID` from the normalized selected profile backend.
|
||||
- Endpoint-only profiles leave all three values empty.
|
||||
- Pass `BackendID` through `GenerateRequest.Target` and repair targets so an
|
||||
injected client can route or observe it.
|
||||
- The built-in OpenAI-compatible client must not serialize `backend_id` as a
|
||||
provider request field. It continues to use the effective endpoint,
|
||||
credential, typed generation fields, and extra parameters.
|
||||
- Backend selection does not change rendered-prompt hashes, prompt hashes,
|
||||
session IDs, validation, artifacts, timing, or synchronous `Prepare` and
|
||||
`Run` behavior.
|
||||
|
||||
## Stage 1 — Registry Foundation
|
||||
|
||||
### Goal
|
||||
|
||||
Introduce the internal domain and immutable registry without changing the
|
||||
public configuration or profile format.
|
||||
|
||||
### Work
|
||||
|
||||
1. Add the internal backend definition to `internal/domain`.
|
||||
2. Create `internal/backend` with:
|
||||
- a registry constructor that installs OpenRouter and accepts additional
|
||||
internal definitions;
|
||||
- validation for IDs, endpoints, environment-variable names, JSON values,
|
||||
and reserved request fields;
|
||||
- deterministic built-in and consumer collision handling;
|
||||
- immutable lookup with defensive map copying; and
|
||||
- an internal recognizable not-found error.
|
||||
3. Reuse or extract one validation rule for OpenAI-compatible reserved request
|
||||
fields so registry validation and `internal/llm` payload construction cannot
|
||||
drift. Do not maintain duplicate reserved-key lists.
|
||||
4. Keep JSON-value copying type-preserving. Do not use a marshal/unmarshal
|
||||
round trip that silently changes integer or container types.
|
||||
5. Update `docs/policy/architecture.md` and `docs/internal/overview.md` only
|
||||
after the package exists, describing `internal/backend` as an implemented
|
||||
immutable configuration registry rather than a runtime service manager.
|
||||
|
||||
### Tests
|
||||
|
||||
`internal/backend` owns focused package tests for:
|
||||
|
||||
- the exact OpenRouter built-in values;
|
||||
- successful unique consumer additions;
|
||||
- built-in and consumer duplicate rejection;
|
||||
- blank and whitespace-normalized IDs;
|
||||
- invalid or non-HTTP(S) endpoints;
|
||||
- invalid environment-variable names;
|
||||
- invalid, cyclic, non-finite, empty-key, and reserved-key extra parameters;
|
||||
- not-found lookup; and
|
||||
- mutation isolation of input maps and returned values.
|
||||
|
||||
Use one representative table per validation family rather than one test per
|
||||
branch. A race-specific registry test is unnecessary if the registry is
|
||||
immutable and the final repository race suite exercises concurrent reads.
|
||||
|
||||
### Completion Gate
|
||||
|
||||
- No public declaration or profile behavior has changed.
|
||||
- The registry has no mutating method after construction.
|
||||
- The built-in definition contains no credential value.
|
||||
- Focused backend tests and the complete repository validation pass.
|
||||
|
||||
## Stage 2 — Profile Selection And Effective Resolution
|
||||
|
||||
### Goal
|
||||
|
||||
Make the built-in OpenRouter backend selectable by file and in-memory profiles,
|
||||
migrate the built-in profiles, and expose effective backend identity.
|
||||
|
||||
### Work
|
||||
|
||||
1. Add `BackendID` to internal execution profiles and targets and add the YAML
|
||||
key `backend`.
|
||||
2. Change file and public in-memory profile validation to require `model` plus
|
||||
at least one of `backend` or `endpoint`. Preserve all existing numeric and
|
||||
extra-parameter validation.
|
||||
3. Add `BackendID` to `Profile` and
|
||||
`OpenAICompatibleProfileConfig`, and add `BackendOpenRouter`.
|
||||
4. Add backend identity to the public and internal execution target, prepared
|
||||
run, and run result exactly as specified under **Public API**.
|
||||
5. Add the `BackendResolver` dependency immediately after the profile
|
||||
repository in both runner constructors and update all internal call sites
|
||||
and test helpers explicitly.
|
||||
6. Resolve the selected backend after profile loading and before effective
|
||||
target resolution. Map lookup failure to `ErrProfileLoad`.
|
||||
7. Implement the exact endpoint, credential, and whole-map precedence rules
|
||||
under **Resolution**. Keep backend ID independent from endpoint overrides.
|
||||
8. Construct the built-in-only registry in `NewEngine` and inject it into the
|
||||
runner.
|
||||
9. Replace `endpoint` and `api_key_env` in every
|
||||
`internal/profile/builtin/assets/*.yaml` profile with
|
||||
`backend: openrouter`. Retain each profile's model and generation settings.
|
||||
10. Update conversions and copying so backend IDs survive every public/internal
|
||||
boundary and registry-owned maps remain isolated.
|
||||
11. Update the GoDoc for all changed public declarations.
|
||||
12. Update `docs/formats.md`, `docs/internal/sources.md`, and
|
||||
`docs/internal/runner.md` with the implemented profile field, conditional
|
||||
endpoint rule, built-in selection, and effective resolution behavior.
|
||||
|
||||
### Tests
|
||||
|
||||
Assign test ownership as follows:
|
||||
|
||||
- `internal/profile` tests own strict YAML decoding and the conditional
|
||||
backend-or-endpoint validation matrix.
|
||||
- `internal/profile/builtin` tests own the invariant that every built-in
|
||||
profile selects `openrouter` and no longer repeats its endpoint or
|
||||
credential environment name.
|
||||
- `internal/usecase` tests own backend/profile/request precedence, unknown-ID
|
||||
error wrapping, endpoint override identity, credential precedence including
|
||||
`APIKeyRequired`, and whole-map replacement.
|
||||
- Root external-package contract tests own the new fields, JSON names and
|
||||
omission behavior, public copying, and public error identities.
|
||||
- One assembled engine test must prove a built-in profile prepares with the
|
||||
expected backend ID, endpoint, and environment-variable name without making
|
||||
an HTTP call.
|
||||
|
||||
Do not duplicate every profile-parser validation case at the engine boundary.
|
||||
|
||||
### Completion Gate
|
||||
|
||||
- All built-in and endpoint-only profiles prepare successfully.
|
||||
- A selected unknown backend matches public `ErrProfileLoad`.
|
||||
- Backend identity reaches prepared values, results, generation targets, and
|
||||
repair targets.
|
||||
- No backend ID or secret is added to the outbound provider payload.
|
||||
- Current-state format and internal documentation match the implemented
|
||||
built-in behavior.
|
||||
- The complete repository validation passes.
|
||||
|
||||
## Stage 3 — Consumer Registration
|
||||
|
||||
### Goal
|
||||
|
||||
Expose the engine-scoped extension point for additional unique
|
||||
OpenAI-compatible backend IDs.
|
||||
|
||||
### Work
|
||||
|
||||
1. Add `Backend` and `WithBackend` exactly as specified under **Public API**,
|
||||
preferably in a cohesive root `backends.go` file.
|
||||
2. Convert, validate, normalize, and deeply copy public backend values during
|
||||
`NewEngine`. Store pending additions in `engineOptions`; do not mutate a
|
||||
registry from option application.
|
||||
3. After all options have applied, construct one immutable registry from the
|
||||
built-in and pending consumer definitions. Map every construction failure
|
||||
to `ErrInvalidConfig`.
|
||||
4. Make unique `WithBackend` calls additive. Reject duplicate IDs even when
|
||||
they arrive through separate calls, and document this exception on
|
||||
`Option` and `WithBackend`.
|
||||
5. Make registered backends available equally to directory, `fs.FS`, single
|
||||
file, in-memory, prompt-default, and explicit request profile selection.
|
||||
6. Preserve the last-valid-option-wins behavior of every existing option
|
||||
category.
|
||||
7. Update `docs/consumers/pkg-promptkit.md` with one minimal custom backend and
|
||||
profile example. Keep exact field-by-field API detail in GoDoc and link to
|
||||
it rather than duplicating it.
|
||||
|
||||
### Tests
|
||||
|
||||
Root public contract tests own:
|
||||
|
||||
- one custom backend used by an in-memory profile;
|
||||
- one custom backend used by a file-backed profile;
|
||||
- multiple unique `WithBackend` calls accumulating;
|
||||
- duplicate consumer and built-in ID failures matching `ErrInvalidConfig`;
|
||||
- invalid public values and nested-map mutation isolation;
|
||||
- endpoint-only compatibility with no `WithBackend` option;
|
||||
- profile endpoint and request endpoint overrides retaining the custom backend
|
||||
ID; and
|
||||
- an injected `LLMClient` observing the effective custom backend ID and
|
||||
settings.
|
||||
|
||||
Use an injected client or `httptest.Server`; never contact the configured
|
||||
external endpoint. Keep lower-level registry validation cases in
|
||||
`internal/backend`.
|
||||
|
||||
### Completion Gate
|
||||
|
||||
- Consumers can add only new IDs and cannot mutate or replace built-ins.
|
||||
- Registration is engine-local and two engines can use different definitions
|
||||
for the same consumer ID without interference.
|
||||
- Existing option-category contract tests still pass unchanged except for the
|
||||
documented additive backend case.
|
||||
- The consumer guide and GoDoc describe the implemented extension point.
|
||||
- The complete repository validation passes.
|
||||
|
||||
## Stage 4 — Contract Hardening And Documentation Completion
|
||||
|
||||
### Goal
|
||||
|
||||
Audit the completed feature across public, profile, use-case, and transport
|
||||
boundaries and make the current-state documentation self-sufficient.
|
||||
|
||||
### Work
|
||||
|
||||
1. Review the preceding tests as a suite. Remove redundant cases and retain a
|
||||
lean precedence matrix that would catch layer-order, credential, copying,
|
||||
compatibility, and error-identity regressions.
|
||||
2. Add or adjust one built-in-client transport test proving that effective
|
||||
backend defaults reach the configured endpoint while `backend_id` is not
|
||||
serialized. Use `httptest.Server` and synthetic credentials.
|
||||
3. Confirm public stable-JSON round trips for `ExecutionTarget`,
|
||||
`PreparedRun`, and `RunResult`, including empty backend omission for legacy
|
||||
profiles.
|
||||
4. Confirm concurrent `Prepare` and `Run` calls can read the immutable registry
|
||||
under the race detector. Do not add concurrency limiting or a queue.
|
||||
5. Update all affected current-state owners:
|
||||
- public GoDoc for the exact API and errors;
|
||||
- `docs/formats.md` for profile YAML and built-ins;
|
||||
- `docs/consumers/pkg-promptkit.md` for construction and use;
|
||||
- `docs/integrations/openai-compatible-chat.md` for resolved endpoint,
|
||||
credentials, request defaults, and the non-serialized routing identity;
|
||||
- `docs/internal/overview.md`, `docs/internal/sources.md`,
|
||||
`docs/internal/runner.md`, and `docs/internal/llm.md` for package
|
||||
responsibilities and data flow; and
|
||||
- `docs/policy/architecture.md` only to keep its implemented component
|
||||
inventory and dependency description accurate.
|
||||
6. Search current-state documentation for obsolete statements that every
|
||||
profile requires an endpoint or that every built-in repeats OpenRouter
|
||||
connection settings. Update the canonical owner and replace duplicates with
|
||||
links.
|
||||
7. Keep future concurrency and queue behavior only in
|
||||
[the future feature catalog](future.md); do not imply that this feature
|
||||
implements either capability.
|
||||
|
||||
### Final Validation
|
||||
|
||||
Run the complete maintainer sequence from the repository root:
|
||||
|
||||
```sh
|
||||
go test ./...
|
||||
go test -race ./...
|
||||
go vet ./...
|
||||
go build ./...
|
||||
go run ./examples/go-library/prepare
|
||||
gofmt -l $(git ls-files '*.go')
|
||||
git diff --check
|
||||
```
|
||||
|
||||
The formatting command must produce no paths. Follow every changed Markdown
|
||||
link and confirm its target exists. Inspect the final diff for raw credentials,
|
||||
global mutable state, generated workspace files, local `replace` directives,
|
||||
and unrelated changes.
|
||||
|
||||
### Completion Gate
|
||||
|
||||
- Every target-end-state item in [the feature roadmap](backends.md) is
|
||||
implemented and protected at its owning boundary.
|
||||
- The public API is limited to the fixed declarations in this plan.
|
||||
- Endpoint-only profiles and injected clients remain compatible.
|
||||
- Built-in profiles obtain OpenRouter endpoint and credential metadata only
|
||||
through the registry.
|
||||
- Current-state documentation no longer relies on either roadmap to explain
|
||||
implemented behavior.
|
||||
- All final validation commands pass offline.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None. The API shape, validation, precedence, compatibility behavior, error
|
||||
identities, package ownership, test ownership, and staging required for
|
||||
implementation are fixed by this plan.
|
||||
Reference in New Issue
Block a user