diff --git a/docs/roadmap/backends.md b/docs/roadmap/backends.md new file mode 100644 index 0000000..d2a7816 --- /dev/null +++ b/docs/roadmap/backends.md @@ -0,0 +1,180 @@ +# Extensible LLM Backend Registry + +## Purpose + +This roadmap defines the scope and target end state for formal +OpenAI-compatible backend support in Promptkit. It establishes the behavioral +boundary and policy choices for the work. + +This document is planning material, not a description of current behavior. +Current public contracts remain owned by Go declarations and GoDoc, framework +file formats by the [format reference](../formats.md), and outbound HTTP +behavior by the +[OpenAI-compatible integration contract](../integrations/openai-compatible-chat.md). + +## Motivation + +Execution profiles currently carry backend connection and authentication +details alongside model and generation settings. This repeats values such as +the OpenRouter endpoint and `OPENROUTER_API_KEY` across profiles, leaves the +built-in catalog implicitly tied to one backend, and gives consumers no +explicit extension point for naming other OpenAI-compatible services. + +Promptkit should distinguish: + +- a **backend**, which identifies reusable connection, authentication, and + limited backend-wide request defaults; and +- a **profile**, which selects a model and its reusable generation settings. + +That distinction should let consumers configure OpenRouter, OpenAI, +local-network services, or other OpenAI-compatible deployments without +duplicating backend defaults in every profile or replacing Promptkit's model +client. + +## Scope + +The work introduces an engine-scoped registry of OpenAI-compatible backend +definitions. + +Each backend definition will provide: + +- a stable, non-empty backend ID; +- a default OpenAI-compatible base endpoint; +- an optional environment-variable name for its API key; and +- limited JSON-compatible request defaults when they are genuinely + backend-wide rather than model-specific. + +Promptkit will provide a small built-in backend catalog, initially containing +OpenRouter. Downstream consumers will be able to register additional backend +IDs during engine construction. Registration will be explicit and local to an +engine; it will not mutate package-global state. + +Profiles will be able to name a backend. A profile that selects a backend may +override its default endpoint and other supported defaults while continuing to +own its model and generation settings. Existing endpoint-based custom profiles +will remain supported without requiring a backend ID. + +The built-in model profile catalog will use the built-in OpenRouter backend +instead of repeating OpenRouter connection and credential defaults in every +profile. + +## Resolution And Precedence + +Effective execution settings will resolve in this order: + +1. application-neutral framework defaults; +2. selected backend defaults, when a backend is named; +3. selected profile values; and +4. explicit per-run overrides. + +A profile endpoint will override its selected backend's endpoint. A per-run +endpoint override will continue to take precedence over both. + +Credential selection will preserve direct request credentials as the highest +precedence. An explicit per-run environment-variable override will take +precedence over profile credential configuration, which will take precedence +over the backend's default environment-variable name. Backend definitions and +profiles will contain credential-source metadata only, never resolved secret +values. + +Backend request defaults, profile request values, and per-run request +overrides will follow one deterministic replacement rule. Resolution must not +introduce implicit deep merging whose result depends on map iteration or +incidental representation. + +## Registration And Validation + +Backend registration will be deterministic and validated during engine +construction. + +- Built-in backend IDs are reserved and cannot be replaced by consumers. +- Consumer registrations may add only new IDs. +- Duplicate consumer IDs are invalid, including duplicates introduced through + repeated configuration. +- Backend IDs, endpoints, credential-source metadata, and request defaults + must be validated before an engine is returned. +- Selecting an unknown backend is an error associated with the profile or + request boundary that selected it. +- A profile without a backend must continue to provide the connection details + required by the current endpoint-based path. +- A backend may omit credential requirements so unauthenticated local-network + services remain supported. + +Failures will retain error identities appropriate to the existing engine +configuration, profile-loading, and per-run validation boundaries. + +## Public And Extension Boundaries + +The effective backend ID will be observable in prepared and completed run +metadata and will be available to injected model clients. This gives consumers +and extensions a stable routing identity without requiring them to infer a +backend from an endpoint URL. + +The existing public model-client injection boundary will remain supported. +Because the initial registry supports only OpenAI-compatible backends, +Promptkit does not need backend-specific transport factories or multiple +protocol implementations in this scope. The built-in client will continue to +send the resolved execution target to the selected OpenAI-compatible endpoint. + +Backend definitions are configuration values, not live service objects. They +will not own mutable connections, credentials, health state, or process +lifecycle. + +## Compatibility + +The target end state preserves the existing ways consumers configure and run +Promptkit: + +- Profiles with explicit endpoints and no backend ID continue to work. +- Profile endpoint overrides remain supported. +- Direct request API keys and environment-variable overrides retain their + precedence. +- Consumers may continue injecting a custom model client. +- Model IDs and generation settings remain profile concerns. +- Prompt selection, rendering, validation, artifacts, and synchronous + `Prepare` and `Run` behavior remain unchanged except for exposing the + resolved backend identity. + +## Non-Goals + +This scope does not include: + +- non-OpenAI-compatible protocols or provider-specific SDKs; +- backend-specific concurrency limits or admission queues; +- asynchronous task submission or durable jobs; +- model discovery, live model catalogs, or provider capability probing; +- retries, rate limiting, failover, load balancing, or health checks; +- automatic backend selection based on model names or endpoint inspection; +- global mutable registration; +- consumer replacement of built-in backend IDs; +- credential discovery beyond configured direct values and environment + variables; or +- provider-specific application, deployment, or security policy. + +The dependent concurrency and bounded-queue work remains a separate future +unit after backend identity and resolution are stable. + +## Target End State + +This roadmap reaches its target end state when: + +- Promptkit has an immutable, engine-scoped registry of validated + OpenAI-compatible backend definitions; +- OpenRouter is available as a built-in backend with its endpoint and + credential environment-variable default; +- consumers can register additional unique backend IDs without modifying + Promptkit; +- profiles can select a backend and optionally override its endpoint; +- existing endpoint-only profiles remain valid; +- backend, profile, and per-run values resolve through documented, + deterministic precedence; +- secret values remain outside backend and profile definitions; +- the effective backend identity is available in preparation, execution, and + injected-client values; +- the built-in model profiles use the OpenRouter backend rather than duplicate + its connection defaults; +- the built-in OpenAI-compatible client and injected clients continue to work + through the existing generation boundary; and +- current-state GoDoc, format, consumer, integration, architecture, and + internal documentation describe the implemented behavior without relying on + this roadmap. diff --git a/docs/roadmap/documentation.md b/docs/roadmap/documentation.md deleted file mode 100644 index 13abaee..0000000 --- a/docs/roadmap/documentation.md +++ /dev/null @@ -1,222 +0,0 @@ -# Documentation Hardening Roadmap - -## Purpose - -This roadmap coordinates a focused pass over Promptkit's documentation and -public contract. The work should close the gaps identified by the documentation -audit, strengthen canonical ownership, and leave consumers and contributors -with guidance that is accurate, navigable, and proportionate to their needs. - -This is planning material, not a description of implemented behavior. Follow -the [documentation policy](../policy/documentation.md) throughout the work and -update current-state documents only when their claims are supported by the -implementation and tests. - -## Scope And Principles - -The work covers public GoDoc, consumer guidance, maintained examples, format -and integration references, architecture ownership, roadmap lifecycle, and -documentation validation. - -- Resolve ambiguous behavior before documenting it as a contract. -- Keep exact exported API semantics in Go declarations and GoDoc. -- Keep task-oriented guidance in consumer documents and complete runnable - artifacts in `examples/`. -- Put security-relevant consumer responsibilities at the public boundary, not - only in internal contributor documents. -- Remove duplicated ownership instead of synchronizing parallel references. -- Make documentation checks reproducible where practical. -- Preserve existing behavior unless a stage explicitly selects and tests an - API change. - -The roadmap does not add new Promptkit features, redesign framework formats, -or implement ideas from [the future feature catalog](future.md). If resolving -an ambiguity requires a behavioral change, treat that change as a separately -reviewable implementation unit and update its canonical documentation in the -same unit. - -## Stage 1: Resolve Public Contract Questions - -Before expanding prose, decide the intended contract for exported behavior -that is currently ambiguous. - -- [x] Decide whether `RunRequest.Metadata` has a supported observable purpose. - Define its propagation and ownership, or remove or deprecate it through an - intentional public API change. -- [x] Decide and test whether one `Engine` supports concurrent `Prepare` and - `Run` calls. -- [x] Decide how repeated options of the same category behave, including - prompt, profile, schema, model-client, and artifact-reader options. -- [x] Define which public values have supported JSON representations. -- [x] Define JSON time units and omission behavior, including the relationship - between prepared-run and run-result durations. -- [x] Decide whether run IDs and exposed hashes have stable formats or must be - treated as opaque values. -- [x] Confirm the intended transport-timeout default and its zero or negative - configuration semantics. -- [x] Confirm the supported JSON Schema dialect and reference boundaries, - including whether remote references are allowed. - -The selected exported API contracts are implemented and tested. Their durable -definitions now belong to the root package declarations and GoDoc. - -One format-level decision remains here until Stage 5 moves it to the framework -format reference: JSON Schema uses Draft 2020-12, with that dialect selected -when `$schema` is omitted. Same-document fragments and relative references -contained by a directory or `fs.FS` schema root are supported. A single-file -source supports only references contained in that document. Absolute, -escaping, and remote references are rejected. - -**Gate:** Each question has an explicit answer backed by existing behavior or -by an accepted implementation change and proportionate tests. No later stage -should invent a contract merely to fill a documentation gap. - -## Stage 2: Make GoDoc The Canonical Public Contract - -Strengthen the root package declarations so `go doc` is sufficient to -understand exact public behavior without relying on internal documents. - -- [x] Add useful field-level GoDoc to configuration, request, profile, - execution-target, result, artifact, validation, structured-output, and model - client values. -- [x] Document required fields and nil, empty, and zero-value semantics. -- [x] Document override, replacement, profile-precedence, and copy-ownership - behavior where it belongs to the exported API. -- [x] Document credential inputs, redaction, and the values intentionally - excluded from serialization. -- [x] Give each public error sentinel an accurate comment and document the - supported `errors.Is` relationships. -- [x] Document engine concurrency and option-composition behavior selected in - Stage 1. -- [x] Document serialization, time, run-ID, and hash semantics selected in - Stage 1. -- [x] Review constructor, option, extension-interface, `Prepare`, and `Run` - GoDoc for complete failure and cancellation expectations. - -Update the [consumer guide](../consumers/pkg-promptkit.md) to summarize and link -to these contracts instead of maintaining exhaustive copies of exported names -or exact semantics. - -**Gate:** `go doc -all .` presents a coherent public contract, exported -declarations have accurate comments, and contract tests protect every newly -documented behavior whose compatibility risk warrants durable coverage. - -## Stage 3: Improve Consumer Safety And Executable Guidance - -Move consumer-relevant security boundaries to the places where consumers will -encounter them and add one representative execution workflow. - -- [x] Explain in public GoDoc and the consumer guide that the default file - artifact reader accepts unrestricted caller-selected paths. -- [x] Make clear that Promptkit does not impose an application root, inbound - request-size policy, or untrusted-input security boundary. -- [x] Explain that rendered messages, artifact bodies, raw model output, and - validation details may be sensitive even when credentials are redacted. -- [x] Clarify the responsibilities of injected artifact readers and model - clients for cancellation, copying, logging, and secret handling. -- [x] Add a maintained offline `Run` example using an injected deterministic - model client, without credentials, live network access, or paid calls. -- [x] Link the consumer guide to the execution example and keep embedded - snippets smaller than the maintained artifact. -- [x] Decide whether the existing preparation example should remain separate - or share reusable fixtures without obscuring either workflow. - -The preparation and execution examples remain separate, self-contained -workflows. Each keeps its own small prompt fixture so consumers can copy or run -one example without depending on the other. - -**Gate:** Both preparation and execution have complete, secret-free, -deterministic consumer examples, and the consumer guide exposes the important -filesystem and data-sensitivity boundaries without leaking internal mechanics. - -## Stage 4: Restore Canonical Ownership - -Remove parallel definitions and make navigation follow the ownership model in -the documentation policy. - -- [ ] Reduce the [architecture policy](../policy/architecture.md) to durable - boundaries, layers, dependency direction, invariants, and non-goals. -- [ ] Keep the exact implemented package and component inventory solely in the - [internal component overview](../internal/overview.md). -- [ ] Review the consumer guide's public error and option lists so they remain - task-oriented summaries rather than duplicate API references. -- [ ] Review internal documents for public-contract statements that should be - links to GoDoc or the format and integration owners. -- [ ] Reconcile the documentation policy's temporary-roadmap lifecycle with - the continuing idea-catalog role of `docs/roadmap/future.md`. -- [ ] Rephrase or link roadmap statements that depend on exact current API - behavior, particularly the runtime reasoning entry. -- [ ] Confirm that every document states its audience or purpose and links to - the canonical owner of adjacent topics. - -**Gate:** Every authoritative fact has one clear owner, package inventory -changes no longer require edits to the architecture policy, and roadmaps -cannot be mistaken for current-state references. - -## Stage 5: Refine Format And Integration References - -Close compatibility gaps in the documents that own file formats and outbound -wire behavior. - -- [ ] State or canonically link the exact session-ID limit enforced by the - OpenAI-compatible client. -- [ ] State the configured and default transport-timeout behavior without - referring to an unnamed internal default. -- [ ] Document the JSON Schema dialect and local, contained, and remote - reference behavior selected in Stage 1. -- [ ] Clarify structured-output naming and strictness when those values are - part of the public or integration contract. -- [ ] Add a caveat that built-in profiles are maintained configurations, not a - guarantee of continuing third-party model availability. -- [ ] Recheck every prompt, profile, schema, credential, request-body, response, - timeout, and precedence statement against its owning implementation and - tests. - -**Gate:** A consumer can determine the supported file and wire compatibility -boundaries without consulting internal source code or relying on unspecified -defaults. - -## Stage 6: Make Documentation Validation Reproducible - -Align contributor and release procedures around a small, consistent set of -checks. - -- [ ] Use one robust command for checking every tracked Go file with `gofmt`. -- [ ] Provide a repository-local or clearly documented command that validates - local Markdown targets and heading fragments. -- [ ] Decide how published external links are checked without making ordinary - validation depend on mutable network services. -- [ ] Reconcile the validation descriptions in the - [development guide](../development.md), - [testing policy](../policy/testing.md), and - [release procedure](../release.md) so one document owns each requirement. -- [ ] Ensure example validation covers every maintained example added by this - roadmap. -- [ ] Keep documentation-only validation proportionate while requiring full Go - validation when commands, examples, generated output, or checked behavior - changes. - -**Gate:** A maintainer can run the documented formatting, link, example, Go, -and repository-hygiene checks exactly as written, with no hidden manual -procedure for local documentation. - -## Completion Criteria - -The roadmap is complete when: - -- all Stage 1 contract questions are resolved; -- GoDoc is the authoritative and sufficient exported API reference; -- consumer guidance covers unrestricted file access and sensitive generated - data; -- maintained offline examples cover both `Prepare` and `Run`; -- architecture, inventory, consumer, internal, format, integration, and - roadmap documents follow their assigned ownership boundaries; -- schema, session, timeout, structured-output, and built-in-profile - compatibility statements are explicit; -- documentation validation is reproducible and consistent across contributor - and release workflows; and -- the complete maintainer validation passes. - -After completion, move any durable decisions to GoDoc, policy, format, -integration, or ADR owners as appropriate. Remove this roadmap after incoming -links are updated; do not retain it as a second current-state reference. diff --git a/docs/roadmap/future.md b/docs/roadmap/future.md index b22b84a..9caeac9 100644 --- a/docs/roadmap/future.md +++ b/docs/roadmap/future.md @@ -33,29 +33,15 @@ consumers. ## Ideas -### Extensible LLM backend registry - -Introduce a registry that separates backend-specific connection, -authentication, and limited request defaults from model execution profiles. -Initial support would cover OpenAI-compatible backends and include a small -built-in catalog, potentially starting with OpenRouter. A profile could select -a backend while optionally overriding its default endpoint, and each backend -could name an optional environment variable for its API key without storing -the credential itself. Downstream consumers could register additional, -uniquely named backends, such as OpenAI or unauthenticated local-network -services, but could not replace built-in IDs. Model selection and generation -settings would remain profile concerns, and custom model clients would remain -available for behavior outside the registry's supported protocol. - ### Backend-specific concurrency management -Extend the proposed LLM backend registry with optional per-backend concurrency -limits and bounded, buffered admission queues. Promptkit could then route -simultaneous generation requests according to backend capacity while -containing accidental runaway submission. Downstream consumers would continue -invoking synchronous `Run` calls, including concurrently from multiple -goroutines, and each admitted call would wait for and return its ordinary -result. +Extend the selected +[LLM backend registry](backends.md) with optional per-backend concurrency limits +and bounded, buffered admission queues. Promptkit could then route simultaneous +generation requests according to backend capacity while containing accidental +runaway submission. Downstream consumers would continue invoking synchronous +`Run` calls, including concurrently from multiple goroutines, and each +admitted call would wait for and return its ordinary result. - Scope limits to an engine instance rather than hidden process-global state. - Give different backend IDs independent capacity pools. A profile endpoint diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md new file mode 100644 index 0000000..e9ce094 --- /dev/null +++ b/docs/roadmap/implementation.md @@ -0,0 +1,497 @@ +# Extensible LLM Backend Registry Implementation Plan + +## Purpose + +This document is the decision-complete implementation plan for the +[extensible LLM backend registry](backends.md). It is written for a coding +agent that will implement each stage in order. + +This is planning material, not a description of current behavior. During +implementation, update current-state GoDoc and documentation only as the +corresponding behavior lands. The feature roadmap owns the intended end state +and policy choices; this document owns API shape, package changes, sequencing, +tests, and completion gates. + +## Implementation Rules + +- Complete the stages in order. Do not begin a later stage while an earlier + stage's completion gate is unmet. +- Preserve unrelated working-tree changes and do not broaden the feature into + concurrency control, queues, retries, provider discovery, or + non-OpenAI-compatible transports. +- Keep the registry immutable and scoped to one `Engine`. Do not add + package-global registration or mutation after construction. +- Keep secrets out of backend definitions, profiles, prepared values, results, + hashes, logs, and JSON. Backends store only an environment-variable name. +- Preserve endpoint-only profiles and custom `LLMClient` injection. +- Follow the architecture, documentation, and testing policies under + `docs/policy/`. In particular, keep the public API in the root package, + place implementation packages under `internal/`, update the component + inventory when the new package lands, and assign each behavior to one + durable test owner. +- Use deterministic, offline tests. No stage may require a real provider, + credential, or network service. + +## Fixed Design + +### Public API + +Add the following root-package declarations: + +```go +const BackendOpenRouter = "openrouter" + +type Backend struct { + ID string + Endpoint string + APIKeyEnv string + ExtraParams map[string]any +} + +func WithBackend(backend Backend) Option +``` + +`Backend` and `WithBackend` have these contracts: + +- `Backend` configures one OpenAI-compatible backend and has no stable JSON + representation. +- `ID` is trimmed, non-empty, case-sensitive, and is the stable registry key. +- `Endpoint` is trimmed and must be an absolute `http` or `https` URL with a + host. Paths are allowed; user information, query strings, and fragments are + rejected because this value is a base endpoint rather than a complete + request URL. +- `APIKeyEnv` is optional. When present, it is trimmed and must match the + portable environment-variable form `[A-Za-z_][A-Za-z0-9_]*`. +- `ExtraParams` follows the existing `Profile.ExtraParams` JSON-value rules: + non-empty string keys, finite numbers, JSON-compatible scalar and container + values, no cycles, and no collisions with `model`, `session_id`, `messages`, + `temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, or + `response_format`. It is deeply copied during `NewEngine`. +- An empty `ExtraParams` map means that the backend supplies no request + defaults. +- Each `WithBackend` call adds one registration. Calls with different IDs + accumulate in option order; a repeated ID is an error rather than a + replacement. This is an explicit additive exception to the last-option-wins + categories documented on `Option`. +- `BackendOpenRouter` is reserved. Consumers cannot replace it. +- There is no public enumerate, lookup, remove, replace, or post-construction + registration API in this scope. + +Extend the existing public values as follows: + +| Value | Addition | Contract | +| --- | --- | --- | +| `Profile` | `BackendID string` | Optional backend selection. `Endpoint` is required only when `BackendID` is blank. | +| `OpenAICompatibleProfileConfig` | `BackendID string` | Copied to `Profile.BackendID`. | +| `ExecutionTarget` | `BackendID string \`json:"backend_id,omitempty"\`` | Effective routing identity supplied to injected clients; empty for endpoint-only profiles. | +| `PreparedRun` | `SelectedBackendID string \`json:"selected_backend_id,omitempty"\`` | Equals the effective target backend ID. | +| `RunResult` | `SelectedBackendID string \`json:"selected_backend_id,omitempty"\`` | Carries the prepared backend identity through execution. | + +Do not add a backend field to `RunRequest` or `ExecutionTargetOverride`. +Backend selection remains a profile concern. Endpoint overrides do not change +the selected backend identity. + +Trim `Profile.BackendID` and YAML `backend` values at their respective +conversion/load boundaries. Treat an all-whitespace value as absent and expose +only the trimmed value in effective targets and metadata. + +These exported struct fields are intentionally additive. Adding fields can +break downstream unkeyed composite literals even though keyed literals and +ordinary field access remain source-compatible. Accept that narrow risk; use +and document keyed literals, and do not add parallel wrapper types or a second +configuration path solely to preserve unkeyed literals. + +### Profile Format + +Add the optional strict-YAML field: + +```yaml +backend: openrouter +``` + +The profile connection rule becomes: + +- `model` remains required; +- at least one of `backend` or `endpoint` is required; +- when both are present, the profile endpoint overrides the backend endpoint; + and +- a profile with only an endpoint continues through the legacy path and has an + empty effective backend ID. + +Do not infer a backend from a model name or endpoint. Do not validate whether a +backend ID exists while decoding a profile; registry membership is +engine-scoped and is checked when the selected profile is prepared. + +### Registry And Internal Boundaries + +Add `internal/backend` as the cohesive owner of: + +- the immutable backend registry; +- backend-definition validation and defensive copying; +- the built-in OpenRouter definition; and +- the internal not-found error used when a selected ID is absent. + +Add an internal domain backend value containing `ID`, `Endpoint`, `APIKeyEnv`, +and `ExtraParams`. The registry constructor accepts consumer additions, installs +built-ins first, rejects all collisions, and returns a fully constructed +read-only registry. Lookup returns a defensive value so callers cannot mutate +registry-owned maps. + +Use these internal declarations: + +```go +var ErrBackendNotFound = errors.New("backend not found") + +func NewRegistry(additions []domain.Backend) (*Registry, error) +func (r *Registry) GetBackend(id string) (domain.Backend, error) +``` + +`internal/usecase` owns the narrow resolver interface it consumes: + +```go +type BackendResolver interface { + GetBackend(string) (domain.Backend, error) +} +``` + +The concrete registry implements this interface. Supply it explicitly to +`Runner` through both runner constructors and from the root engine assembly. +A nil resolver must never panic; if an internally constructed runner selects a +backend without a resolver, preparation fails as a profile-load failure. + +The built-in definition is exactly: + +| Field | Value | +| --- | --- | +| ID | `openrouter` | +| Endpoint | `https://openrouter.ai/api/v1` | +| API key environment variable | `OPENROUTER_API_KEY` | +| Extra parameters | none | + +Define `OpenRouterID` as a constant in `internal/backend` and define the public +`BackendOpenRouter` constant from it so there is one canonical literal. This +root-to-internal implementation import does not expose an internal type in a +public signature. + +### Resolution + +Resolve an execution target in this order: + +1. framework defaults; +2. the selected backend, if any; +3. the selected profile; and +4. the request's `ExecutionTargetOverride`. + +The layers behave as follows: + +| Setting | Backend layer | Profile layer | Request layer | +| --- | --- | --- | --- | +| Backend ID | Supplies selected ID | Selects the backend | Cannot change it | +| Endpoint | Supplies default | Non-empty value replaces backend | Non-empty value replaces profile/backend | +| Model and generation fields | Not supplied | Existing profile behavior | Existing override behavior | +| API-key environment name | Supplies default | Non-empty `api_key_env` replaces it | Non-empty `APIKeyEnv` replaces it | +| Extra parameters | Non-empty map replaces prior map | Non-empty map replaces complete backend map | Non-empty map replaces complete profile/backend map | + +Never deep-merge `ExtraParams` maps. A non-empty higher-precedence map replaces +the complete lower-precedence map. Preserve the existing distinction between a +nil or empty override map and a supplied non-empty replacement. + +Credential handling remains: + +1. a non-blank `RunRequest.APIKey` wins and suppresses environment lookup; +2. a non-blank request `APIKeyEnv` wins over profile and backend metadata; +3. file-profile `api_key_env` wins over the backend default; +4. backend `APIKeyEnv` is used when no higher layer supplies a credential + source; and +5. a profile with `APIKeyRequired: true` clears an inherited backend + environment name and requires a direct key unless the request explicitly + supplies its own `APIKeyEnv`. + +Only the environment-variable name appears in effective targets. Preserve the +current just-in-time environment lookup and missing-variable behavior. + +### Errors + +Do not add a public backend-specific error sentinel. + +| Failure | Required public identity | +| --- | --- | +| Invalid consumer backend definition | `ErrInvalidConfig` from `NewEngine` | +| Consumer ID duplicates another consumer ID | `ErrInvalidConfig` | +| Consumer ID collides with `BackendOpenRouter` | `ErrInvalidConfig` | +| Selected profile names an unknown backend | `ErrProfileLoad`, not `ErrProfileNotFound` or `ErrInvalidRequest` | +| File profile lacks both backend and endpoint | `ErrProfileLoad` | +| In-memory profile lacks both backend and endpoint | `ErrInvalidConfig` | +| Effective credential environment variable is unset | Existing `ErrAPIKeyEnvMissing` and `ErrInvalidRequest` identities | +| Other invalid per-run effective settings | Existing `ErrInvalidRequest` | + +Error text must include the offending backend ID or field where useful, but +exact prose is not a compatibility contract. Preserve wrapped internal causes +where the existing error boundary permits it. + +### Metadata And Transport + +- Set `ExecutionTarget.BackendID`, `PreparedRun.SelectedBackendID`, and + `RunResult.SelectedBackendID` from the normalized selected profile backend. +- Endpoint-only profiles leave all three values empty. +- Pass `BackendID` through `GenerateRequest.Target` and repair targets so an + injected client can route or observe it. +- The built-in OpenAI-compatible client must not serialize `backend_id` as a + provider request field. It continues to use the effective endpoint, + credential, typed generation fields, and extra parameters. +- Backend selection does not change rendered-prompt hashes, prompt hashes, + session IDs, validation, artifacts, timing, or synchronous `Prepare` and + `Run` behavior. + +## Stage 1 — Registry Foundation + +### Goal + +Introduce the internal domain and immutable registry without changing the +public configuration or profile format. + +### Work + +1. Add the internal backend definition to `internal/domain`. +2. Create `internal/backend` with: + - a registry constructor that installs OpenRouter and accepts additional + internal definitions; + - validation for IDs, endpoints, environment-variable names, JSON values, + and reserved request fields; + - deterministic built-in and consumer collision handling; + - immutable lookup with defensive map copying; and + - an internal recognizable not-found error. +3. Reuse or extract one validation rule for OpenAI-compatible reserved request + fields so registry validation and `internal/llm` payload construction cannot + drift. Do not maintain duplicate reserved-key lists. +4. Keep JSON-value copying type-preserving. Do not use a marshal/unmarshal + round trip that silently changes integer or container types. +5. Update `docs/policy/architecture.md` and `docs/internal/overview.md` only + after the package exists, describing `internal/backend` as an implemented + immutable configuration registry rather than a runtime service manager. + +### Tests + +`internal/backend` owns focused package tests for: + +- the exact OpenRouter built-in values; +- successful unique consumer additions; +- built-in and consumer duplicate rejection; +- blank and whitespace-normalized IDs; +- invalid or non-HTTP(S) endpoints; +- invalid environment-variable names; +- invalid, cyclic, non-finite, empty-key, and reserved-key extra parameters; +- not-found lookup; and +- mutation isolation of input maps and returned values. + +Use one representative table per validation family rather than one test per +branch. A race-specific registry test is unnecessary if the registry is +immutable and the final repository race suite exercises concurrent reads. + +### Completion Gate + +- No public declaration or profile behavior has changed. +- The registry has no mutating method after construction. +- The built-in definition contains no credential value. +- Focused backend tests and the complete repository validation pass. + +## Stage 2 — Profile Selection And Effective Resolution + +### Goal + +Make the built-in OpenRouter backend selectable by file and in-memory profiles, +migrate the built-in profiles, and expose effective backend identity. + +### Work + +1. Add `BackendID` to internal execution profiles and targets and add the YAML + key `backend`. +2. Change file and public in-memory profile validation to require `model` plus + at least one of `backend` or `endpoint`. Preserve all existing numeric and + extra-parameter validation. +3. Add `BackendID` to `Profile` and + `OpenAICompatibleProfileConfig`, and add `BackendOpenRouter`. +4. Add backend identity to the public and internal execution target, prepared + run, and run result exactly as specified under **Public API**. +5. Add the `BackendResolver` dependency immediately after the profile + repository in both runner constructors and update all internal call sites + and test helpers explicitly. +6. Resolve the selected backend after profile loading and before effective + target resolution. Map lookup failure to `ErrProfileLoad`. +7. Implement the exact endpoint, credential, and whole-map precedence rules + under **Resolution**. Keep backend ID independent from endpoint overrides. +8. Construct the built-in-only registry in `NewEngine` and inject it into the + runner. +9. Replace `endpoint` and `api_key_env` in every + `internal/profile/builtin/assets/*.yaml` profile with + `backend: openrouter`. Retain each profile's model and generation settings. +10. Update conversions and copying so backend IDs survive every public/internal + boundary and registry-owned maps remain isolated. +11. Update the GoDoc for all changed public declarations. +12. Update `docs/formats.md`, `docs/internal/sources.md`, and + `docs/internal/runner.md` with the implemented profile field, conditional + endpoint rule, built-in selection, and effective resolution behavior. + +### Tests + +Assign test ownership as follows: + +- `internal/profile` tests own strict YAML decoding and the conditional + backend-or-endpoint validation matrix. +- `internal/profile/builtin` tests own the invariant that every built-in + profile selects `openrouter` and no longer repeats its endpoint or + credential environment name. +- `internal/usecase` tests own backend/profile/request precedence, unknown-ID + error wrapping, endpoint override identity, credential precedence including + `APIKeyRequired`, and whole-map replacement. +- Root external-package contract tests own the new fields, JSON names and + omission behavior, public copying, and public error identities. +- One assembled engine test must prove a built-in profile prepares with the + expected backend ID, endpoint, and environment-variable name without making + an HTTP call. + +Do not duplicate every profile-parser validation case at the engine boundary. + +### Completion Gate + +- All built-in and endpoint-only profiles prepare successfully. +- A selected unknown backend matches public `ErrProfileLoad`. +- Backend identity reaches prepared values, results, generation targets, and + repair targets. +- No backend ID or secret is added to the outbound provider payload. +- Current-state format and internal documentation match the implemented + built-in behavior. +- The complete repository validation passes. + +## Stage 3 — Consumer Registration + +### Goal + +Expose the engine-scoped extension point for additional unique +OpenAI-compatible backend IDs. + +### Work + +1. Add `Backend` and `WithBackend` exactly as specified under **Public API**, + preferably in a cohesive root `backends.go` file. +2. Convert, validate, normalize, and deeply copy public backend values during + `NewEngine`. Store pending additions in `engineOptions`; do not mutate a + registry from option application. +3. After all options have applied, construct one immutable registry from the + built-in and pending consumer definitions. Map every construction failure + to `ErrInvalidConfig`. +4. Make unique `WithBackend` calls additive. Reject duplicate IDs even when + they arrive through separate calls, and document this exception on + `Option` and `WithBackend`. +5. Make registered backends available equally to directory, `fs.FS`, single + file, in-memory, prompt-default, and explicit request profile selection. +6. Preserve the last-valid-option-wins behavior of every existing option + category. +7. Update `docs/consumers/pkg-promptkit.md` with one minimal custom backend and + profile example. Keep exact field-by-field API detail in GoDoc and link to + it rather than duplicating it. + +### Tests + +Root public contract tests own: + +- one custom backend used by an in-memory profile; +- one custom backend used by a file-backed profile; +- multiple unique `WithBackend` calls accumulating; +- duplicate consumer and built-in ID failures matching `ErrInvalidConfig`; +- invalid public values and nested-map mutation isolation; +- endpoint-only compatibility with no `WithBackend` option; +- profile endpoint and request endpoint overrides retaining the custom backend + ID; and +- an injected `LLMClient` observing the effective custom backend ID and + settings. + +Use an injected client or `httptest.Server`; never contact the configured +external endpoint. Keep lower-level registry validation cases in +`internal/backend`. + +### Completion Gate + +- Consumers can add only new IDs and cannot mutate or replace built-ins. +- Registration is engine-local and two engines can use different definitions + for the same consumer ID without interference. +- Existing option-category contract tests still pass unchanged except for the + documented additive backend case. +- The consumer guide and GoDoc describe the implemented extension point. +- The complete repository validation passes. + +## Stage 4 — Contract Hardening And Documentation Completion + +### Goal + +Audit the completed feature across public, profile, use-case, and transport +boundaries and make the current-state documentation self-sufficient. + +### Work + +1. Review the preceding tests as a suite. Remove redundant cases and retain a + lean precedence matrix that would catch layer-order, credential, copying, + compatibility, and error-identity regressions. +2. Add or adjust one built-in-client transport test proving that effective + backend defaults reach the configured endpoint while `backend_id` is not + serialized. Use `httptest.Server` and synthetic credentials. +3. Confirm public stable-JSON round trips for `ExecutionTarget`, + `PreparedRun`, and `RunResult`, including empty backend omission for legacy + profiles. +4. Confirm concurrent `Prepare` and `Run` calls can read the immutable registry + under the race detector. Do not add concurrency limiting or a queue. +5. Update all affected current-state owners: + - public GoDoc for the exact API and errors; + - `docs/formats.md` for profile YAML and built-ins; + - `docs/consumers/pkg-promptkit.md` for construction and use; + - `docs/integrations/openai-compatible-chat.md` for resolved endpoint, + credentials, request defaults, and the non-serialized routing identity; + - `docs/internal/overview.md`, `docs/internal/sources.md`, + `docs/internal/runner.md`, and `docs/internal/llm.md` for package + responsibilities and data flow; and + - `docs/policy/architecture.md` only to keep its implemented component + inventory and dependency description accurate. +6. Search current-state documentation for obsolete statements that every + profile requires an endpoint or that every built-in repeats OpenRouter + connection settings. Update the canonical owner and replace duplicates with + links. +7. Keep future concurrency and queue behavior only in + [the future feature catalog](future.md); do not imply that this feature + implements either capability. + +### Final Validation + +Run the complete maintainer sequence from the repository root: + +```sh +go test ./... +go test -race ./... +go vet ./... +go build ./... +go run ./examples/go-library/prepare +gofmt -l $(git ls-files '*.go') +git diff --check +``` + +The formatting command must produce no paths. Follow every changed Markdown +link and confirm its target exists. Inspect the final diff for raw credentials, +global mutable state, generated workspace files, local `replace` directives, +and unrelated changes. + +### Completion Gate + +- Every target-end-state item in [the feature roadmap](backends.md) is + implemented and protected at its owning boundary. +- The public API is limited to the fixed declarations in this plan. +- Endpoint-only profiles and injected clients remain compatible. +- Built-in profiles obtain OpenRouter endpoint and credential metadata only + through the registry. +- Current-state documentation no longer relies on either roadmap to explain + implemented behavior. +- All final validation commands pass offline. + +## Open Questions + +None. The API shape, validation, precedence, compatibility behavior, error +identities, package ownership, test ownership, and staging required for +implementation are fixed by this plan.