promptkit/docs/roadmap/implementation.md

# Extensible LLM Backend Registry Implementation Plan

## Purpose

This document is the decision-complete implementation plan for the
[extensible LLM backend registry](backends.md). It is written for a coding
agent that will implement each stage in order.

This is planning material, not a description of current behavior. During
implementation, update current-state GoDoc and documentation only as the
corresponding behavior lands. The feature roadmap owns the intended end state
and policy choices; this document owns API shape, package changes, sequencing,
tests, and completion gates.

## Implementation Rules

- Complete the stages in order. Do not begin a later stage while an earlier
  stage's completion gate is unmet.
- Preserve unrelated working-tree changes and do not broaden the feature into
  concurrency control, queues, retries, provider discovery, or
  non-OpenAI-compatible transports.
- Keep the registry immutable and scoped to one `Engine`. Do not add
  package-global registration or mutation after construction.
- Keep secrets out of backend definitions, profiles, prepared values, results,
  hashes, logs, and JSON. Backends store only an environment-variable name.
- Preserve endpoint-only profiles and custom `LLMClient` injection.
- Follow the architecture, documentation, and testing policies under
  `docs/policy/`. In particular, keep the public API in the root package,
  place implementation packages under `internal/`, update the component
  inventory when the new package lands, and assign each behavior to one
  durable test owner.
- Use deterministic, offline tests. No stage may require a real provider,
  credential, or network service.

## Fixed Design

### Public API

Add the following root-package declarations:

```go
const BackendOpenRouter = "openrouter"

type Backend struct {
	ID          string
	Endpoint    string
	APIKeyEnv   string
	ExtraParams map[string]any
}

func WithBackend(backend Backend) Option
```

`Backend` and `WithBackend` have these contracts:

- `Backend` configures one OpenAI-compatible backend and has no stable JSON
  representation.
- `ID` is trimmed, non-empty, case-sensitive, and is the stable registry key.
- `Endpoint` is trimmed and must be an absolute `http` or `https` URL with a
  host. Paths are allowed; user information, query strings, and fragments are
  rejected because this value is a base endpoint rather than a complete
  request URL.
- `APIKeyEnv` is optional. When present, it is trimmed and must match the
  portable environment-variable form `[A-Za-z_][A-Za-z0-9_]*`.
- `ExtraParams` follows the existing `Profile.ExtraParams` JSON-value rules:
  non-empty string keys, finite numbers, JSON-compatible scalar and container
  values, no cycles, and no collisions with `model`, `session_id`, `messages`,
  `temperature`, `max_tokens`, `top_p`, `service_tier`, `reasoning_effort`, or
  `response_format`. It is deeply copied during `NewEngine`.
- An empty `ExtraParams` map means that the backend supplies no request
  defaults.
- Each `WithBackend` call adds one registration. Calls with different IDs
  accumulate in option order; a repeated ID is an error rather than a
  replacement. This is an explicit additive exception to the last-option-wins
  categories documented on `Option`.
- `BackendOpenRouter` is reserved. Consumers cannot replace it.
- There is no public enumerate, lookup, remove, replace, or post-construction
  registration API in this scope.

Extend the existing public values as follows:

| Value | Addition | Contract |
| --- | --- | --- |
| `Profile` | `BackendID string` | Optional backend selection. `Endpoint` is required only when `BackendID` is blank. |
| `OpenAICompatibleProfileConfig` | `BackendID string` | Copied to `Profile.BackendID`. |
| `ExecutionTarget` | `BackendID string \`json:"backend_id,omitempty"\`` | Effective routing identity supplied to injected clients; empty for endpoint-only profiles. |
| `PreparedRun` | `SelectedBackendID string \`json:"selected_backend_id,omitempty"\`` | Equals the effective target backend ID. |
| `RunResult` | `SelectedBackendID string \`json:"selected_backend_id,omitempty"\`` | Carries the prepared backend identity through execution. |

Do not add a backend field to `RunRequest` or `ExecutionTargetOverride`.
Backend selection remains a profile concern. Endpoint overrides do not change
the selected backend identity.

Trim `Profile.BackendID` and YAML `backend` values at their respective
conversion/load boundaries. Treat an all-whitespace value as absent and expose
only the trimmed value in effective targets and metadata.

These exported struct fields are intentionally additive. Adding fields can
break downstream unkeyed composite literals even though keyed literals and
ordinary field access remain source-compatible. Accept that narrow risk; use
and document keyed literals, and do not add parallel wrapper types or a second
configuration path solely to preserve unkeyed literals.

### Profile Format

Add the optional strict-YAML field:

```yaml
backend: openrouter
```

The profile connection rule becomes:

- `model` remains required;
- at least one of `backend` or `endpoint` is required;
- when both are present, the profile endpoint overrides the backend endpoint;
  and
- a profile with only an endpoint continues through the legacy path and has an
  empty effective backend ID.

Do not infer a backend from a model name or endpoint. Do not validate whether a
backend ID exists while decoding a profile; registry membership is
engine-scoped and is checked when the selected profile is prepared.

### Registry And Internal Boundaries

Add `internal/backend` as the cohesive owner of:

- the immutable backend registry;
- backend-definition validation and defensive copying;
- the built-in OpenRouter definition; and
- the internal not-found error used when a selected ID is absent.

Add an internal domain backend value containing `ID`, `Endpoint`, `APIKeyEnv`,
and `ExtraParams`. The registry constructor accepts consumer additions, installs
built-ins first, rejects all collisions, and returns a fully constructed
read-only registry. Lookup returns a defensive value so callers cannot mutate
registry-owned maps.

Use these internal declarations:

```go
var ErrBackendNotFound = errors.New("backend not found")

func NewRegistry(additions []domain.Backend) (*Registry, error)
func (r *Registry) GetBackend(id string) (domain.Backend, error)
```

`internal/usecase` owns the narrow resolver interface it consumes:

```go
type BackendResolver interface {
	GetBackend(string) (domain.Backend, error)
}
```

The concrete registry implements this interface. Supply it explicitly to
`Runner` through both runner constructors and from the root engine assembly.
A nil resolver must never panic; if an internally constructed runner selects a
backend without a resolver, preparation fails as a profile-load failure.

The built-in definition is exactly:

| Field | Value |
| --- | --- |
| ID | `openrouter` |
| Endpoint | `https://openrouter.ai/api/v1` |
| API key environment variable | `OPENROUTER_API_KEY` |
| Extra parameters | none |

Define `OpenRouterID` as a constant in `internal/backend` and define the public
`BackendOpenRouter` constant from it so there is one canonical literal. This
root-to-internal implementation import does not expose an internal type in a
public signature.

### Resolution

Resolve an execution target in this order:

1. framework defaults;
2. the selected backend, if any;
3. the selected profile; and
4. the request's `ExecutionTargetOverride`.

The layers behave as follows:

| Setting | Backend layer | Profile layer | Request layer |
| --- | --- | --- | --- |
| Backend ID | Supplies selected ID | Selects the backend | Cannot change it |
| Endpoint | Supplies default | Non-empty value replaces backend | Non-empty value replaces profile/backend |
| Model and generation fields | Not supplied | Existing profile behavior | Existing override behavior |
| API-key environment name | Supplies default | Non-empty `api_key_env` replaces it | Non-empty `APIKeyEnv` replaces it |
| Extra parameters | Non-empty map replaces prior map | Non-empty map replaces complete backend map | Non-empty map replaces complete profile/backend map |

Never deep-merge `ExtraParams` maps. A non-empty higher-precedence map replaces
the complete lower-precedence map. Preserve the existing distinction between a
nil or empty override map and a supplied non-empty replacement.

Credential handling remains:

1. a non-blank `RunRequest.APIKey` wins and suppresses environment lookup;
2. a non-blank request `APIKeyEnv` wins over profile and backend metadata;
3. file-profile `api_key_env` wins over the backend default;
4. backend `APIKeyEnv` is used when no higher layer supplies a credential
   source; and
5. a profile with `APIKeyRequired: true` clears an inherited backend
   environment name and requires a direct key unless the request explicitly
   supplies its own `APIKeyEnv`.

Only the environment-variable name appears in effective targets. Preserve the
current just-in-time environment lookup and missing-variable behavior.

### Errors

Do not add a public backend-specific error sentinel.

| Failure | Required public identity |
| --- | --- |
| Invalid consumer backend definition | `ErrInvalidConfig` from `NewEngine` |
| Consumer ID duplicates another consumer ID | `ErrInvalidConfig` |
| Consumer ID collides with `BackendOpenRouter` | `ErrInvalidConfig` |
| Selected profile names an unknown backend | `ErrProfileLoad`, not `ErrProfileNotFound` or `ErrInvalidRequest` |
| File profile lacks both backend and endpoint | `ErrProfileLoad` |
| In-memory profile lacks both backend and endpoint | `ErrInvalidConfig` |
| Effective credential environment variable is unset | Existing `ErrAPIKeyEnvMissing` and `ErrInvalidRequest` identities |
| Other invalid per-run effective settings | Existing `ErrInvalidRequest` |

Error text must include the offending backend ID or field where useful, but
exact prose is not a compatibility contract. Preserve wrapped internal causes
where the existing error boundary permits it.

### Metadata And Transport

- Set `ExecutionTarget.BackendID`, `PreparedRun.SelectedBackendID`, and
  `RunResult.SelectedBackendID` from the normalized selected profile backend.
- Endpoint-only profiles leave all three values empty.
- Pass `BackendID` through `GenerateRequest.Target` and repair targets so an
  injected client can route or observe it.
- The built-in OpenAI-compatible client must not serialize `backend_id` as a
  provider request field. It continues to use the effective endpoint,
  credential, typed generation fields, and extra parameters.
- Backend selection does not change rendered-prompt hashes, prompt hashes,
  session IDs, validation, artifacts, timing, or synchronous `Prepare` and
  `Run` behavior.

## Stage 1 — Registry Foundation

**Status:** Complete.

### Goal

Introduce the internal domain and immutable registry without changing the
public configuration or profile format.

### Work

1. Add the internal backend definition to `internal/domain`.
2. Create `internal/backend` with:
   - a registry constructor that installs OpenRouter and accepts additional
     internal definitions;
   - validation for IDs, endpoints, environment-variable names, JSON values,
     and reserved request fields;
   - deterministic built-in and consumer collision handling;
   - immutable lookup with defensive map copying; and
   - an internal recognizable not-found error.
3. Reuse or extract one validation rule for OpenAI-compatible reserved request
   fields so registry validation and `internal/llm` payload construction cannot
   drift. Do not maintain duplicate reserved-key lists.
4. Keep JSON-value copying type-preserving. Do not use a marshal/unmarshal
   round trip that silently changes integer or container types.
5. Update `docs/policy/architecture.md` and `docs/internal/overview.md` only
   after the package exists, describing `internal/backend` as an implemented
   immutable configuration registry rather than a runtime service manager.

### Tests

`internal/backend` owns focused package tests for:

- the exact OpenRouter built-in values;
- successful unique consumer additions;
- built-in and consumer duplicate rejection;
- blank and whitespace-normalized IDs;
- invalid or non-HTTP(S) endpoints;
- invalid environment-variable names;
- invalid, cyclic, non-finite, empty-key, and reserved-key extra parameters;
- not-found lookup; and
- mutation isolation of input maps and returned values.

Use one representative table per validation family rather than one test per
branch. A race-specific registry test is unnecessary if the registry is
immutable and the final repository race suite exercises concurrent reads.

### Completion Gate

- No public declaration or profile behavior has changed.
- The registry has no mutating method after construction.
- The built-in definition contains no credential value.
- Focused backend tests and the complete repository validation pass.

## Stage 2 — Profile Selection And Effective Resolution

**Status:** Complete.

### Goal

Make the built-in OpenRouter backend selectable by file and in-memory profiles,
migrate the built-in profiles, and expose effective backend identity.

### Work

1. Add `BackendID` to internal execution profiles and targets and add the YAML
   key `backend`.
2. Change file and public in-memory profile validation to require `model` plus
   at least one of `backend` or `endpoint`. Preserve all existing numeric and
   extra-parameter validation.
3. Add `BackendID` to `Profile` and
   `OpenAICompatibleProfileConfig`, and add `BackendOpenRouter`.
4. Add backend identity to the public and internal execution target, prepared
   run, and run result exactly as specified under **Public API**.
5. Add the `BackendResolver` dependency immediately after the profile
   repository in both runner constructors and update all internal call sites
   and test helpers explicitly.
6. Resolve the selected backend after profile loading and before effective
   target resolution. Map lookup failure to `ErrProfileLoad`.
7. Implement the exact endpoint, credential, and whole-map precedence rules
   under **Resolution**. Keep backend ID independent from endpoint overrides.
8. Construct the built-in-only registry in `NewEngine` and inject it into the
   runner.
9. Replace `endpoint` and `api_key_env` in every
   `internal/profile/builtin/assets/*.yaml` profile with
   `backend: openrouter`. Retain each profile's model and generation settings.
10. Update conversions and copying so backend IDs survive every public/internal
    boundary and registry-owned maps remain isolated.
11. Update the GoDoc for all changed public declarations.
12. Update `docs/formats.md`, `docs/internal/sources.md`, and
    `docs/internal/runner.md` with the implemented profile field, conditional
    endpoint rule, built-in selection, and effective resolution behavior.

### Tests

Assign test ownership as follows:

- `internal/profile` tests own strict YAML decoding and the conditional
  backend-or-endpoint validation matrix.
- `internal/profile/builtin` tests own the invariant that every built-in
  profile selects `openrouter` and no longer repeats its endpoint or
  credential environment name.
- `internal/usecase` tests own backend/profile/request precedence, unknown-ID
  error wrapping, endpoint override identity, credential precedence including
  `APIKeyRequired`, and whole-map replacement.
- Root external-package contract tests own the new fields, JSON names and
  omission behavior, public copying, and public error identities.
- One assembled engine test must prove a built-in profile prepares with the
  expected backend ID, endpoint, and environment-variable name without making
  an HTTP call.

Do not duplicate every profile-parser validation case at the engine boundary.

### Completion Gate

- All built-in and endpoint-only profiles prepare successfully.
- A selected unknown backend matches public `ErrProfileLoad`.
- Backend identity reaches prepared values, results, generation targets, and
  repair targets.
- No backend ID or secret is added to the outbound provider payload.
- Current-state format and internal documentation match the implemented
  built-in behavior.
- The complete repository validation passes.

## Stage 3 — Consumer Registration

### Goal

Expose the engine-scoped extension point for additional unique
OpenAI-compatible backend IDs.

### Work

1. Add `Backend` and `WithBackend` exactly as specified under **Public API**,
   preferably in a cohesive root `backends.go` file.
2. Convert, validate, normalize, and deeply copy public backend values during
   `NewEngine`. Store pending additions in `engineOptions`; do not mutate a
   registry from option application.
3. After all options have applied, construct one immutable registry from the
   built-in and pending consumer definitions. Map every construction failure
   to `ErrInvalidConfig`.
4. Make unique `WithBackend` calls additive. Reject duplicate IDs even when
   they arrive through separate calls, and document this exception on
   `Option` and `WithBackend`.
5. Make registered backends available equally to directory, `fs.FS`, single
   file, in-memory, prompt-default, and explicit request profile selection.
6. Preserve the last-valid-option-wins behavior of every existing option
   category.
7. Update `docs/consumers/pkg-promptkit.md` with one minimal custom backend and
   profile example. Keep exact field-by-field API detail in GoDoc and link to
   it rather than duplicating it.

### Tests

Root public contract tests own:

- one custom backend used by an in-memory profile;
- one custom backend used by a file-backed profile;
- multiple unique `WithBackend` calls accumulating;
- duplicate consumer and built-in ID failures matching `ErrInvalidConfig`;
- invalid public values and nested-map mutation isolation;
- endpoint-only compatibility with no `WithBackend` option;
- profile endpoint and request endpoint overrides retaining the custom backend
  ID; and
- an injected `LLMClient` observing the effective custom backend ID and
  settings.

Use an injected client or `httptest.Server`; never contact the configured
external endpoint. Keep lower-level registry validation cases in
`internal/backend`.

### Completion Gate

- Consumers can add only new IDs and cannot mutate or replace built-ins.
- Registration is engine-local and two engines can use different definitions
  for the same consumer ID without interference.
- Existing option-category contract tests still pass unchanged except for the
  documented additive backend case.
- The consumer guide and GoDoc describe the implemented extension point.
- The complete repository validation passes.

## Stage 4 — Contract Hardening And Documentation Completion

### Goal

Audit the completed feature across public, profile, use-case, and transport
boundaries and make the current-state documentation self-sufficient.

### Work

1. Review the preceding tests as a suite. Remove redundant cases and retain a
   lean precedence matrix that would catch layer-order, credential, copying,
   compatibility, and error-identity regressions.
2. Add or adjust one built-in-client transport test proving that effective
   backend defaults reach the configured endpoint while `backend_id` is not
   serialized. Use `httptest.Server` and synthetic credentials.
3. Confirm public stable-JSON round trips for `ExecutionTarget`,
   `PreparedRun`, and `RunResult`, including empty backend omission for legacy
   profiles.
4. Confirm concurrent `Prepare` and `Run` calls can read the immutable registry
   under the race detector. Do not add concurrency limiting or a queue.
5. Update all affected current-state owners:
   - public GoDoc for the exact API and errors;
   - `docs/formats.md` for profile YAML and built-ins;
   - `docs/consumers/pkg-promptkit.md` for construction and use;
   - `docs/integrations/openai-compatible-chat.md` for resolved endpoint,
     credentials, request defaults, and the non-serialized routing identity;
   - `docs/internal/overview.md`, `docs/internal/sources.md`,
     `docs/internal/runner.md`, and `docs/internal/llm.md` for package
     responsibilities and data flow; and
   - `docs/policy/architecture.md` only to keep its implemented component
     inventory and dependency description accurate.
6. Search current-state documentation for obsolete statements that every
   profile requires an endpoint or that every built-in repeats OpenRouter
   connection settings. Update the canonical owner and replace duplicates with
   links.
7. Keep future concurrency and queue behavior only in
   [the future feature catalog](future.md); do not imply that this feature
   implements either capability.

### Final Validation

Run the complete maintainer sequence from the repository root:

```sh
go test ./...
go test -race ./...
go vet ./...
go build ./...
go run ./examples/go-library/prepare
gofmt -l $(git ls-files '*.go')
git diff --check
```

The formatting command must produce no paths. Follow every changed Markdown
link and confirm its target exists. Inspect the final diff for raw credentials,
global mutable state, generated workspace files, local `replace` directives,
and unrelated changes.

### Completion Gate

- Every target-end-state item in [the feature roadmap](backends.md) is
  implemented and protected at its owning boundary.
- The public API is limited to the fixed declarations in this plan.
- Endpoint-only profiles and injected clients remain compatible.
- Built-in profiles obtain OpenRouter endpoint and credential metadata only
  through the registry.
- Current-state documentation no longer relies on either roadmap to explain
  implemented behavior.
- All final validation commands pass offline.

## Open Questions

None. The API shape, validation, precedence, compatibility behavior, error
identities, package ownership, test ownership, and staging required for
implementation are fixed by this plan.