promptkit/docs/roadmap/implementation.md

Extensible LLM Backend Registry Implementation Plan

Purpose

This document is the decision-complete implementation plan for the extensible LLM backend registry. It is written for a coding agent that will implement each stage in order.

This is planning material, not a description of current behavior. During implementation, update current-state GoDoc and documentation only as the corresponding behavior lands. The feature roadmap owns the intended end state and policy choices; this document owns API shape, package changes, sequencing, tests, and completion gates.

Implementation Rules

• Complete the stages in order. Do not begin a later stage while an earlier stage's completion gate is unmet.
• Preserve unrelated working-tree changes and do not broaden the feature into concurrency control, queues, retries, provider discovery, or non-OpenAI-compatible transports.
• Keep the registry immutable and scoped to one Engine. Do not add package-global registration or mutation after construction.
• Keep secrets out of backend definitions, profiles, prepared values, results, hashes, logs, and JSON. Backends store only an environment-variable name.
• Preserve endpoint-only profiles and custom LLMClient injection.
• Follow the architecture, documentation, and testing policies under docs/policy/. In particular, keep the public API in the root package, place implementation packages under internal/, update the component inventory when the new package lands, and assign each behavior to one durable test owner.
• Use deterministic, offline tests. No stage may require a real provider, credential, or network service.

Fixed Design

Public API

Add the following root-package declarations:

const BackendOpenRouter = "openrouter"

type Backend struct {
	ID          string
	Endpoint    string
	APIKeyEnv   string
	ExtraParams map[string]any
}

func WithBackend(backend Backend) Option

Backend and WithBackend have these contracts:

• Backend configures one OpenAI-compatible backend and has no stable JSON representation.
• ID is trimmed, non-empty, case-sensitive, and is the stable registry key.
• Endpoint is trimmed and must be an absolute http or https URL with a host. Paths are allowed; user information, query strings, and fragments are rejected because this value is a base endpoint rather than a complete request URL.
• APIKeyEnv is optional. When present, it is trimmed and must match the portable environment-variable form [A-Za-z_][A-Za-z0-9_]*.
• ExtraParams follows the existing Profile.ExtraParams JSON-value rules: non-empty string keys, finite numbers, JSON-compatible scalar and container values, no cycles, and no collisions with model, session_id, messages, temperature, max_tokens, top_p, service_tier, reasoning_effort, or response_format. It is deeply copied during NewEngine.
• An empty ExtraParams map means that the backend supplies no request defaults.
• Each WithBackend call adds one registration. Calls with different IDs accumulate in option order; a repeated ID is an error rather than a replacement. This is an explicit additive exception to the last-option-wins categories documented on Option.
• BackendOpenRouter is reserved. Consumers cannot replace it.
• There is no public enumerate, lookup, remove, replace, or post-construction registration API in this scope.

Extend the existing public values as follows:

Value	Addition	Contract
Profile	BackendID string	Optional backend selection. Endpoint is required only when BackendID is blank.
OpenAICompatibleProfileConfig	BackendID string	Copied to Profile.BackendID.
ExecutionTarget	BackendID string \json:"backend_id,omitempty"``	Effective routing identity supplied to injected clients; empty for endpoint-only profiles.
PreparedRun	SelectedBackendID string \json:"selected_backend_id,omitempty"``	Equals the effective target backend ID.
RunResult	SelectedBackendID string \json:"selected_backend_id,omitempty"``	Carries the prepared backend identity through execution.

Do not add a backend field to RunRequest or ExecutionTargetOverride. Backend selection remains a profile concern. Endpoint overrides do not change the selected backend identity.

Trim Profile.BackendID and YAML backend values at their respective conversion/load boundaries. Treat an all-whitespace value as absent and expose only the trimmed value in effective targets and metadata.

These exported struct fields are intentionally additive. Adding fields can break downstream unkeyed composite literals even though keyed literals and ordinary field access remain source-compatible. Accept that narrow risk; use and document keyed literals, and do not add parallel wrapper types or a second configuration path solely to preserve unkeyed literals.

Profile Format

Add the optional strict-YAML field:

backend: openrouter

The profile connection rule becomes:

• model remains required;
• at least one of backend or endpoint is required;
• when both are present, the profile endpoint overrides the backend endpoint; and
• a profile with only an endpoint continues through the legacy path and has an empty effective backend ID.

Do not infer a backend from a model name or endpoint. Do not validate whether a backend ID exists while decoding a profile; registry membership is engine-scoped and is checked when the selected profile is prepared.

Registry And Internal Boundaries

Add internal/backend as the cohesive owner of:

• the immutable backend registry;
• backend-definition validation and defensive copying;
• the built-in OpenRouter definition; and
• the internal not-found error used when a selected ID is absent.

Add an internal domain backend value containing ID, Endpoint, APIKeyEnv, and ExtraParams. The registry constructor accepts consumer additions, installs built-ins first, rejects all collisions, and returns a fully constructed read-only registry. Lookup returns a defensive value so callers cannot mutate registry-owned maps.

Use these internal declarations:

var ErrBackendNotFound = errors.New("backend not found")

func NewRegistry(additions []domain.Backend) (*Registry, error)
func (r *Registry) GetBackend(id string) (domain.Backend, error)

internal/usecase owns the narrow resolver interface it consumes:

type BackendResolver interface {
	GetBackend(string) (domain.Backend, error)
}

The concrete registry implements this interface. Supply it explicitly to Runner through both runner constructors and from the root engine assembly. A nil resolver must never panic; if an internally constructed runner selects a backend without a resolver, preparation fails as a profile-load failure.

The built-in definition is exactly:

Field	Value
ID	openrouter
Endpoint	https://openrouter.ai/api/v1
API key environment variable	OPENROUTER_API_KEY
Extra parameters	none

Define OpenRouterID as a constant in internal/backend and define the public BackendOpenRouter constant from it so there is one canonical literal. This root-to-internal implementation import does not expose an internal type in a public signature.

Resolution

Resolve an execution target in this order:

1. framework defaults;
2. the selected backend, if any;
3. the selected profile; and
4. the request's ExecutionTargetOverride.

The layers behave as follows:

Setting	Backend layer	Profile layer	Request layer
Backend ID	Supplies selected ID	Selects the backend	Cannot change it
Endpoint	Supplies default	Non-empty value replaces backend	Non-empty value replaces profile/backend
Model and generation fields	Not supplied	Existing profile behavior	Existing override behavior
API-key environment name	Supplies default	Non-empty api_key_env replaces it	Non-empty APIKeyEnv replaces it
Extra parameters	Non-empty map replaces prior map	Non-empty map replaces complete backend map	Non-empty map replaces complete profile/backend map

Never deep-merge ExtraParams maps. A non-empty higher-precedence map replaces the complete lower-precedence map. Preserve the existing distinction between a nil or empty override map and a supplied non-empty replacement.

Credential handling remains:

1. a non-blank RunRequest.APIKey wins and suppresses environment lookup;
2. a non-blank request APIKeyEnv wins over profile and backend metadata;
3. file-profile api_key_env wins over the backend default;
4. backend APIKeyEnv is used when no higher layer supplies a credential source; and
5. a profile with APIKeyRequired: true clears an inherited backend environment name and requires a direct key unless the request explicitly supplies its own APIKeyEnv.

Only the environment-variable name appears in effective targets. Preserve the current just-in-time environment lookup and missing-variable behavior.

Errors

Do not add a public backend-specific error sentinel.

Failure	Required public identity
Invalid consumer backend definition	ErrInvalidConfig from NewEngine
Consumer ID duplicates another consumer ID	ErrInvalidConfig
Consumer ID collides with BackendOpenRouter	ErrInvalidConfig
Selected profile names an unknown backend	ErrProfileLoad, not ErrProfileNotFound or ErrInvalidRequest
File profile lacks both backend and endpoint	ErrProfileLoad
In-memory profile lacks both backend and endpoint	ErrInvalidConfig
Effective credential environment variable is unset	Existing ErrAPIKeyEnvMissing and ErrInvalidRequest identities
Other invalid per-run effective settings	Existing ErrInvalidRequest

Error text must include the offending backend ID or field where useful, but exact prose is not a compatibility contract. Preserve wrapped internal causes where the existing error boundary permits it.

Metadata And Transport

• Set ExecutionTarget.BackendID, PreparedRun.SelectedBackendID, and RunResult.SelectedBackendID from the normalized selected profile backend.
• Endpoint-only profiles leave all three values empty.
• Pass BackendID through GenerateRequest.Target and repair targets so an injected client can route or observe it.
• The built-in OpenAI-compatible client must not serialize backend_id as a provider request field. It continues to use the effective endpoint, credential, typed generation fields, and extra parameters.
• Backend selection does not change rendered-prompt hashes, prompt hashes, session IDs, validation, artifacts, timing, or synchronous Prepare and Run behavior.

Stage 1 — Registry Foundation

Status: Complete.

Goal

Introduce the internal domain and immutable registry without changing the public configuration or profile format.

Work

1. Add the internal backend definition to internal/domain.
2. Create internal/backend with:
   • a registry constructor that installs OpenRouter and accepts additional internal definitions;
   • validation for IDs, endpoints, environment-variable names, JSON values, and reserved request fields;
   • deterministic built-in and consumer collision handling;
   • immutable lookup with defensive map copying; and
   • an internal recognizable not-found error.
3. Reuse or extract one validation rule for OpenAI-compatible reserved request fields so registry validation and internal/llm payload construction cannot drift. Do not maintain duplicate reserved-key lists.
4. Keep JSON-value copying type-preserving. Do not use a marshal/unmarshal round trip that silently changes integer or container types.
5. Update docs/policy/architecture.md and docs/internal/overview.md only after the package exists, describing internal/backend as an implemented immutable configuration registry rather than a runtime service manager.

Tests

internal/backend owns focused package tests for:

• the exact OpenRouter built-in values;
• successful unique consumer additions;
• built-in and consumer duplicate rejection;
• blank and whitespace-normalized IDs;
• invalid or non-HTTP(S) endpoints;
• invalid environment-variable names;
• invalid, cyclic, non-finite, empty-key, and reserved-key extra parameters;
• not-found lookup; and
• mutation isolation of input maps and returned values.

Use one representative table per validation family rather than one test per branch. A race-specific registry test is unnecessary if the registry is immutable and the final repository race suite exercises concurrent reads.

Completion Gate

• No public declaration or profile behavior has changed.
• The registry has no mutating method after construction.
• The built-in definition contains no credential value.
• Focused backend tests and the complete repository validation pass.

Stage 2 — Profile Selection And Effective Resolution

Status: Complete.

Goal

Make the built-in OpenRouter backend selectable by file and in-memory profiles, migrate the built-in profiles, and expose effective backend identity.

Work

1. Add BackendID to internal execution profiles and targets and add the YAML key backend.
2. Change file and public in-memory profile validation to require model plus at least one of backend or endpoint. Preserve all existing numeric and extra-parameter validation.
3. Add BackendID to Profile and OpenAICompatibleProfileConfig, and add BackendOpenRouter.
4. Add backend identity to the public and internal execution target, prepared run, and run result exactly as specified under Public API.
5. Add the BackendResolver dependency immediately after the profile repository in both runner constructors and update all internal call sites and test helpers explicitly.
6. Resolve the selected backend after profile loading and before effective target resolution. Map lookup failure to ErrProfileLoad.
7. Implement the exact endpoint, credential, and whole-map precedence rules under Resolution. Keep backend ID independent from endpoint overrides.
8. Construct the built-in-only registry in NewEngine and inject it into the runner.
9. Replace endpoint and api_key_env in every internal/profile/builtin/assets/*.yaml profile with backend: openrouter. Retain each profile's model and generation settings.
10. Update conversions and copying so backend IDs survive every public/internal boundary and registry-owned maps remain isolated.
11. Update the GoDoc for all changed public declarations.
12. Update docs/formats.md, docs/internal/sources.md, and docs/internal/runner.md with the implemented profile field, conditional endpoint rule, built-in selection, and effective resolution behavior.

Tests

Assign test ownership as follows:

• internal/profile tests own strict YAML decoding and the conditional backend-or-endpoint validation matrix.
• internal/profile/builtin tests own the invariant that every built-in profile selects openrouter and no longer repeats its endpoint or credential environment name.
• internal/usecase tests own backend/profile/request precedence, unknown-ID error wrapping, endpoint override identity, credential precedence including APIKeyRequired, and whole-map replacement.
• Root external-package contract tests own the new fields, JSON names and omission behavior, public copying, and public error identities.
• One assembled engine test must prove a built-in profile prepares with the expected backend ID, endpoint, and environment-variable name without making an HTTP call.

Do not duplicate every profile-parser validation case at the engine boundary.

Completion Gate

• All built-in and endpoint-only profiles prepare successfully.
• A selected unknown backend matches public ErrProfileLoad.
• Backend identity reaches prepared values, results, generation targets, and repair targets.
• No backend ID or secret is added to the outbound provider payload.
• Current-state format and internal documentation match the implemented built-in behavior.
• The complete repository validation passes.

Stage 3 — Consumer Registration

Status: Complete.

Goal

Expose the engine-scoped extension point for additional unique OpenAI-compatible backend IDs.

Work

1. Add Backend and WithBackend exactly as specified under Public API, preferably in a cohesive root backends.go file.
2. Convert, validate, normalize, and deeply copy public backend values during NewEngine. Store pending additions in engineOptions; do not mutate a registry from option application.
3. After all options have applied, construct one immutable registry from the built-in and pending consumer definitions. Map every construction failure to ErrInvalidConfig.
4. Make unique WithBackend calls additive. Reject duplicate IDs even when they arrive through separate calls, and document this exception on Option and WithBackend.
5. Make registered backends available equally to directory, fs.FS, single file, in-memory, prompt-default, and explicit request profile selection.
6. Preserve the last-valid-option-wins behavior of every existing option category.
7. Update docs/consumers/pkg-promptkit.md with one minimal custom backend and profile example. Keep exact field-by-field API detail in GoDoc and link to it rather than duplicating it.

Tests

Root public contract tests own:

• one custom backend used by an in-memory profile;
• one custom backend used by a file-backed profile;
• multiple unique WithBackend calls accumulating;
• duplicate consumer and built-in ID failures matching ErrInvalidConfig;
• invalid public values and nested-map mutation isolation;
• endpoint-only compatibility with no WithBackend option;
• profile endpoint and request endpoint overrides retaining the custom backend ID; and
• an injected LLMClient observing the effective custom backend ID and settings.

Use an injected client or httptest.Server; never contact the configured external endpoint. Keep lower-level registry validation cases in internal/backend.

Completion Gate

• Consumers can add only new IDs and cannot mutate or replace built-ins.
• Registration is engine-local and two engines can use different definitions for the same consumer ID without interference.
• Existing option-category contract tests still pass unchanged except for the documented additive backend case.
• The consumer guide and GoDoc describe the implemented extension point.
• The complete repository validation passes.

Stage 4 — Contract Hardening And Documentation Completion

Status: Complete.

Goal

Audit the completed feature across public, profile, use-case, and transport boundaries and make the current-state documentation self-sufficient.

Work

1. Review the preceding tests as a suite. Remove redundant cases and retain a lean precedence matrix that would catch layer-order, credential, copying, compatibility, and error-identity regressions.
2. Add or adjust one built-in-client transport test proving that effective backend defaults reach the configured endpoint while backend_id is not serialized. Use httptest.Server and synthetic credentials.
3. Confirm public stable-JSON round trips for ExecutionTarget, PreparedRun, and RunResult, including empty backend omission for legacy profiles.
4. Confirm concurrent Prepare and Run calls can read the immutable registry under the race detector. Do not add concurrency limiting or a queue.
5. Update all affected current-state owners:
   • public GoDoc for the exact API and errors;
   • docs/formats.md for profile YAML and built-ins;
   • docs/consumers/pkg-promptkit.md for construction and use;
   • docs/integrations/openai-compatible-chat.md for resolved endpoint, credentials, request defaults, and the non-serialized routing identity;
   • docs/internal/overview.md, docs/internal/sources.md, docs/internal/runner.md, and docs/internal/llm.md for package responsibilities and data flow; and
   • docs/policy/architecture.md only to keep its implemented component inventory and dependency description accurate.
6. Search current-state documentation for obsolete statements that every profile requires an endpoint or that every built-in repeats OpenRouter connection settings. Update the canonical owner and replace duplicates with links.
7. Keep future concurrency and queue behavior only in the future feature catalog; do not imply that this feature implements either capability.

Final Validation

Run the complete maintainer sequence from the repository root:

go test ./...
go test -race ./...
go vet ./...
go build ./...
go run ./examples/go-library/prepare
gofmt -l $(git ls-files '*.go')
git diff --check

The formatting command must produce no paths. Follow every changed Markdown link and confirm its target exists. Inspect the final diff for raw credentials, global mutable state, generated workspace files, local replace directives, and unrelated changes.

Completion Gate

• Every target-end-state item in the feature roadmap is implemented and protected at its owning boundary.
• The public API is limited to the fixed declarations in this plan.
• Endpoint-only profiles and injected clients remain compatible.
• Built-in profiles obtain OpenRouter endpoint and credential metadata only through the registry.
• Current-state documentation no longer relies on either roadmap to explain implemented behavior.
• All final validation commands pass offline.

Open Questions

None. The API shape, validation, precedence, compatibility behavior, error identities, package ownership, test ownership, and staging required for implementation are fixed by this plan.