promptkit/docs/roadmap/future.md

# Future Feature Ideas

## Purpose

This document catalogs reasonably specific ideas that may be useful in future
Promptkit development. It is an idea pool, not a commitment, schedule, or
description of current behavior.

Ideas belong here while they are worth retaining but have not been selected
for active development. Keep each entry at the level of intended capability,
consumer value, and important scope boundaries. Defer API design,
implementation details, sequencing, and acceptance criteria until an idea is
selected.

## Using This Catalog

- Add an idea when its purpose and likely value can be stated clearly.
- Keep entries independent enough that maintainers can evaluate and select
  them individually.
- Note significant dependencies or boundary concerns, but do not turn entries
  into implementation plans.
- Treat inclusion as an invitation to evaluate, not as approval or priority.
- When an idea is selected, move its active planning to a focused roadmap or,
  when it requires a durable architectural decision, an ADR. Update
  current-state documentation only when implementation lands.
- Remove ideas that are no longer relevant. Retain a rejected idea only when
  its rationale is likely to prevent repeated reconsideration.

Future capabilities must continue to respect the
[architecture policy](../policy/architecture.md), particularly Promptkit's
role as an application-neutral library and its boundary with downstream
consumers.

## Ideas

### Extensible LLM backend registry

Introduce a registry that separates backend-specific connection,
authentication, and limited request defaults from model execution profiles.
Initial support would cover OpenAI-compatible backends and include a small
built-in catalog, potentially starting with OpenRouter. A profile could select
a backend while optionally overriding its default endpoint, and each backend
could name an optional environment variable for its API key without storing
the credential itself. Downstream consumers could register additional,
uniquely named backends, such as OpenAI or unauthenticated local-network
services, but could not replace built-in IDs. Model selection and generation
settings would remain profile concerns, and custom model clients would remain
available for behavior outside the registry's supported protocol.

### Backend-specific concurrency management

Extend the proposed LLM backend registry with optional per-backend concurrency
limits and bounded, buffered admission queues. Promptkit could then route
simultaneous generation requests according to backend capacity while
containing accidental runaway submission. Downstream consumers would continue
invoking synchronous `Run` calls, including concurrently from multiple
goroutines, and each admitted call would wait for and return its ordinary
result.

- Scope limits to an engine instance rather than hidden process-global state.
- Give different backend IDs independent capacity pools. A profile endpoint
  override would remain part of its selected backend's pool.
- Configure active concurrency and waiting capacity separately. Concurrency
  protects the backend, while queue capacity protects the process from
  admitting an unbounded backlog.
- Give queue capacity a generous, configurable bounded default intended as a
  safety ceiling for bugs or unintended loops rather than a routine
  application constraint. Select an exact default during implementation
  planning and measurement.
- Reject a call with a recognizable capacity error when its backend queue is
  full rather than allowing it to wait outside the bounded queue.
- Admit requests before expensive preparation and artifact copying where
  practical so queued work remains lightweight.
- Apply a limit to each actual generation request, including repair attempts,
  without unnecessarily serializing prompt preparation.
- Make queued and active waits respect caller cancellation and deadlines.
- Treat concurrency as backend policy rather than a profile-level model
  setting.
- Keep the queue ephemeral and in-process, with no survival guarantee across
  engine or process shutdown.
- Preserve the existing execution model as far as practical. Durable jobs,
  polling, priorities, application worker lifecycle, retries, and
  cross-process coordination would be separate future capabilities.

### Explicit per-run session and reasoning controls

Allow consumers to associate a session ID with each run and to inherit,
replace, or explicitly disable the reasoning effort configured by its selected
profile. Prompt definitions can currently derive a session ID from a template,
and a non-empty runtime `ReasoningEffort` can replace the profile value, but
there is no direct request-level session ID and an empty reasoning value means
that no override was supplied. These controls would let consumers reuse one
prompt and model profile across sessions and reasoning levels without
maintaining duplicate definitions.

- Preserve a prompt's session ID template and a profile's reasoning effort as
  reusable defaults.
- Let a directly supplied per-run session ID take precedence over a rendered
  prompt default while retaining the existing validation limit and outbound
  representation.
- Distinguish an omitted runtime reasoning choice from an explicit request to
  disable reasoning.
- Ensure disabling reasoning omits the corresponding provider request setting
  rather than relying on a provider-specific magic value.
- Keep the effective session ID and reasoning choice visible in prepared and
  run metadata and available to injected model clients without introducing
  additional prompt or profile selection mechanisms.

## Entry Format

Use a short heading followed by a concise summary. Add focused bullets when
they help preserve important scope boundaries without becoming an
implementation plan:

```markdown
### Idea name

Describe the intended capability, who benefits, and the most important scope
boundary or dependency.

- Optionally record an important behavior or boundary.
```