86 lines
3.8 KiB
Markdown
86 lines
3.8 KiB
Markdown
# Future Feature Ideas
|
|
|
|
## Purpose
|
|
|
|
This document catalogs reasonably specific ideas that may be useful in future
|
|
Promptkit development. It is an idea pool, not a commitment, schedule, or
|
|
description of current behavior.
|
|
|
|
Ideas belong here while they are worth retaining but have not been selected
|
|
for active development. Keep each entry at the level of intended capability,
|
|
consumer value, and important scope boundaries. Defer API design,
|
|
implementation details, sequencing, and acceptance criteria until an idea is
|
|
selected.
|
|
|
|
## Using This Catalog
|
|
|
|
- Add an idea when its purpose and likely value can be stated clearly.
|
|
- Keep entries independent enough that maintainers can evaluate and select
|
|
them individually.
|
|
- Note significant dependencies or boundary concerns, but do not turn entries
|
|
into implementation plans.
|
|
- Treat inclusion as an invitation to evaluate, not as approval or priority.
|
|
- When an idea is selected, move its active planning to a focused roadmap or,
|
|
when it requires a durable architectural decision, an ADR. Update
|
|
current-state documentation only when implementation lands.
|
|
- Remove ideas that are no longer relevant. Retain a rejected idea only when
|
|
its rationale is likely to prevent repeated reconsideration.
|
|
|
|
Future capabilities must continue to respect the
|
|
[architecture policy](../policy/architecture.md), particularly Promptkit's
|
|
role as an application-neutral library and its boundary with downstream
|
|
consumers.
|
|
|
|
## Ideas
|
|
|
|
### Backend-specific concurrency management
|
|
|
|
Extend the
|
|
[LLM backend registry](../consumers/pkg-promptkit.md#register-a-custom-backend)
|
|
with optional per-backend concurrency limits and bounded, buffered admission
|
|
queues. Promptkit could then route simultaneous
|
|
generation requests according to backend capacity while containing accidental
|
|
runaway submission. Downstream consumers would continue invoking synchronous
|
|
`Run` calls, including concurrently from multiple goroutines, and each
|
|
admitted call would wait for and return its ordinary result.
|
|
|
|
- Scope limits to an engine instance rather than hidden process-global state.
|
|
- Give different backend IDs independent capacity pools. A profile endpoint
|
|
override would remain part of its selected backend's pool.
|
|
- Configure active concurrency and waiting capacity separately. Concurrency
|
|
protects the backend, while queue capacity protects the process from
|
|
admitting an unbounded backlog.
|
|
- Give queue capacity a generous, configurable bounded default intended as a
|
|
safety ceiling for bugs or unintended loops rather than a routine
|
|
application constraint. Select an exact default during implementation
|
|
planning and measurement.
|
|
- Reject a call with a recognizable capacity error when its backend queue is
|
|
full rather than allowing it to wait outside the bounded queue.
|
|
- Admit requests before expensive preparation and artifact copying where
|
|
practical so queued work remains lightweight.
|
|
- Apply a limit to each actual generation request, including repair attempts,
|
|
without unnecessarily serializing prompt preparation.
|
|
- Make queued and active waits respect caller cancellation and deadlines.
|
|
- Treat concurrency as backend policy rather than a profile-level model
|
|
setting.
|
|
- Keep the queue ephemeral and in-process, with no survival guarantee across
|
|
engine or process shutdown.
|
|
- Preserve the existing execution model as far as practical. Durable jobs,
|
|
polling, priorities, application worker lifecycle, retries, and
|
|
cross-process coordination would be separate future capabilities.
|
|
|
|
## Entry Format
|
|
|
|
Use a short heading followed by a concise summary. Add focused bullets when
|
|
they help preserve important scope boundaries without becoming an
|
|
implementation plan:
|
|
|
|
```markdown
|
|
### Idea name
|
|
|
|
Describe the intended capability, who benefits, and the most important scope
|
|
boundary or dependency.
|
|
|
|
- Optionally record an important behavior or boundary.
|
|
```
|