3.8 KiB
Future Feature Ideas
Purpose
This document catalogs reasonably specific ideas that may be useful in future Promptkit development. It is an idea pool, not a commitment, schedule, or description of current behavior.
Ideas belong here while they are worth retaining but have not been selected for active development. Keep each entry at the level of intended capability, consumer value, and important scope boundaries. Defer API design, implementation details, sequencing, and acceptance criteria until an idea is selected.
Using This Catalog
- Add an idea when its purpose and likely value can be stated clearly.
- Keep entries independent enough that maintainers can evaluate and select them individually.
- Note significant dependencies or boundary concerns, but do not turn entries into implementation plans.
- Treat inclusion as an invitation to evaluate, not as approval or priority.
- When an idea is selected, move its active planning to a focused roadmap or, when it requires a durable architectural decision, an ADR. Update current-state documentation only when implementation lands.
- Remove ideas that are no longer relevant. Retain a rejected idea only when its rationale is likely to prevent repeated reconsideration.
Future capabilities must continue to respect the architecture policy, particularly Promptkit's role as an application-neutral library and its boundary with downstream consumers.
Ideas
Backend-specific concurrency management
Extend the
LLM backend registry
with optional per-backend concurrency limits and bounded, buffered admission
queues. Promptkit could then route simultaneous
generation requests according to backend capacity while containing accidental
runaway submission. Downstream consumers would continue invoking synchronous
Run calls, including concurrently from multiple goroutines, and each
admitted call would wait for and return its ordinary result.
- Scope limits to an engine instance rather than hidden process-global state.
- Give different backend IDs independent capacity pools. A profile endpoint override would remain part of its selected backend's pool.
- Configure active concurrency and waiting capacity separately. Concurrency protects the backend, while queue capacity protects the process from admitting an unbounded backlog.
- Give queue capacity a generous, configurable bounded default intended as a safety ceiling for bugs or unintended loops rather than a routine application constraint. Select an exact default during implementation planning and measurement.
- Reject a call with a recognizable capacity error when its backend queue is full rather than allowing it to wait outside the bounded queue.
- Admit requests before expensive preparation and artifact copying where practical so queued work remains lightweight.
- Apply a limit to each actual generation request, including repair attempts, without unnecessarily serializing prompt preparation.
- Make queued and active waits respect caller cancellation and deadlines.
- Treat concurrency as backend policy rather than a profile-level model setting.
- Keep the queue ephemeral and in-process, with no survival guarantee across engine or process shutdown.
- Preserve the existing execution model as far as practical. Durable jobs, polling, priorities, application worker lifecycle, retries, and cross-process coordination would be separate future capabilities.
Entry Format
Use a short heading followed by a concise summary. Add focused bullets when they help preserve important scope boundaries without becoming an implementation plan:
### Idea name
Describe the intended capability, who benefits, and the most important scope
boundary or dependency.
- Optionally record an important behavior or boundary.