promptkit/docs/roadmap/audita-feature-requests.md

# Audita Consumer Feature Requests

**Last updated:** 2026-09-23

**Status:** Proposed consumer requests, based on a review of Promptkit v0.10.0.
These are not approved features, implementation plans, delivery commitments, or
claims about current behavior. Priorities reflect Audita's prospective adoption;
Promptkit should evaluate them against its broader consumer needs.

## Consumer Context

Audita polishes transcripts through ordered correction modules. It makes LLM calls
both to propose corrections and to assess proposed changes, alongside deterministic
domain validation. It needs bounded execution, understandable failures, and useful
run diagnostics without disclosing credentials or transcript content unnecessarily.

Audita would like Promptkit to own shared prompt and LLM execution capabilities,
while Audita retains transcript processing, module orchestration, domain acceptance,
correction application, reports, and diagnostic artifact retention. Suite-wide
consistency is the adoption goal; these requests do not ask Promptkit to reproduce
Audita's existing APIs or internal architecture.

The [consumer guide](../consumers/pkg-promptkit.md),
[format reference](../formats.md), and
[provider integration contract](../integrations/openai-compatible-chat.md)
remain the owners of implemented behavior. This document specifies desired
consumer outcomes only; API shape and implementation are left to Promptkit.

## 1. Transient-Failure Retries

**Priority: High.** Audita should not need its own general-purpose provider retry
policy to recover from temporary rate limiting or service failures.

Requested behavior:

- Retry HTTP 429 and selected transient server failures. Suggested initial defaults
  cover 500, 502, 503, and 504; retrying every 5xx status is not a requirement.
- Supply reasonable documented defaults. Two additional attempts is a proposed
  starting point, with a configurable maximum and an unambiguous explicit zero
  that disables retries.
- Delay retries to avoid immediate repeated requests, honor applicable provider
  retry timing, and bound the operation by the caller's cancellation/deadline.
  Cancellation must interrupt waiting as well as active requests.
- Define which transport failures are eligible. Document that an uncertain response
  can mean the provider already performed billable work; retries do not imply
  exactly-once generation.
- Keep retry attempts tied to the same prepared generation request. An automatic
  retry must not silently select different prompt content, model settings, or
  credential identity.
- Distinguish transient-failure retries from content-repair calls. Document how
  their budgets combine so consumers can reason about maximum request counts,
  latency, and potential cost.
- State whether the policy applies to built-in and injected clients. Consumers
  must be able to avoid duplicate retries when another collaborator already owns
  retry behavior.

Expected consumer evidence includes a transient failure followed by success,
exhaustion, explicit retry disablement, cancellation during a delay, and interaction
with content repair. Permanent request failures must remain distinguishable.

## 2. Execution Observations, Including Failed Runs

**Priority: High.** Audita needs to explain the time and calls consumed by a logical
operation, including when that operation never produces a successful result.

Requested observations include:

- Correlation between the logical operation and its individual attempts, selected
  prompt, profile, and backend.
- Attempt outcomes and enough timing information to distinguish capacity waiting,
  generation, and retry delays.
- Separate counts for transient-failure retries and content-repair calls.
- Available provider-reported token usage, including completed calls preceding a
  later failure, with unknown usage distinguishable from measured zero.
- Clear accounting boundaries so consumers do not double-count aggregate and
  individual-attempt measurements.

Observations should be usable by application-owned reports, logging, and metrics
without requiring a particular telemetry system or artifact layout. Default
observations should contain metadata rather than credentials or prompt/response
bodies. Any content disclosure should be an explicit consumer choice, with its
sensitivity clearly documented.

Current result metadata and usage remain useful; the request is to make execution
and failure accounting sufficiently complete for downstream diagnostics.

## 3. Actionable Failure Classification

**Priority: High; closely related to retries and observations.** Audita needs to
choose between failing a run, rejecting affected work, and recording a warning
without interpreting human-readable error messages.

Build on existing public error and validation-result contracts so consumers can
reliably distinguish cancellation, deadline expiration, transport failure,
non-success provider status, malformed provider response, exhausted retries,
capacity rejection, and completed output-validation failure.

On retry exhaustion, retain the underlying failure classification and attempt
count. Preserve safe default diagnostics while allowing deliberate access to
provider detail under the consumer's disclosure policy. Document which outcomes
carry generated output, validation findings, and partial execution observations.

Promptkit need not decide Audita's recovery or correction-acceptance policy.

## 4. Simple Per-Backend Capacity Limits

**Priority: Adoption requirement, largely supported by existing facilities.**
Audita needs only two protections:

- For a local backend with a hard simultaneous-query limit, keep combined Audita
  traffic within its assigned capacity. For example, a DGX Spark serving
  gemma-4-31b-it may allow four simultaneous queries.
- For a paid API backend, use a reasonable finite concurrency cap so a large
  transcript or pipeline cannot accidentally launch dozens or thousands of
  simultaneous billable calls.

Proposal and validation calls targeting the same backend should share its limit.
The protection must cover initial calls, transient retries, and content-repair
calls. Audita does not need separate proposal/validation budgets, named workload
limits, or an additional aggregate ceiling across backends.

Promptkit's existing engine-local, per-backend capacity controls are the intended
foundation. Audita should select finite limits for the backends it uses; adoption
must not accidentally leave paid endpoints on an unlimited path. Exact defaults
remain to be selected. No new fine-grained scheduling feature is requested.

Verify the documented capacity scope in the consumer example, especially as retries
are added. An engine-local limit does not coordinate other processes or independent
clients using the same service; a shared local backend must have capacity allocated
accordingly or enforced at the service boundary. Cross-process coordination is not
an upstream feature request here.

Concurrency limits bound simultaneous activity, not total token usage or spending
across a long run. A monetary budget or rate-limiting system is not requested.

## 5. Representative Consumer Example

**Priority: High as adoption evidence.** Provide a maintained offline example of a
structured proposal call followed by an independent LLM validation call, using
embedded prompts/schemas and separate application-owned execution profiles.

Demonstrate transient failure handling, schema-invalid output, cancellation, and
application collection of execution observations. Use synthetic inputs and
controlled model responses without credentials or paid provider calls.

Keep schema validity distinct from domain acceptance: a structurally valid response
may still be rejected by Audita's deterministic or semantic checks. The example
should show how an application uses Promptkit's public contracts, not duplicate
Audita's processing pipeline.

## 6. Licensing Recommendation

Recommend offering a future Promptkit release under LGPL 3.0, subject to the
maintainer confirming the relevant rights and dependency terms. Audita's intent is
to retain its BSD-3-Clause application licensing while adopting a shared library
whose modifications can remain subject to library copyleft.

The requested outcome is clear, consistent license notices and consumer guidance
for the published library and its dependencies, including the obligations relevant
to statically linked Go binaries. LGPL is not a waiver of source, notice, or
recombination/relinking requirements; see [LGPLv3 section 4](https://www.gnu.org/licenses/lgpl-3.0.en.html).
This proposal does not change existing releases or their licenses.

## Suggested Evaluation Order and Scope

Evaluate retries, failure classification, and execution observations together, then
use the representative consumer example to assess readiness and verify shared
per-backend limits. Existing capacity controls should suffice unless that exercise
reveals a concrete gap. Resolve licensing before selecting a release for Audita's
migration.

Existing facilities for embedded sources, application-owned profiles, prepared
execution, JSON-schema validation, and request-scoped credentials are useful
adoption foundations rather than new requests. Audita can bridge its application
configuration and credentials into Promptkit's public API; a dependency between
Promptkit and ConfigKit is not requested.

Transcript normalization, chunking, module ordering, semantic validators, correction
application, correction-ledger assembly, and filesystem retention remain Audita's
responsibility. No requested feature requires Promptkit to own those concerns.