audita/docs/roadmap/promptkit.md

PromptKit v0.12.0 Adoption Roadmap

Last updated: 2026-10-03

Status: Accepted target scope; implementation is defined in the implementation plan. Implementation has not begun.

Purpose and Baseline

Adopt PromptKit v0.12.0 now that the independent ConfigKit v0.5.0 upgrade is complete. Align Audita with the libraries' native configuration model while retaining Audita's domain rules, application-owned backend settings, and resilient processing workflow.

At reviewed Audita revision 8cba751835ba9faad340e933a3cfbb59c03924fd, go.mod selects ConfigKit v0.5.0 and PromptKit v0.11.2, with OpenRouter and Rakestrawhome /v2 catalogs at v2.20260926.0. Go 1.25.5 remains the application minimum. The implemented contracts belong to configuration, the LLM integration, and the LLM runtime.

internal/framework/llm.NewRuntime currently assembles one engine using the old prompt/profile filesystem options and programmatic backend registration. Audita embeds seven prompt definitions and separate response schemas. Operators select proposal and validation profiles from profiles_dir; the effective backend must be registered in application configuration, its endpoint must match, and its credential comes from Audita's ConfigKit-resolved settings. Successful profile reads are recorded to protect those sources from output collisions. Both roles share the same engine's backend admission controls.

The reviewed upstream release is PromptKit v0.12.0, commit 4b8169da286917fde7875425bb32b799b12c6af3. It depends on ConfigKit v0.5.0 and both maintained /v3 catalog modules at v3.20260930.0. This is an intentional incompatible pre-1.0 minor release: it replaces construction and configuration APIs, rather than simply updating provider data. PromptKit's format reference and consumer migration guide own the upstream details.

Target End State

• Audita imports released PromptKit v0.12.0 and its /v3 catalogs, retaining ConfigKit v0.5.0 and using no local dependency replacement.
• The runtime uses Config.Sources and native ConfigKit source groups. PromptKit owns composition, inheritance, presence, final-definition validation, prompt capture, and execution. Audita does not translate old profiles or implement a second YAML loader or inheritance resolver.
• Embedded prompts and maintained operator profiles use keyed prompts and profiles mappings. Existing prompt IDs, versions, role selections, instructions, and response-schema associations remain the adoption baseline. The application configuration schema, overlay selection, CLI interface, and main-file-relative profiles_dir resolution remain in place.
• Native profile extends, field-presence semantics, and source-relative resource rules apply. Documents can contain multiple definitions, and IDs come from mapping keys rather than filenames. The existing profiles_dir accepts native unified PromptKit documents containing profiles, backends, and prompts, including intentional operator prompt overrides.
• Application backend endpoints, configured concurrency and queue capacity, and credential ownership remain authoritative for selected roles. Preserve the existing profile-selection and prepared-target checks; do not introduce a second admission controller or implicit provider credential lookup.
• A runtime uses one captured configuration snapshot for both roles. Each CLI invocation constructs a new engine; hot reload is not part of this feature. Existing inspection remains offline and safe, and consumed host sources remain protected from output collisions.
• PromptKit retains responsibility for transport retries, schema repair, observation, and prepared execution. Preserve Audita's configured retry and repair budgets, operation deadlines, cancellation, partial-success behavior, error/recovery reporting, and debug selection.

Configuration Assembly and Ownership

Replace removed WithPromptFS, WithProfileFS, and WithBackend usage with the released source API. Discover embedded YAML through FilesystemInputs and operator YAML through DirectoryInputs, with explicit caller tiers. Keep the retained schema filesystem integration and HTTP client configuration. Use the helpers' default-disabled imports; this adoption does not require new application source-selection flags or operator underlay tiers.

PromptKit supplies catalog definitions first and its own defaults next. Caller tiers then overlay in their supplied order, followed by an optional highest priority main input. Files within one directory group compose strictly; alphabetical filenames do not establish override precedence. Use separate tiers for genuine priority differences rather than manufacturing file-order semantics.

Represent application backend settings as a final programmatic source, using BackendSource or ConfigKit mappings where explicit null is needed. Preserve configured zero queue capacity as an explicit value. Suppress backend-derived credential environment lookup explicitly where catalog layering would otherwise introduce it; omission alone can retain a lower-tier value. Continue detecting competing effective profile credential lookups rather than silently clearing operator profile fields. A missing backend credential remains valid for an unauthenticated local service unless the selected profile requires a key.

Catalog backend IDs are no longer reserved upstream. Audita should not invent a new reservation rule. Catalog selection alone must not bypass application registration, endpoint, capacity, or credential requirements. Catalog profile contents may evolve; this upgrade does not promise fixed model choices or live provider availability.

The accepted caller order is embedded Audita prompts, operator definitions, then application backend settings. The existing profiles_dir accepts native unified PromptKit documents containing profiles, backends, and prompts. Keep the name and path setting for continuity, and document its broader accepted content. This follows the upstream source API without a parallel parser or top-level filtering. Profiles, provider defaults, and intentional prompt overrides use the same native composition model.

Audita continues to ship its domain prompts, enforce its selected-backend requirements, and supply its per-call schemas. Operator prompt overrides are an accepted capability of this adoption; documentation, source accounting, and focused behavioral coverage should reflect them.

Work Required

Runtime Construction and Source Accounting

Migrate internal/framework/llm/runtime.go to layered construction and retain the role-bound adapter boundary. Review startup inspection and failure classification because upstream now validates all final definitions, including unused ones, during engine construction. Distinguish invalid operator inputs from broken embedded assets under the existing CLI exit contract; do not treat every construction failure as an operator error merely because validation moved earlier.

Upstream permits partial profiles and defers endpoint/model completeness until request overrides are applied. Audita supplies no model override, so explicitly validate that both selected role profiles have a nonblank model and a complete effective endpoint during offline configuration validation and runtime setup. Retain the application-backend, endpoint-match, and credential checks. Do not rely on engine construction or profile inspection alone to establish selected profile completeness. Unselected partial profiles remain permitted if they satisfy upstream definition validation; they need not be runnable.

Adapt source accounting to the new discovery/capture boundary. The existing recording filesystem wrapper may be replaceable with discovered inputs or upstream source metadata. Protect the host sources actually consumed by the runtime, including inherited and differently named definitions, without guessing filenames from selected profile IDs. Account for consumed operator message files as well. Keep this proportionate to the existing collision contract; do not add filesystem concurrency or power-loss guarantees.

Keep the existing safe effective-profile summaries and application explanations. Use upstream metadata where useful, but do not expose the full InspectConfiguration snapshot as ordinary CLI or report output: prompt text and arbitrary provider fields are outside those summaries' current scope.

Prompt Assets, Profiles, and Presence Semantics

Move each of the seven files in internal/prompts/assets/prompts to the keyed prompt/version format. Keep the shared instruction fragments and the complete per-call output contracts owned by Audita. This migration does not authorize another prompt-quality rewrite or changes to domain validators.

The adapter supplies HardeningPolicy, ProposalContract, and ValidationContract as template variables. Embedded prompts retain their references to the applicable variables. Operator message overrides control whether those variables are rendered; supplying a variable does not inject its text into a template that omits it. Preserve this native override behavior and document the distinction. Per-call schemas and domain validation remain enforced regardless of those references. Unconditional shared-instruction injection is outside this adoption.

Convert the six maintained profiles in examples/profiles and test-generated profile/prompt fixtures to the new format. Convert any old base_profile usage to extends, and remove embedded identities. Migrate tests that construct engines using removed options or old Profile/Backend values.

Reorganize the maintained profiles into separately selected local and service directory roots. The current shared directory includes profiles referring to backends absent from each example's application configuration; eager validation would reject those unselected definitions after a format-only conversion. Update profiles_dir in examples/minimal-config.yml, examples/production-config.yml, and examples/configkit-inheritance.yml to select the appropriate root. Keep the service testing profiles in the service root so the testing overlay remains usable. Discovery is recursive: placing both groups beneath a common selected root does not isolate them. Update example links and validation recipes to match the final layout.

Review presence deliberately rather than mechanically preserving old serialized values. In particular, explicit zero temperature, top-p, or max-tokens now means an outbound zero; zero timeout disables PromptKit's generation deadline while Audita's outer operation deadline still applies. Omission, blank values, and null have field-specific meanings. Empty provider maps do not erase inherited entries, and changing model/backend does not erase unrelated explicit settings.

Use the native meanings without a compatibility shim. Retaining the examples' literal temperature: 0 as an explicit provider setting is recommended; explain that omitting the field requests an unspecified value, and null suppresses an inherited control. Do not automatically turn every existing zero into omission or every omission into a zero. Demonstrate representative profile inheritance without requiring every maintained example to use it.

Documentation, Migration, and Attribution

Update the profile section of docs/config.md and maintained examples when the implementation lands. Explain the broader accepted content of profiles_dir, operator prompt overrides, their precedence and optional use of shared instruction variables, keyed identities, composition versus precedence, profile inheritance, presence, eager final-definition validation, selected-root errors, and captured configuration. Link to the pinned upstream reference for generic field semantics rather than reproducing its entire contract.

Include operator migration guidance for old profile documents and behaviorally significant zeros/nulls. This is a deliberate format migration, not an automatic upgrade of existing installation files. Preserve supported application overlays, including the cheaper/faster testing-profile workflow.

Update the pinned consumer link in docs/integrations/openai-compatible-llm.md, construction/source-accounting descriptions in docs/internal/llm-runtime.md, and asset-format guidance in docs/internal/prompts.md. Update other affected references only where implemented behavior changes.

Update go.mod, go.sum, and docs/oss/attribution.md for PromptKit and the two catalog modules. Verify exact selected source revisions and upstream notices under the OSS policy; retain the applicable LGPL/GPL and MIT texts rather than duplicating unchanged license documents. Dependency adoption does not change Audita's application license.

Plan this adoption for an Audita major release under the release policy. The public profile format changes incompatibly and no compatibility loader is planned; PromptKit's own pre-1.0 minor version does not change Audita's classification. The exact version and publication timing remain deferred to release preparation.

Verification and Completion

Use existing offline tests as the starting point. Update fixtures intentionally, and add focused behavioral coverage where the source migration creates a distinct risk. Completion means:

• The released dependency set builds and the complete Audita test suite passes.
• All registered embedded prompts load and render from keyed assets, retaining their applicable shared-instruction references and established per-call schemas.
• Offline configuration validation rejects incomplete selected profiles, including a missing or blank model, while accepting valid unselected partial profiles.
• Selected roles use final application backend settings and direct resolved credentials; both roles still share configured capacity, including explicit zero queue capacity. Catalog layering does not introduce a competing lookup.
• Representative keyed profiles, native inheritance, and explicit presence work through the real runtime. Directory conflicts and invalid final definitions produce useful startup errors without filename-derived identity assumptions.
• Operator prompt and backend contributions follow the accepted source order; prompt overrides render, including when their messages omit shared-instruction variables, while Audita retains its per-call schemas, domain validation, and authoritative application backend settings.
• Source collision protection, safe inspection, and operator-versus-asset error classification remain effective at the changed construction boundary.
• Representative prepared execution, schema repair, transient retry, and partial-success tests retain their existing behavior. Reuse adequate coverage rather than duplicating PromptKit's complete upstream test matrix.
• Local, service, testing-overlay, and ConfigKit inheritance examples validate offline using their separately selected profile roots. Canonical documentation describes the migrated format, shared-instruction behavior, and major-release migration; attribution identifies the selected library sources accurately.

No live provider calls, new release gates, or CI-success dependency are needed to establish adoption. Guided domain recovery and the other work recorded in future.md remain separate features. Do not expand this upgrade into new report formats, pipeline scheduling, or general configuration-inspection commands. The implementation plan owns execution steps and their verification.