From b31955fc97adec707b4e5f9a1b0c43c4cbc66750 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sat, 3 Oct 2026 17:56:34 +0000 Subject: [PATCH] Clarify PromptKit adoption requirements --- docs/roadmap/promptkit.md | 63 ++++++++++++++++++++++++++++----------- 1 file changed, 46 insertions(+), 17 deletions(-) diff --git a/docs/roadmap/promptkit.md b/docs/roadmap/promptkit.md index 260c7d3..96d2e89 100644 --- a/docs/roadmap/promptkit.md +++ b/docs/roadmap/promptkit.md @@ -128,10 +128,13 @@ from broken embedded assets under the existing operator error merely because validation moved earlier. Upstream permits partial profiles and defers endpoint/model completeness until -request overrides are applied. Preserve Audita's established checks for its -selected roles without adding a requirement that every unselected profile be -runnable. Native rejection of invalid unused definitions is different from -requiring all profiles to be complete. +request overrides are applied. Audita supplies no model override, so explicitly +validate that both selected role profiles have a nonblank model and a complete +effective endpoint during offline configuration validation and runtime setup. +Retain the application-backend, endpoint-match, and credential checks. Do not +rely on engine construction or profile inspection alone to establish selected +profile completeness. Unselected partial profiles remain permitted if they +satisfy upstream definition validation; they need not be runnable. Adapt source accounting to the new discovery/capture boundary. The existing recording filesystem wrapper may be replaceable with discovered inputs or @@ -153,11 +156,31 @@ prompt/version format. Keep the shared instruction fragments and the complete per-call output contracts owned by Audita. This migration does not authorize another prompt-quality rewrite or changes to domain validators. +The adapter supplies `HardeningPolicy`, `ProposalContract`, and +`ValidationContract` as template variables. Embedded prompts retain their +references to the applicable variables. Operator message overrides control +whether those variables are rendered; supplying a variable does not inject its +text into a template that omits it. Preserve this native override behavior and +document the distinction. Per-call schemas and domain validation remain enforced +regardless of those references. Unconditional shared-instruction injection is +outside this adoption. + Convert the six maintained profiles in `examples/profiles` and test-generated profile/prompt fixtures to the new format. Convert any old `base_profile` usage to `extends`, and remove embedded identities. Migrate tests that construct engines using removed options or old `Profile`/`Backend` values. +Reorganize the maintained profiles into separately selected local and service +directory roots. The current shared directory includes profiles referring to +backends absent from each example's application configuration; eager validation +would reject those unselected definitions after a format-only conversion. +Update `profiles_dir` in `examples/minimal-config.yml`, +`examples/production-config.yml`, and `examples/configkit-inheritance.yml` to +select the appropriate root. Keep the service testing profiles in the service +root so the testing overlay remains usable. Discovery is recursive: placing +both groups beneath a common selected root does not isolate them. Update example +links and validation recipes to match the final layout. + Review presence deliberately rather than mechanically preserving old serialized values. In particular, explicit zero temperature, top-p, or max-tokens now means an outbound zero; zero timeout disables PromptKit's generation deadline while @@ -176,9 +199,10 @@ without requiring every maintained example to use it. Update the profile section of `docs/config.md` and maintained examples when the implementation lands. Explain the broader accepted content of `profiles_dir`, -operator prompt overrides and their precedence, keyed identities, composition -versus precedence, profile inheritance, presence, eager final-definition -validation, selected-root errors, and captured configuration. Link to the pinned +operator prompt overrides, their precedence and optional use of shared instruction +variables, keyed identities, composition versus precedence, profile inheritance, +presence, eager final-definition validation, selected-root errors, and captured +configuration. Link to the pinned upstream reference for generic field semantics rather than reproducing its entire contract. @@ -198,11 +222,11 @@ under the [OSS policy](../policy/oss.md); retain the applicable LGPL/GPL and MIT texts rather than duplicating unchanged license documents. Dependency adoption does not change Audita's application license. -The profile format is publicly documented and incompatible with the old one. -Assess Audita's release classification under the -[release policy](../policy/release.md#versioning-and-compatibility); PromptKit's -own pre-1.0 minor version does not establish that this is an Audita minor update. -Version selection and publication belong to later release preparation. +Plan this adoption for an Audita major release under the +[release policy](../policy/release.md#versioning-and-compatibility). The public +profile format changes incompatibly and no compatibility loader is planned; +PromptKit's own pre-1.0 minor version does not change Audita's classification. +The exact version and publication timing remain deferred to release preparation. ## Verification and Completion @@ -211,8 +235,10 @@ and add focused behavioral coverage where the source migration creates a distinct risk. Completion means: - The released dependency set builds and the complete Audita test suite passes. -- All registered prompts load and render from keyed assets, with the established - per-call schemas and adapter-supplied shared instructions. +- All registered embedded prompts load and render from keyed assets, retaining + their applicable shared-instruction references and established per-call schemas. +- Offline configuration validation rejects incomplete selected profiles, including + a missing or blank model, while accepting valid unselected partial profiles. - Selected roles use final application backend settings and direct resolved credentials; both roles still share configured capacity, including explicit zero queue capacity. Catalog layering does not introduce a competing lookup. @@ -220,15 +246,18 @@ distinct risk. Completion means: through the real runtime. Directory conflicts and invalid final definitions produce useful startup errors without filename-derived identity assumptions. - Operator prompt and backend contributions follow the accepted source order; - prompt overrides render while Audita retains its per-call schemas and + prompt overrides render, including when their messages omit shared-instruction + variables, while Audita retains its per-call schemas, domain validation, and authoritative application backend settings. - Source collision protection, safe inspection, and operator-versus-asset error classification remain effective at the changed construction boundary. - Representative prepared execution, schema repair, transient retry, and partial-success tests retain their existing behavior. Reuse adequate coverage rather than duplicating PromptKit's complete upstream test matrix. -- Maintained examples and canonical documentation describe the migrated format; - attribution identifies the selected PromptKit and catalog sources accurately. +- Local, service, testing-overlay, and ConfigKit inheritance examples validate + offline using their separately selected profile roots. Canonical documentation + describes the migrated format, shared-instruction behavior, and major-release + migration; attribution identifies the selected library sources accurately. No live provider calls, new release gates, or CI-success dependency are needed to establish adoption. Guided domain recovery and the other work recorded in