From afcf58331f50dadcee24709d93bc0b315a106402 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sat, 3 Oct 2026 18:21:48 +0000 Subject: [PATCH] Plan the PromptKit upgrade in nine stages --- docs/roadmap/implementation.md | 648 ++++++++++++++++++++++----------- docs/roadmap/promptkit.md | 8 +- 2 files changed, 435 insertions(+), 221 deletions(-) diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 2473665..8a404af 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,246 +1,460 @@ -# Implementation Plan: ConfigKit v0.5.0 Adoption +# Implementation Plan: PromptKit v0.12.0 Adoption **Last updated:** 2026-10-03 -**Status:** Complete. +**Status:** Ready for implementation; no stages completed. **Audience:** Luna coding agent. ## Scope and Execution -Implement the target state in the [ConfigKit roadmap](configkit.md), which owns -feature intent, scope, and compatibility considerations. Execute the numbered -stages below in order. Each stage is a bounded implementation prompt; the -complete feature comprises all five stages. Commit each completed unit with a -short, plain English message under the [agent policy](../policy/agents.md). -No approval gate is needed between stages. +Implement the accepted target in the [PromptKit roadmap](promptkit.md). That +document owns feature intent and policy choices; this plan owns the ordered +implementation work. The planning baseline is Audita commit +`b31955fc97adec707b4e5f9a1b0c43c4cbc66750`: ConfigKit v0.5.0 is adopted, +PromptKit is v0.11.2, and the finalized roadmap includes the selected-profile, +operator-override, example-layout, and major-release clarifications. -Begin with [development guidance](../development.md) and the applicable agent, +This replaces the completed ConfigKit plan at this path. Its +[completed implementation record](https://gitea.maximumdirect.net/eric/audita/src/commit/b31955fc97adec707b4e5f9a1b0c43c4cbc66750/docs/roadmap/implementation.md) +remains available for readers following the ConfigKit completion roadmap. + +Execute Stage 1 through Stage 9 in order. Each is sized as one bounded Luna +implementation prompt. Commit completed units with short, plain English messages +under the [agent policy](../policy/agents.md); no approval round is needed between +stages. Record actual checks and any remaining expected migration failures. +Stages 1 and 2 change formats before Stage 3 changes their consumer. Intermediate +commits need not be runnable or release-ready. Do not add temporary readers, +compatibility shims, fallback engines for execution, or scaffolding to make them +so. Later stages complete the same migration rather than separate product modes. + +Start with [development guidance](../development.md) and the applicable agent, [architecture](../policy/architecture.md), [testing](../policy/testing.md), [documentation](../policy/documentation.md), and [OSS](../policy/oss.md) -policies. Use graph-first discovery and read the relevant current contracts. -The planning baseline is Audita commit -`23e1df4cfb844d3fdeb1fb64483c1038cc4f2e21`; application code remains at the -roadmap's reviewed revision. Target released ConfigKit v0.5.0, source commit -`dc09959cca4e105f49ec316c6d4ba585bd2340bd`. +policies. Use graph-first discovery. Read the current +[configuration](../config.md), [LLM runtime](../internal/llm-runtime.md), +[prompt registry](../internal/prompts.md), and +[LLM integration](../integrations/openai-compatible-llm.md) contracts for the +established behavior this migration preserves. + +Target the released PromptKit v0.12.0 at commit +`4b8169da286917fde7875425bb32b799b12c6af3`, ConfigKit v0.5.0, and both +maintained `/v3` catalogs at `v3.20260930.0`. Read the pinned +[format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.12.0/docs/formats.md) +and [migration guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.12.0/docs/consumers/pkg-promptkit.md#migrate-from-the-previous-configuration-api). +The sibling checkout is reference material, not a dependency replacement. ### Selected Implementation Decisions -- Keep `internal/core/config.Load` using ordinary `ck.Load`. Native inheritance - already runs before decoding and application resolution; another resolver or - `LoadLayered` conversion is unnecessary. -- Keep PromptKit at v0.11.2 and its `/v2` catalogs at their existing versions. - Retain Go 1.25.5, configuration `version: 2`, existing profile syntax, role - selection, CLI flags, file discovery, overlays, and main-file-relative paths. -- Preserve absence-based defaults. Inherited input is explicit input, including - zero, false, empty, and null; existing field rules still decide validity. - Every backend, including inheritance parents and unselected backends, remains - subject to ordinary validation and surviving credential resolution. -- Preserve existing explanation fields and add an optional `inheritance` - sequence to each field record. Each step has `from`, `to`, and `reference` - string paths copied from `Origin.Inheritance` through `Path.String()`. - This additive, value-free projection serves both `--view explain` and debug - configuration origins through their existing shared function. It is not a - new inspection command or a complete upstream metadata serialization. -- The `extends` keyword is reserved by the library throughout application input - mappings. Document its compatibility impact, including legacy backend IDs - named `extends`; do not add a compatibility translator or a second naming - restriction to application validation. +- Keep Go 1.25.5, application `version: 2`, current flags, role selection, + application overlays, and main-file-relative path rules. Retain the + `profiles_dir` name and default; broaden its accepted documents as agreed. +- Use PromptKit's catalog and owned defaults, then caller underlays for embedded + Audita prompts and operator directory inputs, then `Sources.Main` for + application backend settings. Directory inputs compose strictly within their + group; discovery order supplies no field precedence. Keep imports disabled. +- Use one ConfigKit mapping for the final application source. For each + application backend, supply its endpoint, `concurrency_limit`, + `queue_capacity`, and explicit `api_key_env: null`. The null suppresses + lower-tier credential lookup. Leave other supported provider fields available + to native layering. Do not put resolved credential values in this mapping. +- Keep selected-role registration, endpoint-match, competing-credential, and + required-key checks. Add explicit nonblank effective model and endpoint checks + for both selected roles. Valid unselected partial profiles remain permitted. +- Preserve the adapter's complete per-call schemas, configured repairs/retries, + operation context, shared variables, observation, and prepare/check/run flow. + Operator messages can omit shared-variable references; do not append the + shared text unconditionally or require particular message counts or roles + beyond PromptKit's own format rules. +- Use captured source inspection for consumed host configuration and message + paths. Keep source-relative resource resolution with PromptKit and ConfigKit; + Audita projects paths for its existing output collision checks. +- Replace the old profile-reload test expectation with snapshot behavior. + Successful construction uses only final effective definitions. A failed + construction may make an embedded-only diagnostic construction to distinguish + asset defects from operator errors; it never executes or substitutes for the + requested configuration. +- Maintained example roots are `examples/profiles/local/` and + `examples/profiles/service/`. Service testing profiles stay in the service + root. Keep the literal `temperature: 0` in examples as an explicit outbound + setting, and use native presence semantics without a compatibility translator. -Follow existing ownership boundaries. Production changes beyond the dependency -and explanation projection are warranted only if focused integration checks -expose a defect in the agreed behavior. Do not change report schemas, provider -policy, debug enablement, or application validation to accommodate test fixtures. -Do not add live-provider checks or permanent release gates. This plan does not -select a release version, modify release policy, publish, tag, or push. +This unit is an intentional major-release migration. Exact version selection, +release notes, tag/push, and publication timing remain release preparation under +the [release policy](../policy/release.md). Guided recovery, new report formats, +hot reload, extra configuration commands, and scheduling changes remain outside +this plan. Tests should protect changed integration behavior and established +contracts, not reproduce the libraries' entire validation matrices. -## Stage 1 — Upgrade the Dependency and Its Attribution +## Stage 1 — Convert Embedded Prompt Definitions -**Completion:** Complete. +**Primary files:** The seven YAML files under +`internal/prompts/assets/prompts/`. -**Primary files:** `go.mod`, `go.sum`, `docs/oss/attribution.md`. +1. Wrap each existing definition beneath `prompts`, its exact current ID, and + the string version `v1`. Remove the old inner `id` and `version` fields. + Keep filenames and Audita registry identifiers unchanged; filenames are + asset locations rather than PromptKit identities. +2. Preserve messages, block-scalar text, variable references, task examples, + output settings, and schema paths. Preserve the applicable references to + `HardeningPolicy`, `ProposalContract`, and `ValidationContract`. Do not edit + shared fragments, response schemas, or domain policies to accommodate the + new container format. +3. Review each before/after definition structurally, including YAML indentation + inside block scalars. Do not introduce default profile selections or new + repair counts. The adapter continues to own the runtime output contract. -1. Review the current dependency declarations and upgrade only ConfigKit: - `go get gitea.maximumdirect.net/eric/ConfigKit@v0.5.0`. Review the resolved - module diff; preserve PromptKit, both catalog modules, the Go minimum, and - unrelated dependency selections. Do not add a local replacement. -2. Check the selected release's `COPYRIGHT`, `LICENSE`, and `go.mod` against the - pinned upstream revision. Update the ConfigKit attribution entry's version - and commit-pinned source and notice links. Preserve its verified copyright, - LGPL-3.0-or-later expression, and existing LGPL/GPL text links when unchanged. - Follow the existing House Format; no additional licensing machinery is needed. -3. Leave the loader and profile integration intact. If the dependency introduces - a compilation failure, make only the API adjustment needed for this upgrade, - recording the actual difference rather than broadening scope. +**Verification and completion:** Check all seven migrated identities and +version keys against `RegisteredMetadata`, and run `git diff --check`. Registry +preparation tests become runnable with the new constructor in Stage 3. Commit +the asset conversion; do not introduce an old-format reader to test it here. + +## Stage 2 — Migrate Operator Profiles, Example Roots, and YAML Fixtures + +**Primary files:** `examples/profiles/`, the three maintained configuration +examples, and profile-producing test fixtures in +`internal/framework/llm/{runtime_inspection,adapter,adapter_capacity}_test.go`, +`internal/framework/proposal_generation/adapter_integration_test.go`, +`internal/framework/validators/llm_integration_test.go`, +`internal/framework/runner/cancellation_test.go`, and +`cmd/audita/main_integration_test.go`. + +1. Move local proposal/validation examples beneath `profiles/local/`; move the + four service proposal/validation and testing files beneath + `profiles/service/`. Convert each to `profiles: {: }` in readable + YAML. Keep IDs, models, positive timeouts, and controls, including explicit + zero temperature. No example needs a real credential or live model. +2. Set `profiles_dir: profiles/local` in `examples/minimal-config.yml` and + `examples/configkit-inheritance.yml`; set `profiles_dir: profiles/service` + in `examples/production-config.yml`. Preserve the testing overlay's role IDs + and the existing ConfigKit inheritance example. Select the leaf root, since + recursive discovery at their common parent would still load both groups. +3. Convert profile YAML emitted by the listed fixtures to keyed definitions. + Search for remaining old profile construction strings and removed + `base_profile` syntax, and migrate any genuine profile fixture found. + Do not rewrite transcript/proposal `id` fields or domain schema fixtures. + Keep test source names independent of profile IDs where already exercised. +4. Keep negative tests negative for their selected behavioral reason. Give + cases clean temporary layouts or replace their contribution completely + rather than leaving an unrelated invalid definition behind for later cases. + Eager validation now reads unselected final definitions too. Do not remove + such definitions merely to simulate the old lazy loader where the test is + intended to exercise eager validation. + +**Verification and completion:** Review the six profiles, explicit root +selections, and changed fixture text; run `git diff --check` and format touched +Go test files. These formats become consumable in Stage 3. Commit this unit. +Current documentation links and recipes are updated together in Stage 8. + +## Stage 3 — Adopt the Released Source API and Selected-Role Checks + +**Primary files:** `go.mod`, `go.sum`, +`internal/framework/llm/runtime.go`, and `internal/prompts/registry_test.go`. + +1. Upgrade with `go get gitea.maximumdirect.net/eric/promptkit@v0.12.0` and + reconcile the module files. Verify ConfigKit stays v0.5.0, the catalog + dependencies become `/v3` at `v3.20260930.0`, and obsolete `/v2` requirements + are removed when unused. Retain the Go minimum and unrelated selections; + introduce no local replacements. +2. Resolve `profiles_dir` exactly as the current runtime does. Obtain embedded + inputs with `pk.FilesystemInputs` and operator inputs with `pk.DirectoryInputs`. + An unavailable selected root is an operator error; an empty directory can + contribute no definitions. Do not require one profile per file, IDs matching + filenames, or profile-only top-level content. +3. Construct the agreed `pk.ConfigurationSources`: two ordered caller underlays + for embedded and operator groups, and one highest-priority application + `ck.Input` whose source is the mapping specified above. Retain + `pk.WithSchemaFS`, the supplied HTTP client, and configured transient retries. + Remove old prompt/profile/backend registration options. +4. Inspect registered prompt ID/version selections, then both selected roles. + Reject a selected role whose effective model or endpoint is empty or + whitespace-only. Retain backend registration, normalized endpoint equality, + nonblank effective `APIKeyEnv` rejection, and required-credential checks. + Preserve literal IDs; do not trim/re-key identities or require all profiles + to be complete. Populate the same role inspection map for both clients. +5. Migrate the registry test engine to `FilesystemInputs` and a programmatic + `ProfileSource` fixture in its own tier, retaining its schema filesystem. + A complete endpoint/model fixture without an application backend is valid + for this direct library test; Audita's selected-role guard belongs in its + runtime tests. Keep the embedded prompt preparation assertions scoped to + embedded prompts. +6. Remove the old `profileSourceFS` wrapper once unused. Source accounting is + completed in Stage 5. Preserve the adapter boundary; change other production + callers only to resolve actual API incompatibilities. Search for remaining + uses of removed source options and old `pk.Profile`/`pk.Backend` values. + Run `go mod tidy` after migrating construction callers and review its diff. + +**Verification and completion:** Run `go test ./internal/prompts` and the +existing offline profile assembly, unsafe-selection, and absolute-directory +tests in `internal/framework/llm`. Run `git diff --check` and review the module +diff. The obsolete file-reload assertion is intentionally replaced in Stage 6; +do not restore reload behavior to make that assertion pass. Commit the runtime +API migration. + +## Stage 4 — Preserve Startup Failure Classification and Validate Assembly + +**Primary files:** `internal/framework/llm/runtime.go`, +`internal/framework/llm/runtime_inspection_test.go`, and configuration-command +tests in `internal/cli`. Update `internal/cli/run.go` only where the changed +startup path needs existing exit behavior preserved. + +1. Treat embedded discovery failures as `AssetError`; treat operator-directory + discovery failures as configuration input errors. Preserve wrapped causes + for `errors.Is`/`errors.As` and retain configured-secret redaction at CLI + boundaries. Caller cancellation remains a runtime exit, not an operator + mistake, and should not trigger further diagnostic loading. +2. PromptKit wraps both definition-loading and template/content failures in + `ErrInvalidConfig`; some materialization errors have no public source/path + type. Do not classify by parsing error strings or by that sentinel alone. + On final engine-construction failure, perform an embedded-only diagnostic + construction with the same embedded prompt/schema sources and library + defaults, excluding operator inputs and application backend mappings. If + that also fails, surface an `AssetError` with the asset cause; otherwise + return the original final-construction error as an operator/configuration + error. The diagnostic engine does no provider work and is never retained + for execution. Keep a small unexported assembly seam accepting an embedded + filesystem so tests can exercise asset failures without modifying shipped + files or creating a global test hook. +3. Do not run this diagnostic construction as a successful-path preflight. + A valid final snapshot remains accepted even if an operator contribution + repairs an otherwise invalid lower-tier prompt. Verify that property with + the same small assembly test seam. This avoids strengthening upstream + final-value validation into mandatory validation of displaced definitions. +4. Cover incomplete selected profiles using omitted, empty, and null model + contributions and a cleared selected endpoint. Exercise proposal and + validation selection, including omission-derived validation selection, with + a rejecting HTTP transport. Confirm valid unselected partial profiles are + accepted; malformed unselected definitions, undefined backend references, + and conflicting directory contributions still fail under native rules. +5. Add focused boundary coverage that invalid operator YAML/template/resource + input produces configuration exit `2`, and test the asset classification + directly at the runtime seam. Reuse existing CLI handling of `AssetError` + for exit `1`; no damaged-binary subprocess fixture is needed. Cover the + shared `NewRuntime` validation route and preserve cancellation exit `1` in + configuration commands if the new error path would otherwise classify it + as `2`. Avoid exact complete diagnostic wording assertions. + +**Verification and completion:** Run targeted startup/inspection tests and +`go test ./internal/cli`; check whitespace and commit. Preserve the original +failure cause where available rather than flattening it to a generic error. + +## Stage 5 — Protect Consumed Configuration and Message Files + +**Primary files:** `internal/framework/llm/runtime.go` or a focused source-path +helper in that package, `internal/cli/process.go`, runtime source-accounting +tests, and `cmd/audita/main_integration_test.go`. + +1. Rename the internal `Runtime.ProfilePaths` field to `SourcePaths` and update + its process caller. It now covers consumed host YAML and message resources, + rather than profiles alone. Populate it from the successfully captured final + engine; do not expose source metadata as a new public output. +2. Call `InspectConfiguration`. Index `Sources()` by source ID, and collect + paths with `Kind == ck.SourceKindFile`; embedded filesystem and synthetic + sources are not host paths. All loaded host YAML counts, including unused + definitions and files whose contributions were later overlaid. Deduplicate + paths and sort for a stable internal projection. +3. For message resources, enumerate `Metadata().HistoricalPaths()` and retain + effective, nonnull paths shaped as + `/prompts///messages//content_file`. Use `Lookup` and the + effective field's origin, not an inherited destination's guessed source. + PromptKit successfully captures these fields only with a unique usable + source context. Obtain the effective scalar from `Explain(path).Effective` + and decode that scalar YAML into a string with the existing YAML dependency. + This decodes captured inspection data; it is not a second configuration + loader. When the contributing source is a host file, join the untrimmed + resource name against that source file's directory and add it to the ledger. + Ignore displaced references and non-host resources. Do not reread operator + YAML, infer IDs from filenames, or reimplement resource-root validation. +4. Pass `SourcePaths` through existing `diagnostics.PrepareOutput` protection. + Leave canonicalization, symlink/hardlink checks, permitted in-place transcript + handling, and output initialization with their current owner. Do not protect + every unrelated file in the selected directory or add filesystem mutation + guarantees beyond the existing collision contract. +5. Extend `TestSubprocessProfileSources` and a focused runtime case to cover an + arbitrarily named YAML contribution, a consumed source-relative message + file, and conflicting transcript/report destinations. Use a non-YAML message + filename, such as `instruction.md`, so directory discovery does not mistake + plain prompt text for a configuration document. Include an inherited message + reference from a different directory and a prompt ID/version with a JSON + Pointer escape character so source provenance and literal keys determine + the path. Check rejection before provider dispatch and unchanged input bytes. + Preserve acceptance of an unused alternate output filename; do not turn the + whole directory into a forbidden output location. + +**Verification and completion:** Run focused source-ledger tests and subprocess +collision cases, plus `git diff --check`, then commit. Check that resource +accounting uses the frozen snapshot and never changes the engine's source or +resource-resolution behavior to facilitate tracking. + +## Stage 6 — Verify Operator Prompt Overrides and Captured Execution + +**Primary files:** `internal/framework/llm/adapter_test.go` and focused runtime +tests. Change `adapter.go` only for an actual integration defect. + +1. Add a native keyed operator contribution overriding messages for an active + prompt/version. Use the real runtime and controlled HTTP responses to prove + the operator text reaches both the prepared snapshot and dispatched request. + One case deliberately omits shared-variable references and still succeeds; + do not require Audita to inject their text. A reference-bearing case renders + the adapter-supplied value and retains existing protection against caller + variables replacing the reserved shared variables. +2. Override the prompt's definition-level output contract with another valid + upstream contract. Prove that Audita's per-call JSON schema and configured + repair budget still govern execution: a structurally invalid response is + repaired or exhausted as the controlled budget dictates. Domain validators + remain downstream. Reuse existing validator/proposal integration coverage + for their rules rather than weakening them to fit an operator prompt. +3. Replace the reload portion of + `TestPreparedAdapterRejectsTargetDriftAndPreparationErrors`. After successful + construction, change a profile or message file and verify the existing + runtime still uses its captured settings/text. Construct a new runtime to + observe a valid edit or reject a newly mismatched endpoint. Keep the missing + prompt preparation failure case. +4. Retain a focused test of the prepared-target check independently of reload: + deliberately change the test runtime's application backend endpoint after + construction and verify the inconsistency is blocked before dispatch. This + tests the existing guard, not a supported live mutation/reload feature. + Continue executing the same prepared handle whose details were inspected; + do not add another preparation or an application-side retry loop. + +**Verification and completion:** Run the prompt override, snapshot, and +prepared-adapter tests. Compare semantic messages and selected targets, not old +hash values or exact YAML formatting. Check whitespace and commit. + +## Stage 7 — Verify Native Profiles, Backend Authority, and Resilience + +**Primary files:** Runtime inspection and adapter tests, +`internal/framework/llm/adapter_capacity_test.go`, and existing framework/CLI +integration tests where their established behavior is affected. + +1. Add a compact native profile inheritance fixture with literal keyed IDs + unrelated to filenames, multiple definitions in one file, a partial parent, + and a complete selected child. Verify inherited controls survive a child + model change and child overrides win. Reuse the metadata-facing literal-key + coverage from Stage 5 rather than repeating it in profile tests. +2. Use controlled HTTP request bodies to distinguish explicit zero temperature + from omission and inherited-control suppression by null. Check a + representative recursively overlaid provider parameter and null clearing; + do not promise that all providers accept every valid configured zero. + Reuse existing deadline/cancellation coverage for unchanged outer deadlines; + explicit zero profile timeout must not disable the operation context. +3. Verify an operator backend contribution can add supported provider fields, + while a conflicting endpoint/concurrency/queue/environment contribution + cannot displace the final application fields. Preserve an explicit profile + environment lookup as a selected-role error rather than hiding it with an + adapter override. Include an application backend using a catalog ID to show + it is allowed and its catalog credential lookup is suppressed. Use synthetic + environment values and local HTTP fixtures, not provider catalog endpoints. +4. Reuse `TestRolesShareBackendCapacityAndReleaseAfterCancellation` and + `TestDistinctBackendsHaveIndependentCapacity`; verify the final mapped + limits, including zero queue capacity, still coordinate both roles and + remain backend-local. Strengthen a fixture with conflicting lower-tier + capacity only where needed to prove the new mapping wins. Keep existing + synchronization; do not infer admission with sleeps or add another limiter. +5. Preserve offline selected-profile inspection and value-free application + explanations. Check configured secrets, endpoint strings, arbitrary provider + fields, and prompt text remain outside the current safe profile summary. + Do not replace it with a full upstream dump. Its numeric projection is not + proof that an omitted provider setting is sent; wire presence belongs to + the HTTP coverage above. +6. Run the existing transient retry, structural repair, attempt-count, + observation, partial-validation, and all-model-failure subprocess cases + after fixture migration. Preserve their error/recovery/debug classifications, + cancellation, and continued output. Fix proven integration regressions; + do not rewrite domain outcomes or reduce budgets to obtain a passing suite. **Verification and completion:** Run -`go test ./internal/core/config ./internal/cli ./internal/framework/llm` and -`git diff --check`. Review the dependency and attribution diff, then commit. -This establishes the upgraded baseline; it does not establish feature completion -or replace the final suite in Stage 5. +`go test ./internal/framework/llm ./internal/framework/proposal_generation ./internal/framework/validators ./internal/framework/runner ./internal/cli ./cmd/audita` +and `git diff --check`. Review whether each new test addresses a distinct changed +boundary. Commit the native-profile and runtime verification unit. -## Stage 2 — Verify Inherited Configuration, Defaults, and Credentials +## Stage 8 — Document the Migration and Update Attribution -**Completion:** Complete. +**Primary files:** `docs/config.md`, `docs/internal/llm-runtime.md`, +`docs/internal/prompts.md`, `docs/integrations/openai-compatible-llm.md`, +`docs/operations.md`, `docs/internal/cli-config.md`, `README.md`, and +`docs/oss/attribution.md`. -**Primary files:** `internal/core/config/config_test.go`; change -`internal/core/config/config.go` only if these checks expose an integration defect. +1. Update the configuration contract for unified `profiles_dir` content, exact + keyed identities, directory composition, agreed tier precedence, profile + inheritance/presence, selected-role completeness, eager validation of all + final definitions, selected-root failures, and construction-time capture. + Replace the obsolete statement that PromptKit profiles do not gain ConfigKit + inheritance. Retain application overlays, defaults, and credential ownership. +2. Include concise migration instructions in the profile section: wrap old + profile fields beneath `profiles/`, remove inner `id`, change + `base_profile` to `extends`, review explicit values/nulls, and select roots + whose final definitions have valid backend references. Show a small keyed + inheritance/override snippet and link complete maintained examples. Explain + explicit zero temperature versus omitted or null controls and link upstream + for the full field table. State that this ships as an Audita major migration; + do not invent an exact release version or automatic file conversion. +3. Document intentional prompt overrides and optional shared-variable references. + Supplying shared variables does not inject omitted text, while per-call + schemas and domain validators remain enforced. Internal prompt documentation + distinguishes shipped asset instructions from operator-effective messages; + no prompt-quality rewrite or new global prompt-policy requirement is added. +4. Describe layered runtime assembly, source accounting, snapshot use, and + startup classification in their internal owner. Update the pinned PromptKit + consumer link in the LLM integration contract. Expand operations' loaded + source description to include consumed operator message files; keep physical + output/collision rules with operations and diagnostics. +5. Repair example navigation for separate local/service roots, especially the + README's local-profile link. Update validation recipes to use the maintained + config files' explicit roots and keep the service testing overlay workflow. + Review other affected links without rewriting historical release notes. +6. Update the PromptKit and both catalog attribution entries to selected versions + and exact commit-pinned source/notice URLs. Resolve catalog tag commits from + the selected module origins or authoritative upstream refs, not annotated + tag object IDs. Check the releases' COPYRIGHT/LICENSE files and retain the + applicable unchanged local LGPL/GPL and MIT texts. Keep the existing House + Format; no new license selection, policy rewrite, or attribution inventory + is needed. Review `go.mod`/`go.sum` against those entries. -1. Read `Load`, its resolver, and domain validation. Existing backend capacity - defaults use `Resolution.IsExplicit`; omitted validation-role selection uses - `ck.Default` against the final proposal selection. Keep those mechanisms if - they already handle inherited presence correctly. -2. Add a focused real-loader fixture with valid parent and child backends. The - child inherits an endpoint, concurrency, zero queue capacity, and a synthetic - credential reference. Check effective values and inherited explicitness; - in another case, let the child override concurrency. This protects against - replacing inherited capacity with Audita's absent-field defaults. Reuse - current tests for ordinary omitted defaults. -3. Exercise precedence with a selected overlay that changes a parent's setting - and a child override that wins over its resolved parent. Preserve coverage - that an explicit proposal-profile override wins over the overlay and omitted - validation-profile selection derives from that final value. Use the existing - `TestOverlayOverrideAndInheritance` fixture where useful; its current - "inheritance" check covers role derivation, not native sibling inheritance. -4. Extend credential coverage using controlled environment variables. Check that - an inherited reference resolves and that effective YAML excludes its resolved - value. A child's `credential: null` removes only that child's reference; - the parent's surviving reference still must resolve, even if the parent is - unselected. Check that an unset surviving reference fails and a missing - reference removed by final precedence is not resolved. Reuse - `TestSurvivingCredentials` rather than duplicating all secret cases. -5. Include a representative inherited setting that violates existing Audita - validation, such as zero backend concurrency. Inheritance must not bypass - validation of either the parent or child. Existing explicit-empty/null role - validation cases remain applicable; extend them only if inherited presence - exposes an uncovered interaction. -6. If a regression appears, correct the owning default or resolution integration - using ConfigKit presence/path APIs. Do not parse raw YAML, exempt parent - backends, or introduce abstract backend templates. Do not reproduce upstream - cycle, nesting, or expansion-limit tests. +**Verification and completion:** Check affected local links/anchors, format +examples against the implemented loader, verify source revisions/notices, and +run `git diff --check`. Commit the documentation and attribution unit. -**Verification and completion:** Run `go test ./internal/core/config` and -`git diff --check`. Review assertions for new validation rules or duplicated -library tests, then commit the focused configuration unit. +## Stage 9 — Validate the Integrated End State and Close the Plan -## Stage 3 — Expose Compact Inheritance Evidence in Safe Inspection +**Primary files:** This plan and the feature roadmap; correct an implementation +file only if the final checks reveal an actual gap. -**Completion:** Complete. +1. Verify the full roadmap acceptance criteria against the landed code and the + distinct tests from these stages. Review for unintended requirements: + profile-only filtering, mandatory shared text in operator messages, + filename-derived IDs, complete unselected profiles, catalog ID bans, + successful-path asset preflights, compatibility readers, and new output + guarantees are not part of the accepted target. +2. Run `go build -o ./bin/audita ./cmd/audita` and `go test ./...`. Keep the + generated binary out of commits. Reuse successful checks on unchanged + content; rerun checks affected by any corrective code changes. +3. Validate the maintained examples offline using the built binary: -**Primary files:** `internal/framework/llm/safe_view.go`, -`internal/framework/llm/runtime_inspection_test.go`, `docs/cli.md`, and the -relevant explanation description in `docs/internal/llm-runtime.md` if needed. + ```sh + ./bin/audita config validate --config examples/minimal-config.yml + ./bin/audita config validate --config examples/configkit-inheritance.yml + AUDITA_SERVICE_KEY=synthetic-example ./bin/audita config validate --config examples/production-config.yml + AUDITA_SERVICE_KEY=synthetic-example ./bin/audita config validate --config examples/production-config.yml --overlay testing + ./bin/audita config print-effective --config examples/minimal-config.yml --view profiles + ./bin/audita config print-effective --config examples/configkit-inheritance.yml --view explain + ``` -1. Extend `explanationView` with `inheritance,omitempty` and a small step view - containing `from`, `to`, and `reference`. For effective field records, project - `info.Origin.Inheritance` in its upstream order. Preserve existing `path`, - `source`, `layer`, and application `derivations` behavior. Do not serialize - field values, whole `Explanation` objects, resolved credentials, or arbitrary - configuration content into this view. -2. Continue to use `Metadata.Lookup`, `HistoricalPaths`, and existing derivation - mapping. Do not infer ancestry from labels or YAML, flatten a chain to one - guessed parent, or reinterpret inheritance as an application-derived default. - This addition describes effective inheritance; complete displaced ancestry - and upstream source-coordinate reporting are not new requirements. -3. Test decoded YAML for one inherited field's source/destination/reference paths - and a child field that directly overrides its parent. Include an inherited - credential reference with a distinctive synthetic resolved value and prove - that value is absent. Retain existing application-derived role explanations - and safe profile-view coverage. Assert meaningful records, not complete - YAML snapshots or upstream-generated identifiers. -4. Check the existing shared callers: CLI explain uses `ExplanationYAML`, and - `processState.startDebug` uses it for configuration origins. Keep this shared - path; no separate debug projection is needed. Update the CLI contract with - the optional inheritance step fields and their value-free meaning once they - are implemented. Link internal documentation to that contract for exact shape. + Confirm local/service roots and testing selections work without provider + dispatch. Check safe views exclude resolved secrets and preserve the + established explanation fields. Do not call a model to validate examples. +4. Check the selected dependency graph, attribution, affected links, and final + diff. Run `git diff --check`. Confirm only the released modules are used and + no obsolete `/v2` catalog or removed construction API remains in active code. + Historical release documents and historical roadmap links may retain old + versions as accurate records. +5. Mark this plan and the feature roadmap complete only after the full target + passes. Record the tested implementation revision and checks accurately; + distinguish a subsequent documentation-only completion commit. Commit the + completion documentation and report remaining work, if any. Do not select + a release number, publish, tag, push, poll CI, or add permanent release gates. -**Verification and completion:** Run -`go test ./internal/framework/llm ./internal/cli` and `git diff --check`. -Review emitted fields and affected documentation, then commit the inspection unit. - -## Stage 4 — Verify CLI Failures and Runtime Use of Inherited Backends - -**Completion:** Complete. - -**Primary files:** `internal/framework/llm/adapter_capacity_test.go`, -`internal/cli/config_input_test.go`, and -`cmd/audita/main_integration_test.go`. Touch production wiring only to fix a -proven integration defect. - -1. Reuse `TestRolesShareBackendCapacityAndReleaseAfterCancellation` or its - existing fixture to load a valid parent and a selected child backend through - `config.Load`. Both existing-format role profiles select that child. Inherit - the controlled server endpoint, a test-selected positive capacity, zero - queue capacity, and a synthetic credential; keep the parent valid too. -2. Verify actual requests reach the inherited endpoint and carry the inherited - credential. Existing overlapping role-call coverage proves that the two - roles share the child's capacity. Preserve that check with inherited input - and its release/cancellation coverage; do not build another scheduler or - duplicate the provider retry/repair matrix. Use synchronization already in - the test, not sleeps to infer admission behavior. -3. Add one representative invalid relationship at the CLI boundary: a missing - sibling parent should yield configuration exit code `2` and no provider - dispatch. Use the existing subprocess fixture and semantic diagnostic - assertions; do not freeze full error text or invent early-failure reports. -4. Extend one representative subprocess success/inspection case to use inherited - backend input. With debug explicitly enabled, check that effective settings - and configuration-origin artifacts retain inherited settings and the new - explanation evidence without the resolved synthetic secret. Check the - explain view through the existing CLI route. Use existing coverage for - unchanged report placement, ordinary processing, and debug-disabled behavior. - Keep assertions on the affected fields rather than the entire artifact set. - -**Verification and completion:** Run -`go test ./internal/framework/llm ./internal/cli ./cmd/audita` and -`git diff --check`. Review whether each added test covers a distinct integration -risk, then commit the runtime/CLI verification unit. - -## Stage 5 — Document Native Inheritance and Close the Adoption Unit - -**Completion:** Complete. - -**Primary files:** `docs/config.md`, `examples/configkit-inheritance.yml`, -`docs/internal/cli-config.md`, and `docs/roadmap/configkit.md`. Update another -canonical document only where an implemented statement actually changes. - -1. Add `examples/configkit-inheritance.yml` beside the existing examples, with - `version: 2`, a valid `local-base` backend, and a valid `local` backend that - extends it. Inherit a local endpoint and zero queue capacity; use a positive - parent concurrency and a child concurrency override. Select the existing - `local-proposal` and `local-validation` profiles under `profiles/` and the - grammar module. The example needs no credential or new profile format and - can be validated offline without a running local server. -2. Update the configuration contract to describe native sibling inheritance, - precedence before inheritance, inherited explicit presence, null credential - removal, and normal parent validation. Link the maintained example and the - version-pinned upstream generic grammar. Retain current file/overlay/override - behavior and main-file-relative path rules. Do not imply that ordinary - PromptKit v0.11.2 profile files now accept ConfigKit `extends` syntax. -3. Document that `extends` is reserved across input mappings. Explain the legacy - backend-name conflict and migration: rename the backend to a non-conflicting - ID and update every profile selecting it. Keep application `version: 2`; - do not claim all previously valid configurations are compatible. Record the - compatibility finding in the feature roadmap for later release preparation. - Release classification remains outside this implementation unit, governed by - the existing release policy; no policy exception is selected here. -4. Add the new example's offline validation command to the existing example - recipe. Validate it and the maintained minimal, production, and selected - testing-overlay examples, with a controlled synthetic service credential. - Inspect the new example's settings and explanation output. No live model - request is needed to check configuration or profiles. -5. Run `go build -o ./bin/audita ./cmd/audita`, `go test ./...`, affected-link - checks, and `git diff --check`. Review the final dependency graph and diff: - ConfigKit is the released v0.5.0, PromptKit/catalogs retain their selections, - source attribution is pinned, and no unrelated settings or limits were added. - Keep generated binaries out of the commit. Resolve actual failures rather - than adding gates unrelated to this feature. -6. Mark the feature roadmap and this plan complete only after all stage outcomes - pass. Record the tested source revision and checks accurately; if a final - documentation-only completion commit follows validation, distinguish it from - the tested implementation revision. Review and commit the final documentation - unit, leaving a clean workspace. Reuse successful checks for unchanged content. - -**End state:** Native ConfigKit inheritance reaches typed settings, safe -inspection, selected runtime backends, and maintained operator examples; -PromptKit migration remains a separate unit. No implementation-blocking design -questions remain. The eventual release version and treatment of the known -compatibility impact remain release-preparation decisions, not prerequisites -for these stages. +**End state:** Native PromptKit configuration and catalogs are adopted, +operator overrides and captured resources work through the real runtime, +application backend/domain ownership and resilience remain effective, and +maintained examples and migration documentation match the implementation. +No implementation-blocking decisions remain; exact major-release numbering +and publication timing remain outside this implementation unit. diff --git a/docs/roadmap/promptkit.md b/docs/roadmap/promptkit.md index 96d2e89..4540482 100644 --- a/docs/roadmap/promptkit.md +++ b/docs/roadmap/promptkit.md @@ -2,9 +2,8 @@ **Last updated:** 2026-10-03 -**Status:** Accepted target scope; ready for implementation planning. The -operator-directory scope decision has been resolved. Implementation has not -begun. +**Status:** Accepted target scope; implementation is defined in the +[implementation plan](implementation.md). Implementation has not begun. ## Purpose and Baseline @@ -263,4 +262,5 @@ No live provider calls, new release gates, or CI-success dependency are needed to establish adoption. Guided domain recovery and the other work recorded in [future.md](future.md) remain separate features. Do not expand this upgrade into new report formats, pipeline scheduling, or general configuration-inspection -commands. A staged implementation plan can now be prepared. +commands. The [implementation plan](implementation.md) owns execution steps +and their verification.