Plan the PromptKit upgrade in nine stages
This commit is contained in:
@@ -1,246 +1,460 @@
|
||||
# Implementation Plan: ConfigKit v0.5.0 Adoption
|
||||
# Implementation Plan: PromptKit v0.12.0 Adoption
|
||||
|
||||
**Last updated:** 2026-10-03
|
||||
**Status:** Complete.
|
||||
**Status:** Ready for implementation; no stages completed.
|
||||
**Audience:** Luna coding agent.
|
||||
|
||||
## Scope and Execution
|
||||
|
||||
Implement the target state in the [ConfigKit roadmap](configkit.md), which owns
|
||||
feature intent, scope, and compatibility considerations. Execute the numbered
|
||||
stages below in order. Each stage is a bounded implementation prompt; the
|
||||
complete feature comprises all five stages. Commit each completed unit with a
|
||||
short, plain English message under the [agent policy](../policy/agents.md).
|
||||
No approval gate is needed between stages.
|
||||
Implement the accepted target in the [PromptKit roadmap](promptkit.md). That
|
||||
document owns feature intent and policy choices; this plan owns the ordered
|
||||
implementation work. The planning baseline is Audita commit
|
||||
`b31955fc97adec707b4e5f9a1b0c43c4cbc66750`: ConfigKit v0.5.0 is adopted,
|
||||
PromptKit is v0.11.2, and the finalized roadmap includes the selected-profile,
|
||||
operator-override, example-layout, and major-release clarifications.
|
||||
|
||||
Begin with [development guidance](../development.md) and the applicable agent,
|
||||
This replaces the completed ConfigKit plan at this path. Its
|
||||
[completed implementation record](https://gitea.maximumdirect.net/eric/audita/src/commit/b31955fc97adec707b4e5f9a1b0c43c4cbc66750/docs/roadmap/implementation.md)
|
||||
remains available for readers following the ConfigKit completion roadmap.
|
||||
|
||||
Execute Stage 1 through Stage 9 in order. Each is sized as one bounded Luna
|
||||
implementation prompt. Commit completed units with short, plain English messages
|
||||
under the [agent policy](../policy/agents.md); no approval round is needed between
|
||||
stages. Record actual checks and any remaining expected migration failures.
|
||||
Stages 1 and 2 change formats before Stage 3 changes their consumer. Intermediate
|
||||
commits need not be runnable or release-ready. Do not add temporary readers,
|
||||
compatibility shims, fallback engines for execution, or scaffolding to make them
|
||||
so. Later stages complete the same migration rather than separate product modes.
|
||||
|
||||
Start with [development guidance](../development.md) and the applicable agent,
|
||||
[architecture](../policy/architecture.md), [testing](../policy/testing.md),
|
||||
[documentation](../policy/documentation.md), and [OSS](../policy/oss.md)
|
||||
policies. Use graph-first discovery and read the relevant current contracts.
|
||||
The planning baseline is Audita commit
|
||||
`23e1df4cfb844d3fdeb1fb64483c1038cc4f2e21`; application code remains at the
|
||||
roadmap's reviewed revision. Target released ConfigKit v0.5.0, source commit
|
||||
`dc09959cca4e105f49ec316c6d4ba585bd2340bd`.
|
||||
policies. Use graph-first discovery. Read the current
|
||||
[configuration](../config.md), [LLM runtime](../internal/llm-runtime.md),
|
||||
[prompt registry](../internal/prompts.md), and
|
||||
[LLM integration](../integrations/openai-compatible-llm.md) contracts for the
|
||||
established behavior this migration preserves.
|
||||
|
||||
Target the released PromptKit v0.12.0 at commit
|
||||
`4b8169da286917fde7875425bb32b799b12c6af3`, ConfigKit v0.5.0, and both
|
||||
maintained `/v3` catalogs at `v3.20260930.0`. Read the pinned
|
||||
[format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.12.0/docs/formats.md)
|
||||
and [migration guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.12.0/docs/consumers/pkg-promptkit.md#migrate-from-the-previous-configuration-api).
|
||||
The sibling checkout is reference material, not a dependency replacement.
|
||||
|
||||
### Selected Implementation Decisions
|
||||
|
||||
- Keep `internal/core/config.Load` using ordinary `ck.Load`. Native inheritance
|
||||
already runs before decoding and application resolution; another resolver or
|
||||
`LoadLayered` conversion is unnecessary.
|
||||
- Keep PromptKit at v0.11.2 and its `/v2` catalogs at their existing versions.
|
||||
Retain Go 1.25.5, configuration `version: 2`, existing profile syntax, role
|
||||
selection, CLI flags, file discovery, overlays, and main-file-relative paths.
|
||||
- Preserve absence-based defaults. Inherited input is explicit input, including
|
||||
zero, false, empty, and null; existing field rules still decide validity.
|
||||
Every backend, including inheritance parents and unselected backends, remains
|
||||
subject to ordinary validation and surviving credential resolution.
|
||||
- Preserve existing explanation fields and add an optional `inheritance`
|
||||
sequence to each field record. Each step has `from`, `to`, and `reference`
|
||||
string paths copied from `Origin.Inheritance` through `Path.String()`.
|
||||
This additive, value-free projection serves both `--view explain` and debug
|
||||
configuration origins through their existing shared function. It is not a
|
||||
new inspection command or a complete upstream metadata serialization.
|
||||
- The `extends` keyword is reserved by the library throughout application input
|
||||
mappings. Document its compatibility impact, including legacy backend IDs
|
||||
named `extends`; do not add a compatibility translator or a second naming
|
||||
restriction to application validation.
|
||||
- Keep Go 1.25.5, application `version: 2`, current flags, role selection,
|
||||
application overlays, and main-file-relative path rules. Retain the
|
||||
`profiles_dir` name and default; broaden its accepted documents as agreed.
|
||||
- Use PromptKit's catalog and owned defaults, then caller underlays for embedded
|
||||
Audita prompts and operator directory inputs, then `Sources.Main` for
|
||||
application backend settings. Directory inputs compose strictly within their
|
||||
group; discovery order supplies no field precedence. Keep imports disabled.
|
||||
- Use one ConfigKit mapping for the final application source. For each
|
||||
application backend, supply its endpoint, `concurrency_limit`,
|
||||
`queue_capacity`, and explicit `api_key_env: null`. The null suppresses
|
||||
lower-tier credential lookup. Leave other supported provider fields available
|
||||
to native layering. Do not put resolved credential values in this mapping.
|
||||
- Keep selected-role registration, endpoint-match, competing-credential, and
|
||||
required-key checks. Add explicit nonblank effective model and endpoint checks
|
||||
for both selected roles. Valid unselected partial profiles remain permitted.
|
||||
- Preserve the adapter's complete per-call schemas, configured repairs/retries,
|
||||
operation context, shared variables, observation, and prepare/check/run flow.
|
||||
Operator messages can omit shared-variable references; do not append the
|
||||
shared text unconditionally or require particular message counts or roles
|
||||
beyond PromptKit's own format rules.
|
||||
- Use captured source inspection for consumed host configuration and message
|
||||
paths. Keep source-relative resource resolution with PromptKit and ConfigKit;
|
||||
Audita projects paths for its existing output collision checks.
|
||||
- Replace the old profile-reload test expectation with snapshot behavior.
|
||||
Successful construction uses only final effective definitions. A failed
|
||||
construction may make an embedded-only diagnostic construction to distinguish
|
||||
asset defects from operator errors; it never executes or substitutes for the
|
||||
requested configuration.
|
||||
- Maintained example roots are `examples/profiles/local/` and
|
||||
`examples/profiles/service/`. Service testing profiles stay in the service
|
||||
root. Keep the literal `temperature: 0` in examples as an explicit outbound
|
||||
setting, and use native presence semantics without a compatibility translator.
|
||||
|
||||
Follow existing ownership boundaries. Production changes beyond the dependency
|
||||
and explanation projection are warranted only if focused integration checks
|
||||
expose a defect in the agreed behavior. Do not change report schemas, provider
|
||||
policy, debug enablement, or application validation to accommodate test fixtures.
|
||||
Do not add live-provider checks or permanent release gates. This plan does not
|
||||
select a release version, modify release policy, publish, tag, or push.
|
||||
This unit is an intentional major-release migration. Exact version selection,
|
||||
release notes, tag/push, and publication timing remain release preparation under
|
||||
the [release policy](../policy/release.md). Guided recovery, new report formats,
|
||||
hot reload, extra configuration commands, and scheduling changes remain outside
|
||||
this plan. Tests should protect changed integration behavior and established
|
||||
contracts, not reproduce the libraries' entire validation matrices.
|
||||
|
||||
## Stage 1 — Upgrade the Dependency and Its Attribution
|
||||
## Stage 1 — Convert Embedded Prompt Definitions
|
||||
|
||||
**Completion:** Complete.
|
||||
**Primary files:** The seven YAML files under
|
||||
`internal/prompts/assets/prompts/`.
|
||||
|
||||
**Primary files:** `go.mod`, `go.sum`, `docs/oss/attribution.md`.
|
||||
1. Wrap each existing definition beneath `prompts`, its exact current ID, and
|
||||
the string version `v1`. Remove the old inner `id` and `version` fields.
|
||||
Keep filenames and Audita registry identifiers unchanged; filenames are
|
||||
asset locations rather than PromptKit identities.
|
||||
2. Preserve messages, block-scalar text, variable references, task examples,
|
||||
output settings, and schema paths. Preserve the applicable references to
|
||||
`HardeningPolicy`, `ProposalContract`, and `ValidationContract`. Do not edit
|
||||
shared fragments, response schemas, or domain policies to accommodate the
|
||||
new container format.
|
||||
3. Review each before/after definition structurally, including YAML indentation
|
||||
inside block scalars. Do not introduce default profile selections or new
|
||||
repair counts. The adapter continues to own the runtime output contract.
|
||||
|
||||
1. Review the current dependency declarations and upgrade only ConfigKit:
|
||||
`go get gitea.maximumdirect.net/eric/ConfigKit@v0.5.0`. Review the resolved
|
||||
module diff; preserve PromptKit, both catalog modules, the Go minimum, and
|
||||
unrelated dependency selections. Do not add a local replacement.
|
||||
2. Check the selected release's `COPYRIGHT`, `LICENSE`, and `go.mod` against the
|
||||
pinned upstream revision. Update the ConfigKit attribution entry's version
|
||||
and commit-pinned source and notice links. Preserve its verified copyright,
|
||||
LGPL-3.0-or-later expression, and existing LGPL/GPL text links when unchanged.
|
||||
Follow the existing House Format; no additional licensing machinery is needed.
|
||||
3. Leave the loader and profile integration intact. If the dependency introduces
|
||||
a compilation failure, make only the API adjustment needed for this upgrade,
|
||||
recording the actual difference rather than broadening scope.
|
||||
**Verification and completion:** Check all seven migrated identities and
|
||||
version keys against `RegisteredMetadata`, and run `git diff --check`. Registry
|
||||
preparation tests become runnable with the new constructor in Stage 3. Commit
|
||||
the asset conversion; do not introduce an old-format reader to test it here.
|
||||
|
||||
## Stage 2 — Migrate Operator Profiles, Example Roots, and YAML Fixtures
|
||||
|
||||
**Primary files:** `examples/profiles/`, the three maintained configuration
|
||||
examples, and profile-producing test fixtures in
|
||||
`internal/framework/llm/{runtime_inspection,adapter,adapter_capacity}_test.go`,
|
||||
`internal/framework/proposal_generation/adapter_integration_test.go`,
|
||||
`internal/framework/validators/llm_integration_test.go`,
|
||||
`internal/framework/runner/cancellation_test.go`, and
|
||||
`cmd/audita/main_integration_test.go`.
|
||||
|
||||
1. Move local proposal/validation examples beneath `profiles/local/`; move the
|
||||
four service proposal/validation and testing files beneath
|
||||
`profiles/service/`. Convert each to `profiles: {<ID>: <fields>}` in readable
|
||||
YAML. Keep IDs, models, positive timeouts, and controls, including explicit
|
||||
zero temperature. No example needs a real credential or live model.
|
||||
2. Set `profiles_dir: profiles/local` in `examples/minimal-config.yml` and
|
||||
`examples/configkit-inheritance.yml`; set `profiles_dir: profiles/service`
|
||||
in `examples/production-config.yml`. Preserve the testing overlay's role IDs
|
||||
and the existing ConfigKit inheritance example. Select the leaf root, since
|
||||
recursive discovery at their common parent would still load both groups.
|
||||
3. Convert profile YAML emitted by the listed fixtures to keyed definitions.
|
||||
Search for remaining old profile construction strings and removed
|
||||
`base_profile` syntax, and migrate any genuine profile fixture found.
|
||||
Do not rewrite transcript/proposal `id` fields or domain schema fixtures.
|
||||
Keep test source names independent of profile IDs where already exercised.
|
||||
4. Keep negative tests negative for their selected behavioral reason. Give
|
||||
cases clean temporary layouts or replace their contribution completely
|
||||
rather than leaving an unrelated invalid definition behind for later cases.
|
||||
Eager validation now reads unselected final definitions too. Do not remove
|
||||
such definitions merely to simulate the old lazy loader where the test is
|
||||
intended to exercise eager validation.
|
||||
|
||||
**Verification and completion:** Review the six profiles, explicit root
|
||||
selections, and changed fixture text; run `git diff --check` and format touched
|
||||
Go test files. These formats become consumable in Stage 3. Commit this unit.
|
||||
Current documentation links and recipes are updated together in Stage 8.
|
||||
|
||||
## Stage 3 — Adopt the Released Source API and Selected-Role Checks
|
||||
|
||||
**Primary files:** `go.mod`, `go.sum`,
|
||||
`internal/framework/llm/runtime.go`, and `internal/prompts/registry_test.go`.
|
||||
|
||||
1. Upgrade with `go get gitea.maximumdirect.net/eric/promptkit@v0.12.0` and
|
||||
reconcile the module files. Verify ConfigKit stays v0.5.0, the catalog
|
||||
dependencies become `/v3` at `v3.20260930.0`, and obsolete `/v2` requirements
|
||||
are removed when unused. Retain the Go minimum and unrelated selections;
|
||||
introduce no local replacements.
|
||||
2. Resolve `profiles_dir` exactly as the current runtime does. Obtain embedded
|
||||
inputs with `pk.FilesystemInputs` and operator inputs with `pk.DirectoryInputs`.
|
||||
An unavailable selected root is an operator error; an empty directory can
|
||||
contribute no definitions. Do not require one profile per file, IDs matching
|
||||
filenames, or profile-only top-level content.
|
||||
3. Construct the agreed `pk.ConfigurationSources`: two ordered caller underlays
|
||||
for embedded and operator groups, and one highest-priority application
|
||||
`ck.Input` whose source is the mapping specified above. Retain
|
||||
`pk.WithSchemaFS`, the supplied HTTP client, and configured transient retries.
|
||||
Remove old prompt/profile/backend registration options.
|
||||
4. Inspect registered prompt ID/version selections, then both selected roles.
|
||||
Reject a selected role whose effective model or endpoint is empty or
|
||||
whitespace-only. Retain backend registration, normalized endpoint equality,
|
||||
nonblank effective `APIKeyEnv` rejection, and required-credential checks.
|
||||
Preserve literal IDs; do not trim/re-key identities or require all profiles
|
||||
to be complete. Populate the same role inspection map for both clients.
|
||||
5. Migrate the registry test engine to `FilesystemInputs` and a programmatic
|
||||
`ProfileSource` fixture in its own tier, retaining its schema filesystem.
|
||||
A complete endpoint/model fixture without an application backend is valid
|
||||
for this direct library test; Audita's selected-role guard belongs in its
|
||||
runtime tests. Keep the embedded prompt preparation assertions scoped to
|
||||
embedded prompts.
|
||||
6. Remove the old `profileSourceFS` wrapper once unused. Source accounting is
|
||||
completed in Stage 5. Preserve the adapter boundary; change other production
|
||||
callers only to resolve actual API incompatibilities. Search for remaining
|
||||
uses of removed source options and old `pk.Profile`/`pk.Backend` values.
|
||||
Run `go mod tidy` after migrating construction callers and review its diff.
|
||||
|
||||
**Verification and completion:** Run `go test ./internal/prompts` and the
|
||||
existing offline profile assembly, unsafe-selection, and absolute-directory
|
||||
tests in `internal/framework/llm`. Run `git diff --check` and review the module
|
||||
diff. The obsolete file-reload assertion is intentionally replaced in Stage 6;
|
||||
do not restore reload behavior to make that assertion pass. Commit the runtime
|
||||
API migration.
|
||||
|
||||
## Stage 4 — Preserve Startup Failure Classification and Validate Assembly
|
||||
|
||||
**Primary files:** `internal/framework/llm/runtime.go`,
|
||||
`internal/framework/llm/runtime_inspection_test.go`, and configuration-command
|
||||
tests in `internal/cli`. Update `internal/cli/run.go` only where the changed
|
||||
startup path needs existing exit behavior preserved.
|
||||
|
||||
1. Treat embedded discovery failures as `AssetError`; treat operator-directory
|
||||
discovery failures as configuration input errors. Preserve wrapped causes
|
||||
for `errors.Is`/`errors.As` and retain configured-secret redaction at CLI
|
||||
boundaries. Caller cancellation remains a runtime exit, not an operator
|
||||
mistake, and should not trigger further diagnostic loading.
|
||||
2. PromptKit wraps both definition-loading and template/content failures in
|
||||
`ErrInvalidConfig`; some materialization errors have no public source/path
|
||||
type. Do not classify by parsing error strings or by that sentinel alone.
|
||||
On final engine-construction failure, perform an embedded-only diagnostic
|
||||
construction with the same embedded prompt/schema sources and library
|
||||
defaults, excluding operator inputs and application backend mappings. If
|
||||
that also fails, surface an `AssetError` with the asset cause; otherwise
|
||||
return the original final-construction error as an operator/configuration
|
||||
error. The diagnostic engine does no provider work and is never retained
|
||||
for execution. Keep a small unexported assembly seam accepting an embedded
|
||||
filesystem so tests can exercise asset failures without modifying shipped
|
||||
files or creating a global test hook.
|
||||
3. Do not run this diagnostic construction as a successful-path preflight.
|
||||
A valid final snapshot remains accepted even if an operator contribution
|
||||
repairs an otherwise invalid lower-tier prompt. Verify that property with
|
||||
the same small assembly test seam. This avoids strengthening upstream
|
||||
final-value validation into mandatory validation of displaced definitions.
|
||||
4. Cover incomplete selected profiles using omitted, empty, and null model
|
||||
contributions and a cleared selected endpoint. Exercise proposal and
|
||||
validation selection, including omission-derived validation selection, with
|
||||
a rejecting HTTP transport. Confirm valid unselected partial profiles are
|
||||
accepted; malformed unselected definitions, undefined backend references,
|
||||
and conflicting directory contributions still fail under native rules.
|
||||
5. Add focused boundary coverage that invalid operator YAML/template/resource
|
||||
input produces configuration exit `2`, and test the asset classification
|
||||
directly at the runtime seam. Reuse existing CLI handling of `AssetError`
|
||||
for exit `1`; no damaged-binary subprocess fixture is needed. Cover the
|
||||
shared `NewRuntime` validation route and preserve cancellation exit `1` in
|
||||
configuration commands if the new error path would otherwise classify it
|
||||
as `2`. Avoid exact complete diagnostic wording assertions.
|
||||
|
||||
**Verification and completion:** Run targeted startup/inspection tests and
|
||||
`go test ./internal/cli`; check whitespace and commit. Preserve the original
|
||||
failure cause where available rather than flattening it to a generic error.
|
||||
|
||||
## Stage 5 — Protect Consumed Configuration and Message Files
|
||||
|
||||
**Primary files:** `internal/framework/llm/runtime.go` or a focused source-path
|
||||
helper in that package, `internal/cli/process.go`, runtime source-accounting
|
||||
tests, and `cmd/audita/main_integration_test.go`.
|
||||
|
||||
1. Rename the internal `Runtime.ProfilePaths` field to `SourcePaths` and update
|
||||
its process caller. It now covers consumed host YAML and message resources,
|
||||
rather than profiles alone. Populate it from the successfully captured final
|
||||
engine; do not expose source metadata as a new public output.
|
||||
2. Call `InspectConfiguration`. Index `Sources()` by source ID, and collect
|
||||
paths with `Kind == ck.SourceKindFile`; embedded filesystem and synthetic
|
||||
sources are not host paths. All loaded host YAML counts, including unused
|
||||
definitions and files whose contributions were later overlaid. Deduplicate
|
||||
paths and sort for a stable internal projection.
|
||||
3. For message resources, enumerate `Metadata().HistoricalPaths()` and retain
|
||||
effective, nonnull paths shaped as
|
||||
`/prompts/<id>/<version>/messages/<index>/content_file`. Use `Lookup` and the
|
||||
effective field's origin, not an inherited destination's guessed source.
|
||||
PromptKit successfully captures these fields only with a unique usable
|
||||
source context. Obtain the effective scalar from `Explain(path).Effective`
|
||||
and decode that scalar YAML into a string with the existing YAML dependency.
|
||||
This decodes captured inspection data; it is not a second configuration
|
||||
loader. When the contributing source is a host file, join the untrimmed
|
||||
resource name against that source file's directory and add it to the ledger.
|
||||
Ignore displaced references and non-host resources. Do not reread operator
|
||||
YAML, infer IDs from filenames, or reimplement resource-root validation.
|
||||
4. Pass `SourcePaths` through existing `diagnostics.PrepareOutput` protection.
|
||||
Leave canonicalization, symlink/hardlink checks, permitted in-place transcript
|
||||
handling, and output initialization with their current owner. Do not protect
|
||||
every unrelated file in the selected directory or add filesystem mutation
|
||||
guarantees beyond the existing collision contract.
|
||||
5. Extend `TestSubprocessProfileSources` and a focused runtime case to cover an
|
||||
arbitrarily named YAML contribution, a consumed source-relative message
|
||||
file, and conflicting transcript/report destinations. Use a non-YAML message
|
||||
filename, such as `instruction.md`, so directory discovery does not mistake
|
||||
plain prompt text for a configuration document. Include an inherited message
|
||||
reference from a different directory and a prompt ID/version with a JSON
|
||||
Pointer escape character so source provenance and literal keys determine
|
||||
the path. Check rejection before provider dispatch and unchanged input bytes.
|
||||
Preserve acceptance of an unused alternate output filename; do not turn the
|
||||
whole directory into a forbidden output location.
|
||||
|
||||
**Verification and completion:** Run focused source-ledger tests and subprocess
|
||||
collision cases, plus `git diff --check`, then commit. Check that resource
|
||||
accounting uses the frozen snapshot and never changes the engine's source or
|
||||
resource-resolution behavior to facilitate tracking.
|
||||
|
||||
## Stage 6 — Verify Operator Prompt Overrides and Captured Execution
|
||||
|
||||
**Primary files:** `internal/framework/llm/adapter_test.go` and focused runtime
|
||||
tests. Change `adapter.go` only for an actual integration defect.
|
||||
|
||||
1. Add a native keyed operator contribution overriding messages for an active
|
||||
prompt/version. Use the real runtime and controlled HTTP responses to prove
|
||||
the operator text reaches both the prepared snapshot and dispatched request.
|
||||
One case deliberately omits shared-variable references and still succeeds;
|
||||
do not require Audita to inject their text. A reference-bearing case renders
|
||||
the adapter-supplied value and retains existing protection against caller
|
||||
variables replacing the reserved shared variables.
|
||||
2. Override the prompt's definition-level output contract with another valid
|
||||
upstream contract. Prove that Audita's per-call JSON schema and configured
|
||||
repair budget still govern execution: a structurally invalid response is
|
||||
repaired or exhausted as the controlled budget dictates. Domain validators
|
||||
remain downstream. Reuse existing validator/proposal integration coverage
|
||||
for their rules rather than weakening them to fit an operator prompt.
|
||||
3. Replace the reload portion of
|
||||
`TestPreparedAdapterRejectsTargetDriftAndPreparationErrors`. After successful
|
||||
construction, change a profile or message file and verify the existing
|
||||
runtime still uses its captured settings/text. Construct a new runtime to
|
||||
observe a valid edit or reject a newly mismatched endpoint. Keep the missing
|
||||
prompt preparation failure case.
|
||||
4. Retain a focused test of the prepared-target check independently of reload:
|
||||
deliberately change the test runtime's application backend endpoint after
|
||||
construction and verify the inconsistency is blocked before dispatch. This
|
||||
tests the existing guard, not a supported live mutation/reload feature.
|
||||
Continue executing the same prepared handle whose details were inspected;
|
||||
do not add another preparation or an application-side retry loop.
|
||||
|
||||
**Verification and completion:** Run the prompt override, snapshot, and
|
||||
prepared-adapter tests. Compare semantic messages and selected targets, not old
|
||||
hash values or exact YAML formatting. Check whitespace and commit.
|
||||
|
||||
## Stage 7 — Verify Native Profiles, Backend Authority, and Resilience
|
||||
|
||||
**Primary files:** Runtime inspection and adapter tests,
|
||||
`internal/framework/llm/adapter_capacity_test.go`, and existing framework/CLI
|
||||
integration tests where their established behavior is affected.
|
||||
|
||||
1. Add a compact native profile inheritance fixture with literal keyed IDs
|
||||
unrelated to filenames, multiple definitions in one file, a partial parent,
|
||||
and a complete selected child. Verify inherited controls survive a child
|
||||
model change and child overrides win. Reuse the metadata-facing literal-key
|
||||
coverage from Stage 5 rather than repeating it in profile tests.
|
||||
2. Use controlled HTTP request bodies to distinguish explicit zero temperature
|
||||
from omission and inherited-control suppression by null. Check a
|
||||
representative recursively overlaid provider parameter and null clearing;
|
||||
do not promise that all providers accept every valid configured zero.
|
||||
Reuse existing deadline/cancellation coverage for unchanged outer deadlines;
|
||||
explicit zero profile timeout must not disable the operation context.
|
||||
3. Verify an operator backend contribution can add supported provider fields,
|
||||
while a conflicting endpoint/concurrency/queue/environment contribution
|
||||
cannot displace the final application fields. Preserve an explicit profile
|
||||
environment lookup as a selected-role error rather than hiding it with an
|
||||
adapter override. Include an application backend using a catalog ID to show
|
||||
it is allowed and its catalog credential lookup is suppressed. Use synthetic
|
||||
environment values and local HTTP fixtures, not provider catalog endpoints.
|
||||
4. Reuse `TestRolesShareBackendCapacityAndReleaseAfterCancellation` and
|
||||
`TestDistinctBackendsHaveIndependentCapacity`; verify the final mapped
|
||||
limits, including zero queue capacity, still coordinate both roles and
|
||||
remain backend-local. Strengthen a fixture with conflicting lower-tier
|
||||
capacity only where needed to prove the new mapping wins. Keep existing
|
||||
synchronization; do not infer admission with sleeps or add another limiter.
|
||||
5. Preserve offline selected-profile inspection and value-free application
|
||||
explanations. Check configured secrets, endpoint strings, arbitrary provider
|
||||
fields, and prompt text remain outside the current safe profile summary.
|
||||
Do not replace it with a full upstream dump. Its numeric projection is not
|
||||
proof that an omitted provider setting is sent; wire presence belongs to
|
||||
the HTTP coverage above.
|
||||
6. Run the existing transient retry, structural repair, attempt-count,
|
||||
observation, partial-validation, and all-model-failure subprocess cases
|
||||
after fixture migration. Preserve their error/recovery/debug classifications,
|
||||
cancellation, and continued output. Fix proven integration regressions;
|
||||
do not rewrite domain outcomes or reduce budgets to obtain a passing suite.
|
||||
|
||||
**Verification and completion:** Run
|
||||
`go test ./internal/core/config ./internal/cli ./internal/framework/llm` and
|
||||
`git diff --check`. Review the dependency and attribution diff, then commit.
|
||||
This establishes the upgraded baseline; it does not establish feature completion
|
||||
or replace the final suite in Stage 5.
|
||||
`go test ./internal/framework/llm ./internal/framework/proposal_generation ./internal/framework/validators ./internal/framework/runner ./internal/cli ./cmd/audita`
|
||||
and `git diff --check`. Review whether each new test addresses a distinct changed
|
||||
boundary. Commit the native-profile and runtime verification unit.
|
||||
|
||||
## Stage 2 — Verify Inherited Configuration, Defaults, and Credentials
|
||||
## Stage 8 — Document the Migration and Update Attribution
|
||||
|
||||
**Completion:** Complete.
|
||||
**Primary files:** `docs/config.md`, `docs/internal/llm-runtime.md`,
|
||||
`docs/internal/prompts.md`, `docs/integrations/openai-compatible-llm.md`,
|
||||
`docs/operations.md`, `docs/internal/cli-config.md`, `README.md`, and
|
||||
`docs/oss/attribution.md`.
|
||||
|
||||
**Primary files:** `internal/core/config/config_test.go`; change
|
||||
`internal/core/config/config.go` only if these checks expose an integration defect.
|
||||
1. Update the configuration contract for unified `profiles_dir` content, exact
|
||||
keyed identities, directory composition, agreed tier precedence, profile
|
||||
inheritance/presence, selected-role completeness, eager validation of all
|
||||
final definitions, selected-root failures, and construction-time capture.
|
||||
Replace the obsolete statement that PromptKit profiles do not gain ConfigKit
|
||||
inheritance. Retain application overlays, defaults, and credential ownership.
|
||||
2. Include concise migration instructions in the profile section: wrap old
|
||||
profile fields beneath `profiles/<id>`, remove inner `id`, change
|
||||
`base_profile` to `extends`, review explicit values/nulls, and select roots
|
||||
whose final definitions have valid backend references. Show a small keyed
|
||||
inheritance/override snippet and link complete maintained examples. Explain
|
||||
explicit zero temperature versus omitted or null controls and link upstream
|
||||
for the full field table. State that this ships as an Audita major migration;
|
||||
do not invent an exact release version or automatic file conversion.
|
||||
3. Document intentional prompt overrides and optional shared-variable references.
|
||||
Supplying shared variables does not inject omitted text, while per-call
|
||||
schemas and domain validators remain enforced. Internal prompt documentation
|
||||
distinguishes shipped asset instructions from operator-effective messages;
|
||||
no prompt-quality rewrite or new global prompt-policy requirement is added.
|
||||
4. Describe layered runtime assembly, source accounting, snapshot use, and
|
||||
startup classification in their internal owner. Update the pinned PromptKit
|
||||
consumer link in the LLM integration contract. Expand operations' loaded
|
||||
source description to include consumed operator message files; keep physical
|
||||
output/collision rules with operations and diagnostics.
|
||||
5. Repair example navigation for separate local/service roots, especially the
|
||||
README's local-profile link. Update validation recipes to use the maintained
|
||||
config files' explicit roots and keep the service testing overlay workflow.
|
||||
Review other affected links without rewriting historical release notes.
|
||||
6. Update the PromptKit and both catalog attribution entries to selected versions
|
||||
and exact commit-pinned source/notice URLs. Resolve catalog tag commits from
|
||||
the selected module origins or authoritative upstream refs, not annotated
|
||||
tag object IDs. Check the releases' COPYRIGHT/LICENSE files and retain the
|
||||
applicable unchanged local LGPL/GPL and MIT texts. Keep the existing House
|
||||
Format; no new license selection, policy rewrite, or attribution inventory
|
||||
is needed. Review `go.mod`/`go.sum` against those entries.
|
||||
|
||||
1. Read `Load`, its resolver, and domain validation. Existing backend capacity
|
||||
defaults use `Resolution.IsExplicit`; omitted validation-role selection uses
|
||||
`ck.Default` against the final proposal selection. Keep those mechanisms if
|
||||
they already handle inherited presence correctly.
|
||||
2. Add a focused real-loader fixture with valid parent and child backends. The
|
||||
child inherits an endpoint, concurrency, zero queue capacity, and a synthetic
|
||||
credential reference. Check effective values and inherited explicitness;
|
||||
in another case, let the child override concurrency. This protects against
|
||||
replacing inherited capacity with Audita's absent-field defaults. Reuse
|
||||
current tests for ordinary omitted defaults.
|
||||
3. Exercise precedence with a selected overlay that changes a parent's setting
|
||||
and a child override that wins over its resolved parent. Preserve coverage
|
||||
that an explicit proposal-profile override wins over the overlay and omitted
|
||||
validation-profile selection derives from that final value. Use the existing
|
||||
`TestOverlayOverrideAndInheritance` fixture where useful; its current
|
||||
"inheritance" check covers role derivation, not native sibling inheritance.
|
||||
4. Extend credential coverage using controlled environment variables. Check that
|
||||
an inherited reference resolves and that effective YAML excludes its resolved
|
||||
value. A child's `credential: null` removes only that child's reference;
|
||||
the parent's surviving reference still must resolve, even if the parent is
|
||||
unselected. Check that an unset surviving reference fails and a missing
|
||||
reference removed by final precedence is not resolved. Reuse
|
||||
`TestSurvivingCredentials` rather than duplicating all secret cases.
|
||||
5. Include a representative inherited setting that violates existing Audita
|
||||
validation, such as zero backend concurrency. Inheritance must not bypass
|
||||
validation of either the parent or child. Existing explicit-empty/null role
|
||||
validation cases remain applicable; extend them only if inherited presence
|
||||
exposes an uncovered interaction.
|
||||
6. If a regression appears, correct the owning default or resolution integration
|
||||
using ConfigKit presence/path APIs. Do not parse raw YAML, exempt parent
|
||||
backends, or introduce abstract backend templates. Do not reproduce upstream
|
||||
cycle, nesting, or expansion-limit tests.
|
||||
**Verification and completion:** Check affected local links/anchors, format
|
||||
examples against the implemented loader, verify source revisions/notices, and
|
||||
run `git diff --check`. Commit the documentation and attribution unit.
|
||||
|
||||
**Verification and completion:** Run `go test ./internal/core/config` and
|
||||
`git diff --check`. Review assertions for new validation rules or duplicated
|
||||
library tests, then commit the focused configuration unit.
|
||||
## Stage 9 — Validate the Integrated End State and Close the Plan
|
||||
|
||||
## Stage 3 — Expose Compact Inheritance Evidence in Safe Inspection
|
||||
**Primary files:** This plan and the feature roadmap; correct an implementation
|
||||
file only if the final checks reveal an actual gap.
|
||||
|
||||
**Completion:** Complete.
|
||||
1. Verify the full roadmap acceptance criteria against the landed code and the
|
||||
distinct tests from these stages. Review for unintended requirements:
|
||||
profile-only filtering, mandatory shared text in operator messages,
|
||||
filename-derived IDs, complete unselected profiles, catalog ID bans,
|
||||
successful-path asset preflights, compatibility readers, and new output
|
||||
guarantees are not part of the accepted target.
|
||||
2. Run `go build -o ./bin/audita ./cmd/audita` and `go test ./...`. Keep the
|
||||
generated binary out of commits. Reuse successful checks on unchanged
|
||||
content; rerun checks affected by any corrective code changes.
|
||||
3. Validate the maintained examples offline using the built binary:
|
||||
|
||||
**Primary files:** `internal/framework/llm/safe_view.go`,
|
||||
`internal/framework/llm/runtime_inspection_test.go`, `docs/cli.md`, and the
|
||||
relevant explanation description in `docs/internal/llm-runtime.md` if needed.
|
||||
```sh
|
||||
./bin/audita config validate --config examples/minimal-config.yml
|
||||
./bin/audita config validate --config examples/configkit-inheritance.yml
|
||||
AUDITA_SERVICE_KEY=synthetic-example ./bin/audita config validate --config examples/production-config.yml
|
||||
AUDITA_SERVICE_KEY=synthetic-example ./bin/audita config validate --config examples/production-config.yml --overlay testing
|
||||
./bin/audita config print-effective --config examples/minimal-config.yml --view profiles
|
||||
./bin/audita config print-effective --config examples/configkit-inheritance.yml --view explain
|
||||
```
|
||||
|
||||
1. Extend `explanationView` with `inheritance,omitempty` and a small step view
|
||||
containing `from`, `to`, and `reference`. For effective field records, project
|
||||
`info.Origin.Inheritance` in its upstream order. Preserve existing `path`,
|
||||
`source`, `layer`, and application `derivations` behavior. Do not serialize
|
||||
field values, whole `Explanation` objects, resolved credentials, or arbitrary
|
||||
configuration content into this view.
|
||||
2. Continue to use `Metadata.Lookup`, `HistoricalPaths`, and existing derivation
|
||||
mapping. Do not infer ancestry from labels or YAML, flatten a chain to one
|
||||
guessed parent, or reinterpret inheritance as an application-derived default.
|
||||
This addition describes effective inheritance; complete displaced ancestry
|
||||
and upstream source-coordinate reporting are not new requirements.
|
||||
3. Test decoded YAML for one inherited field's source/destination/reference paths
|
||||
and a child field that directly overrides its parent. Include an inherited
|
||||
credential reference with a distinctive synthetic resolved value and prove
|
||||
that value is absent. Retain existing application-derived role explanations
|
||||
and safe profile-view coverage. Assert meaningful records, not complete
|
||||
YAML snapshots or upstream-generated identifiers.
|
||||
4. Check the existing shared callers: CLI explain uses `ExplanationYAML`, and
|
||||
`processState.startDebug` uses it for configuration origins. Keep this shared
|
||||
path; no separate debug projection is needed. Update the CLI contract with
|
||||
the optional inheritance step fields and their value-free meaning once they
|
||||
are implemented. Link internal documentation to that contract for exact shape.
|
||||
Confirm local/service roots and testing selections work without provider
|
||||
dispatch. Check safe views exclude resolved secrets and preserve the
|
||||
established explanation fields. Do not call a model to validate examples.
|
||||
4. Check the selected dependency graph, attribution, affected links, and final
|
||||
diff. Run `git diff --check`. Confirm only the released modules are used and
|
||||
no obsolete `/v2` catalog or removed construction API remains in active code.
|
||||
Historical release documents and historical roadmap links may retain old
|
||||
versions as accurate records.
|
||||
5. Mark this plan and the feature roadmap complete only after the full target
|
||||
passes. Record the tested implementation revision and checks accurately;
|
||||
distinguish a subsequent documentation-only completion commit. Commit the
|
||||
completion documentation and report remaining work, if any. Do not select
|
||||
a release number, publish, tag, push, poll CI, or add permanent release gates.
|
||||
|
||||
**Verification and completion:** Run
|
||||
`go test ./internal/framework/llm ./internal/cli` and `git diff --check`.
|
||||
Review emitted fields and affected documentation, then commit the inspection unit.
|
||||
|
||||
## Stage 4 — Verify CLI Failures and Runtime Use of Inherited Backends
|
||||
|
||||
**Completion:** Complete.
|
||||
|
||||
**Primary files:** `internal/framework/llm/adapter_capacity_test.go`,
|
||||
`internal/cli/config_input_test.go`, and
|
||||
`cmd/audita/main_integration_test.go`. Touch production wiring only to fix a
|
||||
proven integration defect.
|
||||
|
||||
1. Reuse `TestRolesShareBackendCapacityAndReleaseAfterCancellation` or its
|
||||
existing fixture to load a valid parent and a selected child backend through
|
||||
`config.Load`. Both existing-format role profiles select that child. Inherit
|
||||
the controlled server endpoint, a test-selected positive capacity, zero
|
||||
queue capacity, and a synthetic credential; keep the parent valid too.
|
||||
2. Verify actual requests reach the inherited endpoint and carry the inherited
|
||||
credential. Existing overlapping role-call coverage proves that the two
|
||||
roles share the child's capacity. Preserve that check with inherited input
|
||||
and its release/cancellation coverage; do not build another scheduler or
|
||||
duplicate the provider retry/repair matrix. Use synchronization already in
|
||||
the test, not sleeps to infer admission behavior.
|
||||
3. Add one representative invalid relationship at the CLI boundary: a missing
|
||||
sibling parent should yield configuration exit code `2` and no provider
|
||||
dispatch. Use the existing subprocess fixture and semantic diagnostic
|
||||
assertions; do not freeze full error text or invent early-failure reports.
|
||||
4. Extend one representative subprocess success/inspection case to use inherited
|
||||
backend input. With debug explicitly enabled, check that effective settings
|
||||
and configuration-origin artifacts retain inherited settings and the new
|
||||
explanation evidence without the resolved synthetic secret. Check the
|
||||
explain view through the existing CLI route. Use existing coverage for
|
||||
unchanged report placement, ordinary processing, and debug-disabled behavior.
|
||||
Keep assertions on the affected fields rather than the entire artifact set.
|
||||
|
||||
**Verification and completion:** Run
|
||||
`go test ./internal/framework/llm ./internal/cli ./cmd/audita` and
|
||||
`git diff --check`. Review whether each added test covers a distinct integration
|
||||
risk, then commit the runtime/CLI verification unit.
|
||||
|
||||
## Stage 5 — Document Native Inheritance and Close the Adoption Unit
|
||||
|
||||
**Completion:** Complete.
|
||||
|
||||
**Primary files:** `docs/config.md`, `examples/configkit-inheritance.yml`,
|
||||
`docs/internal/cli-config.md`, and `docs/roadmap/configkit.md`. Update another
|
||||
canonical document only where an implemented statement actually changes.
|
||||
|
||||
1. Add `examples/configkit-inheritance.yml` beside the existing examples, with
|
||||
`version: 2`, a valid `local-base` backend, and a valid `local` backend that
|
||||
extends it. Inherit a local endpoint and zero queue capacity; use a positive
|
||||
parent concurrency and a child concurrency override. Select the existing
|
||||
`local-proposal` and `local-validation` profiles under `profiles/` and the
|
||||
grammar module. The example needs no credential or new profile format and
|
||||
can be validated offline without a running local server.
|
||||
2. Update the configuration contract to describe native sibling inheritance,
|
||||
precedence before inheritance, inherited explicit presence, null credential
|
||||
removal, and normal parent validation. Link the maintained example and the
|
||||
version-pinned upstream generic grammar. Retain current file/overlay/override
|
||||
behavior and main-file-relative path rules. Do not imply that ordinary
|
||||
PromptKit v0.11.2 profile files now accept ConfigKit `extends` syntax.
|
||||
3. Document that `extends` is reserved across input mappings. Explain the legacy
|
||||
backend-name conflict and migration: rename the backend to a non-conflicting
|
||||
ID and update every profile selecting it. Keep application `version: 2`;
|
||||
do not claim all previously valid configurations are compatible. Record the
|
||||
compatibility finding in the feature roadmap for later release preparation.
|
||||
Release classification remains outside this implementation unit, governed by
|
||||
the existing release policy; no policy exception is selected here.
|
||||
4. Add the new example's offline validation command to the existing example
|
||||
recipe. Validate it and the maintained minimal, production, and selected
|
||||
testing-overlay examples, with a controlled synthetic service credential.
|
||||
Inspect the new example's settings and explanation output. No live model
|
||||
request is needed to check configuration or profiles.
|
||||
5. Run `go build -o ./bin/audita ./cmd/audita`, `go test ./...`, affected-link
|
||||
checks, and `git diff --check`. Review the final dependency graph and diff:
|
||||
ConfigKit is the released v0.5.0, PromptKit/catalogs retain their selections,
|
||||
source attribution is pinned, and no unrelated settings or limits were added.
|
||||
Keep generated binaries out of the commit. Resolve actual failures rather
|
||||
than adding gates unrelated to this feature.
|
||||
6. Mark the feature roadmap and this plan complete only after all stage outcomes
|
||||
pass. Record the tested source revision and checks accurately; if a final
|
||||
documentation-only completion commit follows validation, distinguish it from
|
||||
the tested implementation revision. Review and commit the final documentation
|
||||
unit, leaving a clean workspace. Reuse successful checks for unchanged content.
|
||||
|
||||
**End state:** Native ConfigKit inheritance reaches typed settings, safe
|
||||
inspection, selected runtime backends, and maintained operator examples;
|
||||
PromptKit migration remains a separate unit. No implementation-blocking design
|
||||
questions remain. The eventual release version and treatment of the known
|
||||
compatibility impact remain release-preparation decisions, not prerequisites
|
||||
for these stages.
|
||||
**End state:** Native PromptKit configuration and catalogs are adopted,
|
||||
operator overrides and captured resources work through the real runtime,
|
||||
application backend/domain ownership and resilience remain effective, and
|
||||
maintained examples and migration documentation match the implementation.
|
||||
No implementation-blocking decisions remain; exact major-release numbering
|
||||
and publication timing remain outside this implementation unit.
|
||||
|
||||
@@ -2,9 +2,8 @@
|
||||
|
||||
**Last updated:** 2026-10-03
|
||||
|
||||
**Status:** Accepted target scope; ready for implementation planning. The
|
||||
operator-directory scope decision has been resolved. Implementation has not
|
||||
begun.
|
||||
**Status:** Accepted target scope; implementation is defined in the
|
||||
[implementation plan](implementation.md). Implementation has not begun.
|
||||
|
||||
## Purpose and Baseline
|
||||
|
||||
@@ -263,4 +262,5 @@ No live provider calls, new release gates, or CI-success dependency are needed
|
||||
to establish adoption. Guided domain recovery and the other work recorded in
|
||||
[future.md](future.md) remain separate features. Do not expand this upgrade into
|
||||
new report formats, pipeline scheduling, or general configuration-inspection
|
||||
commands. A staged implementation plan can now be prepared.
|
||||
commands. The [implementation plan](implementation.md) owns execution steps
|
||||
and their verification.
|
||||
|
||||
Reference in New Issue
Block a user