Finalize the ConfigKit and PromptKit implementation plan
This commit is contained in:
@@ -2,9 +2,9 @@
|
||||
|
||||
**Last updated:** 2026-09-24
|
||||
|
||||
**Status:** Prototype boundary and selected overlays verified. The ordered
|
||||
production handoff below is planned, not implemented; no production migration or
|
||||
release has begun.
|
||||
**Status:** Final target selected; production implementation pending. The boundary
|
||||
and selected overlays are verified. The [implementation plan](implementation.md)
|
||||
owns execution stages; no production migration or release has begun.
|
||||
|
||||
## Purpose and Direction
|
||||
|
||||
@@ -27,10 +27,10 @@ application must depend on a versioned module, not a sibling checkout or local
|
||||
module replacement.
|
||||
|
||||
The shared boundary/prototype unit is complete. Its [selected contract, inventory,
|
||||
and implementation handoff](library-boundary-findings.md) refine the provisional
|
||||
directions here; production configuration migration has not begun.
|
||||
and evidence](library-boundary-findings.md) define the detailed target alongside
|
||||
this roadmap. Production configuration migration has not begun.
|
||||
|
||||
## Proposed End State
|
||||
## Target End State
|
||||
|
||||
- ConfigKit owns strict YAML loading, input presence, default application, explicit
|
||||
override mechanics, recorded resolution, secret references, and safe inspection.
|
||||
@@ -40,17 +40,17 @@ directions here; production configuration migration has not begun.
|
||||
Audita's current per-layer validation and environment-overlay behavior.
|
||||
- General configuration environment overrides are retired. Credentials use typed
|
||||
environment-name secret references; supplied references must resolve according
|
||||
to ConfigKit's contract. Review config-file selection separately as an
|
||||
application-owned startup decision.
|
||||
to ConfigKit's contract. File selection follows the application-owned
|
||||
discovery rules in the selected boundary.
|
||||
- Standardize duration representations, explicit empty/null/zero handling, and
|
||||
inheritance on the shared conventions. Express genuine Audita domain rules
|
||||
through the supported resolution and validation APIs.
|
||||
- Review the configuration-related CLI afresh. Useful explicit flags can become
|
||||
ConfigKit override records; old names and aliases are not requirements. Direct
|
||||
credential-value flags should give way to the secret-reference convention.
|
||||
- Retained domain-setting flags and proposal/validation profile flags become
|
||||
explicit ConfigKit overrides according to the migration inventory. Remove old
|
||||
provider/key/retry/concurrency flags and aliases. Credentials use secret references.
|
||||
- Use ConfigKit's redacted effective output and provide source/explanation access
|
||||
where useful. Existing JSON inspection output need not be preserved. Coordinate
|
||||
diagnostic configuration snapshots with the new model.
|
||||
through the selected views below. Legacy JSON inspection is replaced.
|
||||
Diagnostic configuration snapshots use the same new model.
|
||||
- Delete superseded loading and override machinery. Do not hide the old model
|
||||
behind adapters that defeat the adoption's consistency benefit.
|
||||
|
||||
@@ -102,228 +102,69 @@ effective-config inspection, selected-profile inspection, and processing must al
|
||||
use the same selected overlay. Preserve overlay provenance in configuration
|
||||
explanations and diagnostics while retaining secret redaction.
|
||||
|
||||
The completed prototype extension and production-planning handoff are recorded
|
||||
in the [prototype roadmap](library-boundary.md#next-unit-testing-overlays-and-production-migration-plan)
|
||||
and its ordered [implementation plan](implementation.md). Production CLI changes
|
||||
below remain subsequent migration work.
|
||||
The [recorded overlay results](library-boundary-findings.md#selected-overlay-extension-2026-09-24)
|
||||
prove the representative load, inspection, and HTTP path. Production acceptance
|
||||
requires the same selection to reach both actual LLM roles, with explicit flags
|
||||
winning over overlays and surviving secret references resolved/redacted correctly.
|
||||
The isolated prototype is evidence, not a second supported runtime.
|
||||
|
||||
Implementation and acceptance work:
|
||||
## Inspection and Invocation Contract
|
||||
|
||||
- The isolated prototype pins ConfigKit v0.4.0 and includes the testing-overlay
|
||||
fixture. Preserve the original v0.3.1 evidence as historical; the
|
||||
[extension results](library-boundary-findings.md#selected-overlay-extension-2026-09-24)
|
||||
establish the selected-overlay behavior.
|
||||
- Verify unchanged base selection when no overlay is requested, both testing role
|
||||
selections, and explicit setting flags taking precedence over the overlay.
|
||||
- Verify role inheritance after overlay application, effective profile/backend
|
||||
assembly, and secret resolution from the surviving configuration.
|
||||
- Reject missing/invalid overlay selections and prohibited version/directive changes
|
||||
through ConfigKit; rely on its tests for exhaustive merge/parser cases.
|
||||
- Verify redacted inspection and overlay provenance, then run both LLM roles through
|
||||
the offline provider using the selected testing profiles. Provider requests must
|
||||
use their model selections; no paid calls are needed.
|
||||
- Add production CLI coverage for the three commands and document selection,
|
||||
precedence, example layout, and migration guidance when implemented.
|
||||
`process`, `config validate`, and `config print-effective` share file/overlay
|
||||
selection and retained setting overrides. Invocation I/O paths remain process-only.
|
||||
Only explicitly supplied flags override settings. Validation requires an explicit
|
||||
config file; processing and inspection use the selected discovery order. Help is
|
||||
available before configuration loading. All three commands validate selected
|
||||
profiles without requiring a provider call during setup.
|
||||
|
||||
This expands the next configuration unit; it does not reopen the completed
|
||||
prototype's original scope or change the ConfigKit → PromptKit → combined release
|
||||
sequence. The production LLM migration remains necessary before promising the full
|
||||
new runtime behavior.
|
||||
`config print-effective --view settings|profiles|explain` exposes three YAML views:
|
||||
|
||||
## Ordered Production Migration Handoff
|
||||
- `settings` (default): ConfigKit's redacted effective configuration.
|
||||
- `profiles`: an Audita-owned summary of each selected role's profile, backend,
|
||||
model, temperature, token limit, top-p, timeout, service tier, and reasoning
|
||||
effort where present. Arbitrary extra provider parameters and credentials are
|
||||
omitted; this is a selected-controls view, not every execution input.
|
||||
- `explain`: field paths, sources, and derivation records from ConfigKit, without
|
||||
resolved secret values.
|
||||
|
||||
The [migration inventory](library-boundary-findings.md#migration-inventory) is the
|
||||
field-by-field old-to-new map. The [selected boundary](library-boundary-findings.md#configuration-and-profile-contract)
|
||||
owns defaults and semantic choices; the [v0.4.0 overlay results](library-boundary-findings.md#selected-overlay-extension-2026-09-24)
|
||||
prove the representative load, inspection, and HTTP path. These packages are
|
||||
future work within one coordinated refactor to adopt both libraries. The order
|
||||
organizes implementation dependencies, not independently runnable product states.
|
||||
Intermediate commits need not build or run; do not add temporary scaffolding,
|
||||
compatibility shims, or a processing gate to support them. Remove superseded code
|
||||
as its replacement lands. Run focused checks as components become testable, and
|
||||
validate the complete application before declaring the refactor finished. The
|
||||
isolated prototype is evidence, not a second production runtime.
|
||||
A ConfigKit snapshot alone does not describe profile-owned controls. Profile
|
||||
inspection must enforce the registered-backend and credential boundary. Diagnostic
|
||||
snapshots use these safe views and preserve overlay/explicit-override provenance.
|
||||
The migration guide explains the intentional replacement of legacy inspection JSON.
|
||||
|
||||
### 1. Define and load the complete version-2 configuration
|
||||
## Integration and Completion Criteria
|
||||
|
||||
**Depends on:** the verified prototype. **Owns:** `internal/core/config` and its
|
||||
tests. Replace the version-1 YAML/env application path with one ConfigKit v0.4.0
|
||||
definition and a version-2 rule. Pin the published module in the root Go module
|
||||
and raise the production Go minimum as needed. Keep the existing YAML dependency
|
||||
used for glossary parsing. ConfigKit imports and secret-directory bootstrap stay
|
||||
disabled.
|
||||
ConfigKit and PromptKit form one intentional breaking refactor. Intermediate
|
||||
commits need not build or run. The target contains no temporary old-runtime adapter,
|
||||
compatibility shim, processing gate, dual loader, or automatic legacy conversion.
|
||||
The [combined feature roadmap](library-adoption.md) owns execution/reporting intent;
|
||||
[implementation.md](implementation.md) alone owns stage order and implementation tasks.
|
||||
|
||||
The definition must cover every retained domain setting in the inventory:
|
||||
ordered/repeated `pipeline.modules`, `output.schema`, validation prompt budget,
|
||||
chunking, normalization, thresholds, context, and diagnostics, plus `profiles_dir`,
|
||||
backend endpoint/concurrency/queue capacity/optional credential, and LLM proposal
|
||||
and validation profile selection, transient retries, repair attempts, and total
|
||||
operation timeout. Apply the inventory's defaults once. Preserve presence:
|
||||
omitted validation profile inherits the **final** proposal profile; explicit
|
||||
empty/null validation fails. Omitted or null section target means automatic, zero
|
||||
fails. Explicit empty module list means no modules. Optional credential null means
|
||||
unauthenticated; supplied references must resolve to nonempty values. Keep the
|
||||
selected numeric-second normalization fields numeric, and use a duration string
|
||||
for `llm.operation_timeout`; do not revive version-1 duration coercion. Validate
|
||||
backend limits, references, profile selections, thresholds, and cross-field domain
|
||||
rules on the resolved result. Preserve ConfigKit provenance for derived values.
|
||||
The final application loads all retained domain settings, assembles both selected
|
||||
profiles, and processes through PromptKit. Controlled HTTP evidence must connect
|
||||
configuration selection to actual requests, domain outcomes, safe diagnostics,
|
||||
and shared finite backend capacity. Maintained examples and one migration guide
|
||||
cover the complete file/flag/environment transition. Obsolete application-owned
|
||||
loading, transport, rendering, and scheduler machinery is removed.
|
||||
|
||||
**Acceptance/tests:** `internal/core/config` tests use temporary files to cover
|
||||
version rejection, strict fields, defaults, omission/null/zero, module order,
|
||||
final role inheritance, overlays and explicit override precedence, secret
|
||||
replacement/missing/null, and retained domain validation. Compare behavior to the
|
||||
prototype extension rather than duplicating ConfigKit's parser matrix. No legacy
|
||||
environment overlay or old-schema acceptance remains.
|
||||
The combined target is Audita v2.0.0. Release compliance and publication remain
|
||||
last and separate from implementing the refactor. Retention and transcript timestamp
|
||||
changes are not included; the selected retry/repair contract resolves the retry
|
||||
follow-up in [runtime behavior](runtime-behavior.md).
|
||||
|
||||
### 2. Select files and map invocation input
|
||||
|
||||
**Depends on:** package 1. **Owns:** `internal/cli/run.go`,
|
||||
`internal/cli/process_flags.go`, `internal/cli` tests, and `cmd/audita` subprocess
|
||||
tests. Select exactly one main file: explicit `--config`, then the two system
|
||||
locations in the selected boundary. Remove `AUDITA_CONFIG`; a missing explicit
|
||||
path fails, while absent system files are skipped. Require an explicit file for
|
||||
`config validate`. Parse help before loading configuration.
|
||||
|
||||
Add invocation-only `--overlay <name>` to `process`, `config validate`, and
|
||||
`config print-effective`, passing it to `Options.Overlay` for the same selected
|
||||
main file. Omitted, empty, or `default` selects the base. Keep transcript,
|
||||
glossary, output, and report paths outside persistent configuration. Map only
|
||||
flags actually supplied by the user into ConfigKit overrides; parser defaults
|
||||
must never defeat file or overlay values. Retain useful domain-setting flags from
|
||||
the inventory. Add profile-selection flags. Remove direct API-key/model/endpoint,
|
||||
old retry/role/global concurrency flags and aliases rather than translating them
|
||||
silently. Retire all general `AUDITA_*` configuration overlays listed in the
|
||||
inventory; environment names retain meaning only when explicitly referenced as
|
||||
secrets.
|
||||
|
||||
**Acceptance/tests:** CLI and subprocess tests cover discovery order, explicit
|
||||
missing files, all three commands with base/testing/default/missing selections,
|
||||
overlay then explicit-setting precedence, omitted flags, usage/error streams and
|
||||
exit classifications. Reject old flags and implicit environment configuration.
|
||||
The same loaded result must feed validation, effective inspection, and processing.
|
||||
|
||||
### 3. Assemble profiles and safe inspection
|
||||
|
||||
**Depends on:** packages 1–2. **Owns:** a small CLI/runtime assembly boundary,
|
||||
config inspection, and diagnostic snapshot wiring in `internal/cli` and
|
||||
`internal/core/diagnostics`; verify against `internal/framework/processreport`
|
||||
where snapshots are referenced. Resolve `profiles_dir` relative to the selected
|
||||
**main** file, including when an overlay is selected; absolute paths remain valid.
|
||||
Inspect both selected PromptKit profiles and verify their named registered backend,
|
||||
endpoint equality, and absence of a competing profile credential lookup. Treat
|
||||
ConfigKit settings and PromptKit generation controls as separate views.
|
||||
|
||||
Use `WriteEffective` for redacted effective YAML and expose relevant
|
||||
explanation/provenance and selected-profile model/backend controls through an
|
||||
Audita-owned safe view. A ConfigKit snapshot alone cannot claim to describe the
|
||||
full execution target. Validation and inspection resolve surviving secret
|
||||
references without provider calls; a missing or empty surviving reference fails,
|
||||
while a replaced reference is not resolved. Snapshot and surfaced error paths
|
||||
must redact secret **values** and preserve overlay/CLI origin. Avoid serializing
|
||||
raw library objects into stable report formats.
|
||||
|
||||
**Acceptance/tests:** config-command and diagnostics tests verify relative paths,
|
||||
same selection across views, overlay/CLI origin, surviving credentials,
|
||||
unauthenticated null, and no secret values in YAML, explanations, selected-profile
|
||||
inspection, diagnostics, or errors. Update `internal/cli/run_test.go`,
|
||||
`internal/core/config/*_test.go`, and report/diagnostics tests that own these
|
||||
outputs.
|
||||
|
||||
### 4. Connect the final configuration directly to PromptKit execution
|
||||
|
||||
**Depends on:** packages 1–3. **Owns:** the final CLI/runtime assembly boundary,
|
||||
`internal/framework/llm`, prompt/schema loading, runner wiring, and relevant tests.
|
||||
Implement PromptKit adoption as part of the same refactor. Convert domain settings
|
||||
into Audita's existing domain structs and supply the selected profiles, registered
|
||||
backends, resolved secrets, retry/repair budgets, and logical-operation deadlines
|
||||
directly to PromptKit. Keep library types out of correction algorithms and ledgers.
|
||||
|
||||
Migrate all module and validator prompts, preserve their domain checks, and map
|
||||
PromptKit results, classified failures, and observations into Audita reporting and
|
||||
redacted diagnostics. Replace the old provider transport, prompt loader,
|
||||
structural-output handling, and global/role scheduler composition. Follow the
|
||||
[coordinated execution scope](library-adoption.md#4-complete-promptkit-adoption)
|
||||
and [execution/observation contract](library-boundary-findings.md#execution-and-failure-mapping).
|
||||
Do not build an adapter to feed the old LLM runtime or introduce an interim
|
||||
`process` rejection mode.
|
||||
|
||||
**Acceptance/tests:** controlled HTTP tests prove both roles' final model/backend
|
||||
selection, overlays and explicit overrides, finite per-backend capacity, explicit
|
||||
zero retries, positive structural repair, total-operation deadlines, cancellation,
|
||||
domain acceptance/rejection, and safe observations including partial failures.
|
||||
Processing uses the new configuration and execution path together. Reuse existing
|
||||
domain coverage and the prototype's integration evidence; update tests tied to the
|
||||
retired transport and scheduler contracts.
|
||||
|
||||
### 5. Remove superseded configuration paths and update maintained contracts
|
||||
|
||||
**Depends on:** packages 1–4. **Owns:** removal of old file/env merge, aliases,
|
||||
direct-key flags, legacy redaction/inspection paths, and obsolete tests in
|
||||
`internal/core/config` and `internal/cli`. Retain Audita's domain checks,
|
||||
normalization/chunking, output encoding, module and validator behavior. Update
|
||||
`examples/` with version-2 base configuration, `overlays/testing.yml`, complete
|
||||
PromptKit profiles, and secret-free credentials by reference. Update the canonical
|
||||
`docs/config.md`, `docs/cli.md`, operations/integration/internal references as
|
||||
their contracts change, and provide one migration guide with the complete
|
||||
inventory's file/flag/environment mapping, duration and empty-list changes,
|
||||
credentials, profiles/overlays, inspection, and changed runtime/diagnostic
|
||||
semantics. Do not leave dual loaders or a permanent translation mode.
|
||||
|
||||
**Acceptance/tests:** maintained examples validate offline; removed flags and old
|
||||
schema are rejected; relevant config/CLI/subprocess/report tests pass; documentation
|
||||
links and `git diff --check` pass. Remove the isolated prototype when its useful
|
||||
coverage and examples have moved into the production suite.
|
||||
|
||||
### 6. Validate the combined end state and prepare release
|
||||
|
||||
**Depends on:** packages 1–5. Verify the assembled version-2 configuration,
|
||||
overlay, profile, execution, reporting, and artifact workflows. Run the relevant
|
||||
full build/test checks, confirm obsolete configuration and LLM machinery is gone,
|
||||
and verify maintained examples and the migration guide against the final behavior.
|
||||
There is no requirement to validate or preserve intermediate product states.
|
||||
|
||||
The refactor is complete when both ConfigKit and PromptKit are adopted and the
|
||||
complete application passes its required checks. Release compliance, license/
|
||||
distribution review, and publication follow last under the established release
|
||||
procedure. No publication is authorized by this plan.
|
||||
|
||||
## Sequencing Considerations
|
||||
|
||||
Changes to module configuration, runtime settings, CLI design, or diagnostic
|
||||
reporting are likely to overlap this migration. Design their shared target model
|
||||
first. If another change will substantially reshape those areas, avoid migrating
|
||||
the same concepts twice. Work independent of configuration can proceed separately.
|
||||
|
||||
The [coordinated library-adoption roadmap](library-adoption.md) owns the proposed
|
||||
sequence with PromptKit: shared boundary design and a PromptKit prototype, then
|
||||
ConfigKit and PromptKit implementation in dependency order within one refactor,
|
||||
followed by a combined v2.0.0 release. Intermediate commits are work in progress,
|
||||
not supported configurations or runnable milestones.
|
||||
|
||||
The separate [runtime behavior decisions](runtime-behavior.md) track remaining
|
||||
questions and the selected retry direction.
|
||||
The coordinated plan adopts PromptKit's explicit retry semantics and enables
|
||||
structural repair from the outset. Include separate transient-retry and positive
|
||||
repair budgets, plus the total-operation deadline policy, in the shared settings
|
||||
design. The selected defaults and ownership are recorded in the
|
||||
[boundary findings](library-boundary-findings.md#configuration-and-profile-contract).
|
||||
Implement the runtime change explicitly as part of the combined refactor.
|
||||
Retention and transcript timestamp decisions are not automatically included.
|
||||
|
||||
## Implementation Boundaries
|
||||
## Scope Boundaries
|
||||
|
||||
- Pin ConfigKit v0.4.0 in production and align Go/toolchain requirements. ConfigKit requires
|
||||
Go 1.25; the combined PromptKit prototype requires Go 1.25.5. Config-only
|
||||
adoption does not automatically remove Audita's existing YAML dependency used for glossary parsing.
|
||||
Go 1.25; the combined PromptKit prototype requires Go 1.25.5. Adopting these libraries
|
||||
does not automatically remove Audita's existing YAML dependency used for glossary parsing.
|
||||
- Implement the selected configuration and final PromptKit runtime directly,
|
||||
without temporary runtime adapters, compatibility shims, or process gates.
|
||||
- Enable selected overlays as specified above. Keep imports and secret-directory
|
||||
bootstrap disabled unless a separate concrete requirement justifies them.
|
||||
- Validation and inspection resolve surviving credential references and fail on
|
||||
missing or empty values; no credential-free bypass is planned.
|
||||
- Add the safe selected-profile view and ConfigKit provenance to inspection and
|
||||
diagnostics as specified in package 3, then account for changed formats in the
|
||||
migration guide. No schedule or effort estimate is committed.
|
||||
- Inspection and diagnostics use the safe selected-profile view and ConfigKit
|
||||
provenance described above. Changed formats are covered by the migration guide.
|
||||
|
||||
The eventual completion test is that Audita follows the suite's ConfigKit
|
||||
conventions while retaining its intended domain capabilities, with a clear
|
||||
|
||||
Reference in New Issue
Block a user