Document verified prototype overlay behavior
This commit is contained in:
@@ -225,7 +225,9 @@ actual gaps instead of silently relaxing tests or claiming complete integration.
|
|||||||
|
|
||||||
## Stage 5: Record Overlay Findings and Executable Instructions
|
## Stage 5: Record Overlay Findings and Executable Instructions
|
||||||
|
|
||||||
**Status:** Pending. **Depends on:** Stage 4.
|
**Status:** Complete (2026-09-24). **Depends on:** Stage 4. Dated v0.4.0 evidence,
|
||||||
|
test coverage, commands, limitations, README guidance, and roadmap links updated;
|
||||||
|
affected links and `git diff --check` verified.
|
||||||
|
|
||||||
This stage changes documentation, not production code. Read the new tests/results,
|
This stage changes documentation, not production code. Read the new tests/results,
|
||||||
then update the existing owners rather than creating another parallel design:
|
then update the existing owners rather than creating another parallel design:
|
||||||
|
|||||||
@@ -1,9 +1,11 @@
|
|||||||
# Coordinated ConfigKit and PromptKit Adoption
|
# Coordinated ConfigKit and PromptKit Adoption
|
||||||
|
|
||||||
**Last updated:** 2026-09-23
|
**Last updated:** 2026-09-24
|
||||||
|
|
||||||
**Status:** Shared boundary and offline prototype completed; production migrations
|
**Status:** Shared boundary and offline prototype completed; production migrations
|
||||||
remain planned. See the [findings](library-boundary-findings.md). Detailed design,
|
remain planned. The [v0.4.0 overlay evidence](library-boundary-findings.md#selected-overlay-extension-2026-09-24)
|
||||||
|
extends the completed prototype; its production-planning handoff remains pending.
|
||||||
|
Detailed design,
|
||||||
implementation tasks, dependency versions, and acceptance criteria must be revisited
|
implementation tasks, dependency versions, and acceptance criteria must be revisited
|
||||||
before migration begins. This document authorizes neither implementation nor release
|
before migration begins. This document authorizes neither implementation nor release
|
||||||
publication and sets no schedule.
|
publication and sets no schedule.
|
||||||
|
|||||||
@@ -102,7 +102,7 @@ Selected for the prototype and recommended for production:
|
|||||||
Repeated modules retain their existing ordered meaning.
|
Repeated modules retain their existing ordered meaning.
|
||||||
- The original prototype disabled ConfigKit imports, selected configuration layers
|
- The original prototype disabled ConfigKit imports, selected configuration layers
|
||||||
(then called configuration profiles), and secret-directory bootstrap. Selected
|
(then called configuration profiles), and secret-directory bootstrap. Selected
|
||||||
overlays are now planned under the linked overlay decision; the other two
|
overlays are now verified under the linked overlay decision; the other two
|
||||||
features remain disabled. PromptKit execution profiles retain their meaning.
|
features remain disabled. PromptKit execution profiles retain their meaning.
|
||||||
|
|
||||||
Configuration validation/inspection must assemble and inspect both selected profiles
|
Configuration validation/inspection must assemble and inspect both selected profiles
|
||||||
@@ -369,3 +369,53 @@ and rebuild/relink materials needed for Audita's Go binary/container distributio
|
|||||||
This unit does not publish binaries or claim that release compliance is complete.
|
This unit does not publish binaries or claim that release compliance is complete.
|
||||||
No upstream relicensing request or library implementation change is required for
|
No upstream relicensing request or library implementation change is required for
|
||||||
this prototype.
|
this prototype.
|
||||||
|
|
||||||
|
## Selected-Overlay Extension: 2026-09-24
|
||||||
|
|
||||||
|
The isolated prototype now pins published ConfigKit v0.4.0. ConfigKit loads
|
||||||
|
`overlays/testing.yml` relative to the selected main file; the overlay selects
|
||||||
|
complete PromptKit execution profiles from the separate `profiles/` directory.
|
||||||
|
The original v0.3.1 observations above are historical and do not prove this
|
||||||
|
extension. No production Audita behavior changed.
|
||||||
|
|
||||||
|
With Go 1.26.8 on linux/amd64, from `examples/library-adoption/`, these offline
|
||||||
|
checks passed with `GOTOOLCHAIN=local GOWORK=off GOPROXY=off GOSUMDB=off`:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go test ./...
|
||||||
|
go test -race ./...
|
||||||
|
go vet ./...
|
||||||
|
go run .
|
||||||
|
go run . --overlay testing
|
||||||
|
go run . --overlay testing --proposal-profile proposal
|
||||||
|
```
|
||||||
|
|
||||||
|
Each demonstration reported one accepted correction, four attempts, zero
|
||||||
|
transient retries, and two structural repairs. The base run selected
|
||||||
|
`synthetic-grammar`/`synthetic-review`; testing selected
|
||||||
|
`synthetic-grammar-testing`/`synthetic-review-testing`; the explicit proposal
|
||||||
|
override restored only `synthetic-grammar`. These are synthetic routing IDs,
|
||||||
|
not price, latency, quality, or provider-compatibility measurements.
|
||||||
|
|
||||||
|
| Requirement | Consumer evidence |
|
||||||
|
| --- | --- |
|
||||||
|
| Base, empty/default, testing selection, role overrides, final inheritance, and relative profile resolution | `TestSelectedOverlayConfiguration` loads temporary on-disk layouts and assembles both roles. |
|
||||||
|
| Actual base/testing/override HTTP models and repairs | `TestDemonstrationSelectsOutboundModels` captures every request body and matches model IDs to the reported selections and domain outcome. |
|
||||||
|
| Missing/invalid selection, version/directive rejection, and ordinary backend-setting validation | `TestOverlayFailuresAndApplicationValidation` checks ConfigKit diagnostic codes and effective backend limits. |
|
||||||
|
| Surviving credential resolution, absent/empty failures, null unauthenticated use, and profile-only switching | `TestOverlayCredentialResolutionAndInspection` controls synthetic environment values and observes HTTP Authorization. |
|
||||||
|
| Redacted effective output, explanation and metadata origin, profile inspection, and observations | `TestOverlayCredentialResolutionAndInspection` and `TestOverlaySelectionProvenance` inspect safe output and overlay/CLI origins. |
|
||||||
|
| Existing retry, repair, domain, capacity, and concurrent-observation behavior | The unchanged nested suite passes, including under `-race`. |
|
||||||
|
|
||||||
|
Changing a profile does not remove a backend credential reference from the
|
||||||
|
effective configuration. ConfigKit validates a surviving reference when only
|
||||||
|
profile selection changes. Replacing the reference in an
|
||||||
|
overlay resolves only the replacement; explicit null allows unauthenticated use.
|
||||||
|
This distinction is part of the selected configuration boundary, not an
|
||||||
|
application fallback. Imports and secret-directory bootstrap remain disabled.
|
||||||
|
|
||||||
|
The tests use a controlled built-in HTTP transport and synthetic credentials;
|
||||||
|
there were no live provider calls. The prototype still covers one representative
|
||||||
|
proposal/assessment pair, not all production modules, prompts, CLI paths, or
|
||||||
|
diagnostic artifacts. Root `go test ./...` does not run this nested module. No
|
||||||
|
upstream API gap appeared in this extension. The production migration handoff is
|
||||||
|
the remaining work for this unit.
|
||||||
|
|||||||
@@ -3,7 +3,7 @@
|
|||||||
**Last updated:** 2026-09-24
|
**Last updated:** 2026-09-24
|
||||||
|
|
||||||
**Status:** Original boundary-design and offline-prototype unit completed;
|
**Status:** Original boundary-design and offline-prototype unit completed;
|
||||||
overlay extension and production-planning handoff planned.
|
overlay extension verified; production-planning handoff pending.
|
||||||
Production migration has not started. The work packages below retain the original
|
Production migration has not started. The work packages below retain the original
|
||||||
scope; [decisions, evidence, limitations, and handoff](library-boundary-findings.md)
|
scope; [decisions, evidence, limitations, and handoff](library-boundary-findings.md)
|
||||||
record the outcome. The [executable prototype](../../examples/library-adoption/README.md)
|
record the outcome. The [executable prototype](../../examples/library-adoption/README.md)
|
||||||
@@ -11,7 +11,8 @@ is isolated from production. No publication or upstream changes were made.
|
|||||||
|
|
||||||
## Next Unit: Testing Overlays and Production Migration Plan
|
## Next Unit: Testing Overlays and Production Migration Plan
|
||||||
|
|
||||||
**Status:** Planned; not implemented. Execute the ordered stages in
|
**Status:** Overlay implementation verified; production-planning handoff pending.
|
||||||
|
Execute the remaining ordered work in
|
||||||
[implementation.md](implementation.md). The original work packages below remain
|
[implementation.md](implementation.md). The original work packages below remain
|
||||||
historical scope for the completed first unit.
|
historical scope for the completed first unit.
|
||||||
|
|
||||||
|
|||||||
@@ -36,6 +36,9 @@ from `profiles/`. An explicitly supplied `--proposal-profile` or
|
|||||||
requested overlay and each effective profile, backend, and model. Synthetic model
|
requested overlay and each effective profile, backend, and model. Synthetic model
|
||||||
IDs demonstrate operator-selected routing; they are not measurements of price,
|
IDs demonstrate operator-selected routing; they are not measurements of price,
|
||||||
speed, or model quality.
|
speed, or model quality.
|
||||||
|
`overlays/` contains ConfigKit application-setting layers; `profiles/` contains
|
||||||
|
PromptKit execution profiles. Profile paths resolve from the main `config.yaml`
|
||||||
|
directory, including when an overlay is selected.
|
||||||
An overlay can also change ordinary backend settings and credential references.
|
An overlay can also change ordinary backend settings and credential references.
|
||||||
ConfigKit resolves only the surviving reference at a replaced credential path;
|
ConfigKit resolves only the surviving reference at a replaced credential path;
|
||||||
changing role profiles alone leaves other configured backend credential references
|
changing role profiles alone leaves other configured backend credential references
|
||||||
|
|||||||
Reference in New Issue
Block a user