284 lines
15 KiB
Markdown
284 lines
15 KiB
Markdown
# Configuration Handoff And Repair Review
|
||
|
||
Reviewed implementation stages 1–17 against
|
||
[the implementation plan](implementation.md),
|
||
[the feature roadmap](next-release-considerations.md), and accepted ADRs
|
||
[0001](../adr/0001-configuration-boundary.md) and
|
||
[0002](../adr/0002-shared-configuration-data-model.md).
|
||
Implementation baseline: `40f8a6825359b4923645256e230a050dfdb00710`;
|
||
reviewed changes begin after `248d0db`.
|
||
|
||
## Findings
|
||
|
||
### 1. Narrow The Pre-Handoff Mutation Wording To The Accepted Expectation
|
||
|
||
Status: complete. Priority: P3. Implementation stage: 16.
|
||
|
||
Completion: narrowed constructor GoDoc and consumer guidance to the consistency
|
||
expectation; reviewed against the roadmap. No runtime behavior changed.
|
||
|
||
Locations: `engine.go:280` and `docs/consumers/pkg-promptkit.md:132`.
|
||
|
||
`NewFromConfig` says callers **must not mutate** selected settings between loading
|
||
and handoff; the consumer guide repeats that blanket instruction. The accepted
|
||
roadmap instead asks callers to hand off before changes that invalidate the
|
||
frozen inspection/provenance evidence, and expressly treats this as a usage
|
||
expectation rather than a new enforcement mechanism. The published wording
|
||
prohibits even value-preserving reassignment or copying, which needs no such
|
||
restriction.
|
||
|
||
Required change: describe the narrower consistency expectation in both owners.
|
||
Explain that construction consumes the resolved settings while inspection retains
|
||
the completed load's frozen evidence, so callers should keep those consistent at
|
||
handoff. Preserve the existing protection against mutation after construction.
|
||
No mutation detection, extra validation pass, or new error is needed.
|
||
|
||
Verification: review both passages against the feature roadmap's ownership
|
||
paragraph; ensure they no longer impose an unconditional mutation prohibition.
|
||
This is a documentation correction and does not require a new runtime test.
|
||
|
||
### 2. Keep Application-Field Sensitivity Policy With The Application
|
||
|
||
Status: complete. Priority: P3. Implementation stage: 16.
|
||
|
||
Completion: backend and profile guidance now assigns extension sensitivity and
|
||
disclosure to applications, retaining the credential/redaction warning. Reviewed
|
||
against the accepted extension contract; no runtime changes were needed.
|
||
|
||
Location: `docs/formats.md:235–236`.
|
||
|
||
The backend format reference says an unrecognized backend field is application
|
||
data and **must not contain secrets**. The accepted extension-point contract
|
||
assigns interpretation, validation, and sensitivity rules for those fields to
|
||
the application. The implementation follows that contract: inline fields remain
|
||
in native inspection, application sensitivity predicates compose with PromptKit's,
|
||
and those fields are omitted from resource reports and provider parameters.
|
||
The added prohibition unnecessarily narrows the application-owned data contract.
|
||
|
||
Required change: retain the useful warning that a raw `api_key` extension is not
|
||
a PromptKit credential input and is not automatically redacted by PromptKit.
|
||
State that the application owns sensitivity declarations and disclosure policy
|
||
for its extension fields, rather than prohibiting their contents. Review the
|
||
adjacent profile wording for the same distinction between PromptKit credentials
|
||
and application-owned data. Do not add automatic credential behavior or a new
|
||
validator for application fields.
|
||
|
||
Verification: compare the revised text with the feature roadmap's
|
||
application-specific-fields contract and existing focused-inspection sensitivity
|
||
coverage. No change to execution or redaction implementation is needed.
|
||
|
||
### 3. Preserve Effective Input Metadata Supplied By Application Defaults
|
||
|
||
Status: complete. Priority: P2. Implementation stages: 5–6 and 13.
|
||
|
||
Completion: capture reports effective required=true and preserves nullable input
|
||
metadata from typed values. Public regressions for Go defaults and recorded
|
||
collection defaults failed before the fix and pass afterward, alongside focused
|
||
race-enabled presence/normalization tests and offline preflight checks.
|
||
|
||
Locations: `internal/configuration/load.go:208–216`,
|
||
`internal/configuration/prompts.go:174–180`, and
|
||
`internal/configuration/resources.go:81–87`.
|
||
|
||
The new application-owned loading path preserves ConfigKit `Defaults` and
|
||
application resolution callbacks, but the existing input-inspection machinery
|
||
still equates `FieldInfo.Explicit == false` with an absent effective value.
|
||
`finalFieldState` consequently discards metadata for values supplied by Go
|
||
defaults or recorded derived defaults. These are valid final values, not missing
|
||
configuration.
|
||
|
||
Reproduced through the public API with a completed ConfigKit load whose
|
||
`Definition.Defaults` returns a prompt input equivalent to:
|
||
|
||
```go
|
||
promptkit.PromptInputSettings{
|
||
Name: "doc", Required: true,
|
||
ContentType: new("text/plain"), Description: new("default description"),
|
||
}
|
||
```
|
||
|
||
With a valid prompt and default profile, `NewFromConfig` succeeds. The final
|
||
typed input retains those values; native metadata describes their defaulted,
|
||
nonnull presence (`Explicit: false`, `Null: false`, default layer), but:
|
||
|
||
- `InspectPrompt` returns `Required: nil` instead of the effective true value.
|
||
- `InspectRequirements` correctly classifies `doc` as required but returns nil
|
||
content type and description, contrary to its effective-metadata contract.
|
||
- `ValidateRequest` without `doc` fails with `ErrPromptRender` and a missing
|
||
required-input error, confirming that execution uses the true value that
|
||
resource inspection fails to report.
|
||
|
||
A second public-API reproduction using an application `Resolve` callback and
|
||
`RecordDerivedDefault` for an omitted `required` and `content_type` has the same
|
||
result. Source-loaded inputs ordinarily have explicit metadata, which is why
|
||
the migrated source fixtures and current handoff tests do not expose this gap.
|
||
|
||
Required change: capture effective input metadata from the final typed values
|
||
and native final metadata without treating source explicitness as a universal
|
||
presence test. Preserve the existing ordinary omission/null and explicit
|
||
false/empty-string behavior. Keep source provenance in native inspection;
|
||
do not inspect raw sources, replay history, reject application defaults, or
|
||
rerun resolution to repair the report.
|
||
|
||
Verification: add public integration regressions for both Go defaults and
|
||
recorded application-derived defaults. Compare `InspectPrompt`,
|
||
`InspectRequirements`, and preparation/preflight for the same captured input,
|
||
and retain existing source-presence cases. Include a collection case if the fix
|
||
introduces separate root/collection handling; the current capture logic is shared.
|
||
|
||
### 4. Avoid Expanding Inheritance Provenance For Input-Presence Flags
|
||
|
||
Status: open for consideration. Priority: P3. Implementation stage: 6.
|
||
Related to finding 3; consider addressing them together.
|
||
|
||
Locations: `internal/configuration/load.go:173–178` and `208–216`.
|
||
|
||
Capture calls `Metadata.Lookup` three times per declared input, then retains only
|
||
the `Explicit` and `Null` flags. In the selected ConfigKit v1.4.0 implementation,
|
||
`Lookup` also materializes and copies the field's complete inheritance-step list.
|
||
ConfigKit keeps that evidence compact internally; the full projection is useful
|
||
for operator inspection, but unnecessary for these flags. Its public
|
||
`Metadata.IsExplicit` method performs the effective-field query without that
|
||
projection.
|
||
|
||
Complexity: for `I` input declarations and inheritance depths `h_i`, these
|
||
presence queries perform `O(I + sum(h_i))` provenance work, apart from path
|
||
processing. A chain of `N` prompts inheriting a constant-sized input declaration
|
||
has depths `0..N-1`, producing quadratic ancestry-step work and cumulative
|
||
temporary allocations merely to capture linear-sized presence data. Reading one
|
||
metadata snapshot, as selected in the plan, does not eliminate this per-query
|
||
expansion.
|
||
|
||
Refactoring opportunity: combine effective typed values/pointer presence with
|
||
`Metadata.IsExplicit` where source explicitness is actually needed. Nullable
|
||
content-type and description pointers already distinguish retained values from
|
||
absent/null metadata for their public reports. Avoid requesting a complete origin
|
||
solely to classify presence, while correcting the default-value handling in
|
||
finding 3. Presence classification can be linear in the number of declarations,
|
||
plus ordinary path-query costs, independent of inheritance-chain length.
|
||
|
||
Keep full `Lookup` operations where source origins are genuinely needed for
|
||
resource binding. This finding does not propose a different inheritance algorithm,
|
||
metadata snapshot policy, inspection contract, or source-origin fallback.
|
||
|
||
Verification: preserve inherited, omitted/null, explicit-false, empty-string, and
|
||
application-default metadata outcomes. Confirm from the call path that the
|
||
presence-only phase no longer projects inheritance steps. No timing threshold or
|
||
real-world benchmark is needed to establish that algorithmic change.
|
||
|
||
### 5. Share Immutable Backend Defaults Across Profiles Without Overrides
|
||
|
||
Status: open for consideration. Priority: P3. Implementation stage: 6.
|
||
|
||
Locations: `internal/configuration/targets.go:27–29` and `68–78`,
|
||
`internal/backend/registry.go:39–48`, and
|
||
`internal/domain/smoke_diagnostics.go:124–128`.
|
||
|
||
Every captured profile selecting a backend calls the registry's defensive-copy
|
||
lookup. That lookup deep-copies the backend's provider parameters, and the
|
||
profile retains the copied map even when it has no provider-parameter overrides.
|
||
Many profiles selecting one backend therefore retain many identical immutable
|
||
trees. This is an inherited cost reused by the new capture path, not a newly
|
||
introduced regression. The plan selects precomputed execution profiles and
|
||
ownership boundaries, not duplication of identical defaults within the owned
|
||
snapshot.
|
||
|
||
Complexity: for `P` profiles sharing one backend with `S` provider-value nodes and
|
||
no profile-level provider maps, this step costs `Theta(P*S)` copying time and
|
||
retained parameter storage, despite only `O(P+S)` distinct input data. The cost is
|
||
quadratic when both dimensions grow together. Existing ID sorting is additional
|
||
work and is not the target of this finding.
|
||
|
||
Refactoring opportunity: obtain an owned backend-default snapshot once per
|
||
backend and share its provider map read-only among profiles without overrides.
|
||
Keep separate merged maps when a profile needs its own provider parameters.
|
||
For the no-override case, default-map copying and storage can become `O(P+S)`;
|
||
the existing sorting and public result-copy costs remain unchanged. A small
|
||
construction-local cache or use of already owned immutable definitions is
|
||
sufficient; a general persistent-map framework is unnecessary.
|
||
|
||
Preserve defensive `GetBackend` semantics and copies at runtime/caller/client
|
||
boundaries. `GetProfile` already detaches the selected target's provider map, so
|
||
sharing construction-owned defaults need not expose shared mutable state to
|
||
consumers or merge caller requests into retained configuration.
|
||
|
||
Verification: review copy placement to establish the reduction, and retain
|
||
backend/profile mutation isolation, override-merge, prepared-state ownership,
|
||
and injected-client mutation coverage. Do not make internal pointer sharing a
|
||
public guarantee or introduce benchmark thresholds.
|
||
|
||
## Initial Review Results And Validation
|
||
|
||
- Stage 1: ConfigKit v1.4.0, unchanged Go minimum, and commit-pinned attribution
|
||
match the selected module and its notices.
|
||
- Stages 2–3: configuration and runtime inputs resolve mode-dependent repair
|
||
budgets; explicit zero and null/omission, whole-contract replacement, prepared
|
||
retention, and existing repair accounting are covered.
|
||
- Stages 4–9: shared settings aliases, composed stateless rules, final-metadata
|
||
resource capture, native focused inspection, explicit source mounting, and
|
||
shared constructor/assembly paths match the target architecture.
|
||
- Stages 10–15: migrated tests and maintained offline examples exercise the
|
||
changed boundaries, including concurrent loads, cross-filesystem inheritance,
|
||
application extensions, ownership, and standalone/handoff equivalence.
|
||
- Stages 16–17: canonical documentation is updated, subject to the two wording
|
||
corrections above. No additional functional or architectural defect was found.
|
||
|
||
Fresh checks passed with Go 1.27.4, `GOTOOLCHAIN=local`, and `GOWORK=off`:
|
||
`go test -race ./...` and `go vet ./...`. Changed Go files have no `gofmt`
|
||
differences, and the implementation diff passes `git diff --check`.
|
||
|
||
This review changes only this findings document; it does not implement fixes.
|
||
|
||
## Integration-Focused Follow-Up
|
||
|
||
The second review revisited stages 1–17 at their interaction boundaries: the
|
||
ConfigKit upgrade and native metadata semantics; repair-input conversion and
|
||
prepared budgets; shared settings and application callbacks; capture, inspection,
|
||
mounting, and constructor wiring; migrated tests and examples; and their contracts
|
||
with the existing runtime. Finding 3 records the newly reproduced integration
|
||
gap. Findings 1 and 2 remain unchanged.
|
||
|
||
Additional disposable external-package checks, outside the repository, passed
|
||
under the race detector for both `NewEngine` and `NewFromConfig`. They combined
|
||
configured default repair with continuation, appended messages, direct session
|
||
selection, namespace views, frozen prepared execution, structured output,
|
||
numeric-presence settings, usage/observer accounting, capacity release, and
|
||
single-pass smoke diagnostics. Both default-metadata reproductions failed their
|
||
expected inspection assertions, including under the race detector; those failures
|
||
are the evidence for finding 3, not failures in the existing test suite.
|
||
|
||
Fresh targeted race-enabled tests passed for the public handoff and runtime
|
||
contracts, configuration and use-case packages, maintained examples, capacity,
|
||
generation, observation, rendering, and validation. No other integration defect
|
||
was found. The prior full-suite and vet results remain applicable to unchanged
|
||
implementation code. No repository implementation or test files were changed by
|
||
this review.
|
||
|
||
## Algorithmic Review
|
||
|
||
The third review considered stages 1–17 in order, excluding algorithm choices
|
||
expressly selected by the plan. Findings 4 and 5 record two local optimization
|
||
opportunities; findings 1–3 remain unchanged. No additional ordinary correctness
|
||
bug was found.
|
||
|
||
- Stages 1–3: checked the selected metadata API costs, constant-time budget
|
||
defaulting/conversion, and the unchanged bounded repair/preparation paths.
|
||
- Stages 4–6: checked shared data declarations, per-resource normalization and
|
||
validation, known-value ownership traversal, presence capture, and profile
|
||
construction. Nested resource/input loops traverse distinct data; the two
|
||
findings concern hidden work in helper calls rather than loop syntax.
|
||
- Stages 7–9: native reporting remains delegated; diagnostic copying is linear
|
||
in returned records/locations, and the constructors add no repeated load or
|
||
configuration-sized work to ordinary execution.
|
||
- Stages 10–15: checked migrated test and example helpers for additional passes
|
||
or algorithm changes; they do not introduce another production loader or
|
||
runtime resource scan.
|
||
- Stages 16–17: checked the findings against the accepted ownership, inspection,
|
||
ordering, and configuration-boundary decisions. Deliberate lexical inventories,
|
||
native ConfigKit algorithms, and full-prompt repair remain outside reconsideration.
|
||
|
||
This was static complexity analysis of PromptKit and the relevant calls into the
|
||
selected ConfigKit release. No performance measurements or benchmarks were run.
|
||
Implementation code is unchanged, so prior functional validation remains
|
||
applicable; this pass validates the findings-document diff only.
|