diff --git a/docs/roadmap/audit-remediation.md b/docs/roadmap/audit-remediation.md deleted file mode 100644 index 315773c..0000000 --- a/docs/roadmap/audit-remediation.md +++ /dev/null @@ -1,646 +0,0 @@ -# Audit Remediation Roadmap - -## Purpose And Status - -Prepare PromptKit's audited implementation for a focused round of correctness, -configuration simplification, resource-use improvements, and internal cleanup. -This document defines the target and work boundaries. The companion -[implementation plan](implementation.md) records the completed 33-stage sprint -and validation. The implementation landed through `6673284`; the follow-up -[remediation record](remediation.md) owns review findings and their completion. - -Baseline: repository commit `bd3dfa9`, with the -[audit catalog](audit/catalog.md) covering 41 source findings consolidated into -34 remediation entries. All six catalog decisions were implemented in the -completed sprint. The planning assessment was completed at -`3ef2a9e` against the unchanged production baseline, with evidence gathering -starting at `21fea46`. Its recommendations, measurements, and subsequent review -refinements are consolidated here; the separate assessment document is retired. -The catalog and supporting audits own finding evidence and decision rationale. -This roadmap owns the selected end state, work boundaries, assessment evidence, -and workstream status. Earlier remediation suggestions in the source audits do -not override accepted decisions. - -All six workstreams below were implemented; the companion plan records their -stage-level completion and post-change evidence, including the overlay, feedback, -listing, and evidence-union improvements. No selected workstream was deferred. -The designs below preserve the selected target and rationale. Historical planning -evidence later in this document describes the original baseline, not current -replacement measurements. Follow-up corrections are tracked in the remediation -record; all five follow-up findings are now addressed. -Private helper names are descriptive unless identifying an existing declaration. -Catalog numbers are traceability identifiers, not a required implementation -order or 34 separate delivery stages. The scope -contains confirmed defects, documentation/test gaps, and maintainability work; -those categories do not have equal urgency or automatically become release gates. - -## Targeted End State - -PromptKit remains an application-neutral Go library with the existing public -facade and ConfigKit composition boundary. The intended changes are: - -- Programmatic backend/profile sources hand supported values to ConfigKit - directly, removing PromptKit's recursive compatibility conversion. -- Schemas supplied for validation and native structured output are self-contained - documents, while in-document references remain supported. -- Artifact-reader ownership, public errors, and smoke-test outcomes accurately - express the owning operation and collected evidence. -- Authentication response consumption is bounded by the selected internal - allowance, without treating overflow as rejected credentials. -- Identified unnecessary retention, repeated traversal, and temporary allocation - are reduced with proportionate changes that preserve established behavior. -- Production execution and its tests use the real generation coordinator; - obsolete adapters, state, helpers, and duplicate test expectations are removed. -- Current documentation describes actual behavior, and deliberate compatibility - changes are explained where consumers encounter them. - -The [resolved decisions](audit/catalog.md#resolved-decisions) are authoritative -for the selected changes. They do not authorize stronger numerical precision, -provider compatibility, scheduling, hash stability, or performance guarantees. - -## Scope Boundaries - -Keep the work within the [architecture policy](../policy/architecture.md) and -[configuration format boundary](../formats.md). ConfigKit owns source composition -and its raw-source diagnostics; PromptKit validates its final effective values. - -This round does not include a schema bundler, a new local-only schema mode, an -upstream ConfigKit extension, a numeric conversion subsystem, new inference -limits or application deadlines, or a general reflection/validation framework. -[Schema bundling](future.md#schema-bundling) remains a future idea. Runtime -provider overrides retain their distinct supported-value and copying behavior. - -Refactoring does not authorize expanding recognized template expressions, -introducing additional retries, adding pagination or runtime inventory caches, -or redesigning the capacity manager. Tests should protect established semantics, -not turn incidental implementation choices into new requirements. Existing -privacy and ownership boundaries remain applicable throughout. - -## Workstreams - -### A. Maintainer Toolchain Baseline - -**Catalog:** 1. **Decision:** 5. - -**Current gap:** The audited local toolchain predates the networking fixes -identified by the security audit. Toolchain selection and consumer compatibility -must remain separate concerns. - -**Assessed baseline:** Default `/snap/bin/go` reports 1.27.0. The assessment -found an executable Go 1.27.3 installation at -`/home/eric/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.27.3.linux-amd64/bin/go`; -other inspected installations were older. No installation is needed for the -assessed environment. ConfigKit 1.1.0, jsonschema/v6 6.0.2, and ReleaseKit 0.9.0 -remain pinned. The focused version/applicability check found that 1.27.3 excludes -the affected ranges of GO-2026-6611 and GO-2026-6809 identified in the audit. -This was not a new full scan or an assessment of consumer binaries. - -**Selected work:** Put the newest available local Go binary directory first on -`PATH` and use `GOTOOLCHAIN=local` for ordinary builds, tests, and release -validation. The existing ReleaseKit launcher already uses local selection. -Use the assessed 1.27.3 installation unless a newer local version is available -when implementation begins. Clarify this in contributor guidance. Assessment -commands also used `GOWORK=off` for isolation; that is not a new restriction on -consumer workspaces. - -Leave `go 1.27.0` unchanged for the assessed toolchain. Follow the -[release procedure](../release.md#prepare-the-candidate) and -[consumer-minimum policy](../policy/release.md#consumer-minimum-and-release-toolchain) -for later series changes and existing exceptions. Selecting a newer validation -patch does not itself raise the consumer minimum. Continuing with default -1.27.0 is unnecessary given the installed fixed patch. - -**Completion evidence:** Record the actual validation toolchain and retain the -focused reassessment above rather than repeating it without changed evidence. -Local installation work is needed only if the selected installation becomes -unavailable or new evidence calls for it. No recurring scan gate or consumer -advisory about generally applicable Go vulnerabilities is introduced. - -### B. Configuration And Schema Simplification - -**Catalog:** 3, 5, 6, 15, 21. **Decisions:** 2 and 4. - -**Current gap:** Convenience-source conversion broadens ConfigKit's accepted -input forms by recursively rebuilding them, causing silent numeric changes and -potentially exponential expansion. Schema validation loads external resources -locally but sends only the root document to the provider. Compilation is -repeated across direct/prepared and host-file/`fs.FS` paths. - -**Selected source handoff:** Assign `config.ExtraParams` directly into the -ordinary fields map in `ProfileSource` and `BackendSource`, then call -`configkit.Mapping` as now. Optional typed fields and the `[]any` definition -wrapper already use supported forms. The recursive compatibility adapter in -`provider_values.go` is retired. ConfigKit's immediate snapshot and deferred -unsupported/cyclic diagnostics make an extra shallow copy unnecessary. Do not -add a replacement converter, precision checker, memoization, or raw-source -validation pass. - -**Selected schema design:** Retain source-specific root loading, then share -`compileSchemaDocument(ctx, resourceURL, document)` for compilation. Register -the decoded root with `Compiler.AddResource`, retain the existing Draft2020 -selection, and use a context-aware loader that rejects requests for external -resources with an actionable error and no filesystem/network I/O. The compiler -continues to own URI resolution, embedded IDs, anchors, and bundled metaschemas. -Send the self-contained root document to the provider unchanged in meaning. - -The dependency probe found that unused `$defs` entries are not necessarily -compiled with the root: a denying loader alone misses their external references. -Enumerate schema locations with a small explicit table of schema-bearing -keyword positions supported by the pinned dependency, and compile those -locations through the same compiler/cache. Document the table's dependency -relationship. Keep this a location traversal, not a second resolver or general -schema implementation. A lexical ban on non-fragment references would reject -valid embedded IDs; traversing arbitrary objects would mistake annotation data -for schemas. Neither is selected. - -Cover escaped JSON-pointer locations, annotation/example objects, embedded IDs, -anchors, recursion, and dynamic references. Check embedded-resource dialect -handling explicitly rather than assuming the root dialect governs every nested -resource. Preserve existing supported dialect behavior; do not use this walk to -introduce broader dialect support or new restrictions. The probe recipe and -observed results are retained under [planning evidence](#planning-evidence). - -Direct validation invokes shared compilation only after instance decoding; -prepared validation compiles eagerly as before. Remove external-resource loaders, -not root containment/read helpers. Change runner fields/constructor parameters -and the internal engine Options validator slot to `validate.ValidationPreparer`. -Return concrete validator pointer types from internal constructors so direct -`Validate` and preparation callers both compile. Preserve nil-validator no-op, -nil-plan error, cancellation, and frozen-plan behavior. No public facade -signature change is required. - -Cover `WithSchemaFile` through public construction and preparation/validation, -including source replacement/precedence and relevant file errors. Its unused -private helper result is retired as described in F. External-resource success -fixtures become early-error cases, retaining independent root-source tests. - -**Completion evidence:** Focused source-helper cases demonstrate supported-value -handoff, immediate source capture, and ConfigKit's deferred unsupported/cyclic -value diagnostics. Include formerly converted input forms to demonstrate the -migration boundary: nested built-in maps/lists, mutation after source creation, -and representative `json.Number`, named scalar/typed container, pointer, and -cyclic inputs for both helpers. Keep optional-field and construction-order -coverage; do not duplicate ConfigKit's entire limit suite. Schema cases establish external-reference rejection before provider dispatch, retained -in-document references, unused external-definition rejection, and an outbound -self-contained constraint usable without other local files. Retain independent source-path and direct/prepared coverage. -These checks establish packaging completeness, not universal provider support. - -**Compatibility:** Explain caller conversion of `json.Number`, named scalars, -typed containers, arrays, and nested pointers into ConfigKit-supported forms. -For example, replace `[]string` with `[]any` containing strings; callers choose -and check numeric conversion themselves. Explain removal of external schema -loading. No arbitrary-precision promise or change to runtime request overrides -accompanies this simplification. - -### C. Ownership, Errors, And Diagnostic Evidence - -**Catalog:** 2, 4, 7, 8, 9, 11, 26, 27. **Decisions:** 1 and 3. - -**Current gap:** The documented artifact buffer lifetime permits races; nested -errors can obscure their owning operation; response read failures can be -misclassified; some pre-dispatch smoke failures produce a completed report; -and successful authentication responses are read without a byte bound. - -**Target properties:** - -- Artifact GoDoc, adapters, and tests align with successful `Read` transferring - the artifact and mutable body storage to PromptKit. Readers stop mutating - that storage, including retained aliases. Keep the interface signature and - existing defensive copy initially. Readers may reuse private buffers when - they return an independent copy; no release protocol is introduced. -- Preserve the owning public error category independently of nested causes, - retaining appropriate `errors.Is`/`errors.As` access. Do not require category - exclusivity: retained causes can make more than one category match. Preserve - meaningful direct-not-found versus load-failure distinctions. Use a small - internal operation error carrying category/cause with `Unwrap`/`Is` behavior. - Apply it at owning boundaries and select its category in facade translation, - including before nested capacity/provider/exhaustion projections can obscure - it. Preserve those specialized public projections and matchable child causes. - Reordering a whole-chain sentinel switch cannot express operation ownership. -- Distinguish successful-response body I/O failures from malformed JSON at both - initial decoding and end verification. Preserve the redacted transport - boundary, cancellation precedence, and genuine malformed, truncated, or - oversized-content behavior without adding retries. Record actual non-EOF - reader errors beneath the existing limited reader. On decode/end-check - failure, prefer observed context errors, retain size-overflow classification, - and otherwise propagate recorded I/O failures through `requestFailedError`. - Distinguish reader-returned errors from parser-generated syntax/type/truncation - errors, rather than treating every `io.ErrUnexpectedEOF` alike. -- Return local inference-diagnostic construction failures as `ErrInvalidRequest` - using explicit preparation-stage information. Preserve completed diagnostics - for missing credentials, capacity outcomes, and unsupported/inapplicable - operations; absence of dispatch alone does not imply an invalid request. - Add internal `GenerationEvidence.PreparationFailed bool`, set only at the - built-in `PrepareGeneration` failure return. Diagnostic orchestration checks - it after caller cancellation. Injected-client errors retain their established - semantics; no new public stage field is needed. -- Define the authentication response allowance as a named internal constant of - 1 MiB (1,048,576 bytes). Enforce against actual reads regardless of declared - length, detect overflow, and close without unbounded draining. Report an - explicit limit diagnostic with credential acceptance unknown and retain - available HTTP status/timing. Add matching public/domain reason constants - `SmokeTestReasonAuthenticationResponseLimitExceeded` with value - `authentication_response_limit_exceeded`. For HTTP 200 recognition overflow, - use existing `ExecutionFailureOther`, nil `CredentialsAccepted`, and a - completed report with nil Go error unless caller cancellation applies. - No new report field, failure enum, or public budget knob is needed. - Read at most allowance plus one byte using `io.LimitReader`/`io.ReadAll`; - check overflow before JSON recognition. Preserve 401 rejection without body - consumption and other non-200 behavior independently of unused bodies. -- OpenRouter-specific authentication request/response interpretation belongs in - the outbound adapter. Leave selection, applicability, completion, and neutral - reporting in the use case. Preserve anonymous probing and credential/body - privacy. Narrow smoke copying to actual consumers, removing unused request - cloning and replacing full-report cloning used only to detach a target. - Use an `internal/llm` authentication operation returning HTTP evidence, - optional acceptance, overflow/unrecognized indicators, and operational error; - the use case maps those facts to report outcomes. Move default endpoint - derivation to an llm helper used by selection, retaining explicit overrides. - Consume a narrow authentication capability alongside reachability HTTP, - without reverse dependencies or a plugin registry. Directly detach the - target/provider map; retain facade and active-backend copying. -- Exercise `RunRequest`'s privacy formatting and JSON omission boundary with - synthetic markers in omitted variables, bodies, and execution settings, not - IDs deliberately present in summaries. Cover `%v`, `%+v`, `%#v`, direct - String/GoString, value/pointer forms, and direct-key JSON omission. - -**Completion evidence:** Focused concurrent artifact tests follow the selected -ownership contract; they do not promise safety for readers that violate it. -Offline regressions cover composed public causes, both response read-failure -locations, construction failures with no dispatch, and authentication boundary -and overflow cases including exact limit/limit-plus-one, missing/inaccurate -lengths, closure without draining, 401 without consumption, context interruption, -and redaction. Update reason GoDoc and public conversion with the bound. Assert -diagnostic facts and privacy rather than complete incidental serialized shapes. Preserve active -backend copying, facade projection, and provider-map error behavior. - -The public-boundary outcome matrix for planning is: - -| Case | Owning outcome | Evidence preserved | -| --- | --- | --- | -| Client returns a child missing-prompt error | `ErrLLMGenerate` | Matchable child cause/category. | -| Reader returns that child error | `ErrArtifactLoad` | Matchable child cause/category. | -| Direct missing prompt/profile | Existing not-found error | No generic load-error substitution. | -| 2xx body I/O error at either decode stage | Generation/transport failure | Original read cause with redacted formatting. | -| Parser-invalid/truncated JSON, extra value, inference size overflow | Existing malformed-response behavior | No additional retry or parser-detail disclosure. | -| Built-in diagnostic preparation failure | `ErrInvalidRequest`, not a completed report | Zero dispatch and preparation cause. | -| Missing required credential or capacity outcome | Existing completed diagnostic | No inferred construction failure. | -| Authentication recognition overflow | Explicit limit reason; acceptance unknown | HTTP/timing facts, not secret-bearing response data. | -| Caller cancellation | Existing cancellation precedence | No publication of success over cancellation. | - -### D. Resource Use And Inspection Efficiency - -**Catalog:** 10, 16, 17, 18, 19, 20. - -**Current gap:** The audited paths repeatedly copy requirement evidence, retain -unneeded backing storage or engine ownership, serialize unbounded diagnostic -text before truncation, reclone overlay descendants, and scan complete catalogs -for namespace-filtered listings. - -**Selected remedies and completion evidence:** - -- **Requirement evidence (10):** Intermediate missing-name sets are owned by - their evaluation results. Merge the smaller set into the larger, transfer nil - sets directly, and combine flags as now. Operands subsequently read values and - known flags, not independent evidence snapshots. This avoids repeated growing - copies without a global accumulator that could mix branch-local evidence. - Preserve union/flag outcomes and conservative inspection coverage with - representative left-heavy cases and independent renderer comparisons. -- **Prepared handles (16):** Replace `owner *Runner` with a pointer to an immutable, - non-zero-sized private identity object allocated once per runner constructor. - Keep the token after claim/discard so foreign-owner checks retain their order - without retaining the runner graph or adding mutable owner-field races. - Retain mutex-protected state transitions/payload clearing and detached details. - Cover foreign owners, namespace views, concurrent claim/discard, one-shot use, - and details after use/discard. A GC-timing assertion is unnecessary. -- **Repair feedback (17):** Attempt bounded construction of the complete-list - result; if it cannot fit, construct the omission form with existing order and - reservation rules. Normalize only potentially useful prefixes, preserving - `strings.ToValidUTF8` behavior for invalid-byte runs and rune-safe truncation. - Use standard `json.Marshal` on bounded normalized strings and existing prefix - selection rather than a custom escaper. A limited writer around `json.Encoder` - does not suffice if encoding first allocates the whole original string. - Preserve the 64 KiB output rule, omission marker, valid JSON, HTML/control - escaping, Unicode, complete stored diagnostics, and previous candidate. - Cover fitting/exact-boundary/oversized/many-entry/escaped/invalid inputs. -- **Session IDs (18):** Apply `strings.Clone` to the validated nonempty normalized - value before returning from `NormalizeSessionID`. Preserve trimming, Unicode, - accepted content, and wire/hash values without adding a raw-input limit. -- **Provider overlays (19):** Clone the lower map once, then recurse into owned - descendants for overlapping built-in maps and copy only upper replacements. - Preserve null/array replacement, accepted types, error boundaries, and input - isolation. This directly removes the repeated descendant cloning measured in - the assessment without an index or framework. -- **Namespace listings (20):** Resolve scope in the engine, then pass private - `configuration.IdentityFilter{All bool, Namespace string}` to captured - inventories. All copies the sorted slice once; named selection binary-searches - the exact `namespace + "/"` prefix range and copies only matches. Unqualified - selection scans immutable state directly, without first copying everything. - Update the private inventory interface/fakes for backend/profile IDs and - prompt identities. Preserve final context checks, ordering, exact prefix - matching, absent results, view/default/All scope, and mutation isolation. - -Existing sorted inventories supply named ranges without persistent buckets or -extra construction traversal. Buckets could reduce repeated lookup work but add -retained state even for consumers that never list resources. Unqualified entries -need not form a contiguous range; their selection remains a scan. No new public -signature, cache, pagination, or complexity guarantee accompanies this remedy. - -The [baseline measurements](#planning-evidence) support implementing 17, 19, and -20 rather than deferring them. Repeat comparable cases after implementation; -expected savings and algorithmic estimates are not measured replacement results. -No exact peak-heap promise, benchmark threshold, or recurring release gate is -introduced. If new evidence changes the tradeoff, record it and the proposed -disposition rather than silently dropping selected work. The companion plan -coordinates the related traversal and trusted-copying changes in E. - -### E. Internal Coordination And Test Assembly - -**Catalog:** 22, 23, 24, 25, 34. **Decision:** 6. - -**Current gap:** Large functions combine distinct responsibilities, JSON copying -is duplicated across boundaries with different policies, and a test-only legacy -capacity wrapper remains a production constructor. - -**Work:** - -- Extract focused configuration validation/materialization helpers with visible - inputs and ownership. Preserve final-handoff validation, presence/null rules, - origins, and meaningful error precedence without pinning incidental ordering - of independent failures. Keep `validateEffective`'s resource/index traversal - visible and extract backend/profile/prompt validators with record/path/index - inputs. Keep typed presence interpretation together in profile construction. - Split prompt capture into input/metadata projection, message content/template - capture, and output-contract conversion; retain lazy origins, source-relative - reads, and one final inspection/hash assembly. -- Separate requirement recognition/evaluation from reachability and evidence - bookkeeping, sharing `if`/`with` orchestration where appropriate. Preserve - scope differences, inactive computed-name behavior, and independent comparison - with the actual renderer. Share the existing `parse.BranchNode` protocol, - handling With's changed body dot and unchanged else dot explicitly. Preserve - known/skipped/unknown classification and limitation bookkeeping; no generic - visitor or additional template syntax is needed. -- Consolidate structural copying only where policies genuinely agree. Keep - facade projection and runtime validation distinct; preserve existing types, - nils, supported numbers, schema keys, and ownership. Infallible projection - must not acquire new errors, panics, or silent fallbacks. Retain separate walkers - if sharing would require a configurable reflection framework. B's removed - source compatibility adapter must not migrate into this shared machinery. - Put infallible cloning in `internal/jsonvalue.CloneTrusted(any) any` and route - facade nested values through it. Runtime `Copy`/`CopyMap` validate, normalize - indirection/null, and enforce limits/key policy; trusted cloning preserves - accepted values and existing fast-path/reflection type/nil behavior without - interpreting numbers. Do not substitute fallible `Copy` and hide errors. - This centralizes ownership, not necessarily the traversal algorithm: two small - walkers remain appropriate where their policies differ. -- Separate generation attempt results and accounting from dispatch/retry control - flow with small internal helpers. Preserve built-in/injected differences, - permit release before callbacks or backoff, frozen requests, existing retry - classification, private evidence, and known/unknown usage distinctions. - Replace execute's parallel returns with `attemptResult` containing - `Outcome llm.AttemptOutcome`, `Dispatched`, `CapacityWait`, `RequestDuration`, - and `StartedAt`. Share usage/evidence/event projections only where facts match; - built-in retry and injected one-shot control flow remain distinct. C's - preparation-failure marker is independent of attempt accounting. -- The initial/repair runner scheduling test uses the production generation - coordinator via `generation.NewInjected` with the same manager. The production - `internal/capacity/client.go` wrapper and its retention prose are retired. - Preserve useful FIFO, cancellation, independent-pool, and panic-release scenarios with their - owning components. Keep any useful one-shot scaffolding test-only; do not wrap - the coordinator in another permit-acquisition layer. - -**Completion evidence:** Existing meaningful behavioral tests and focused cases -for changed seams continue to exercise real internal collaborators. Tests no -longer require the retired production wrapper. Review demonstrates simpler -ownership/control flow; lowering complexity metrics alone is not an acceptance -criterion. No capacity-manager redesign or public DTO unification is needed. - -### F. Documentation, Representation, And Test Maintenance - -**Catalog:** 12, 13, 14, 28, 29, 30, 31, 32, 33. - -**Current gap:** Several comments contradict implemented behavior, expectations -are duplicated or over-specified, and internal fields/helpers have no consumer. - -**Work:** - -- Correct credential-inspection GoDoc and its example: explicit profile credential - environment selection can coexist with required credentials; only the - backend-derived default is suppressed in that case. Correct provider-map wire - documentation to distinguish configured recursive overlays from runtime - replacement. Remove stale trimmed-ID and prompt-version comments. -- Give complete bundled-catalog expectations one independently specified test - owner. Retain smaller tests for distinct loading, wire, and smoke behavior, - using controlled fixtures where bundled content is irrelevant. Keep the full - independent oracle in `internal/configuration/bundled_test.go` and replace its - formatted-map plugin comparisons with structural assertions. Public tests own - toggles/order/isolation/overrides; use a controlled model/plugin wire case and - a configured-default smoke case instead of repeating the complete matrix. -- Replace literal artifact-hash algorithm assertions with documented opaque - equality/ownership and byte/size behavior; keep benchmark setup outside timed - work. Use same-body inline/file equality for size-hint cases and prepare an - opaque expected hash outside benchmark timing. This does not require changing - the artifact hash algorithm. -- Remove unused backend presence records and `QueueCapacitySet`, preserving - effective queue defaults, explicit zero, provenance, and needed profile/input - presence tracking. Remove the unused backend field list/map, resolution capture, - and runtime queue-presence flag, not ConfigKit provenance. -- Remove duplicated captured prompt output-format and unused message-file fields. - Preserve actual `content_file` loading, origins, and override-default behavior. - Account for observable definition-hash changes without promising permanent - encoding or bit-identical hashes across this change. Remove the unreachable - fallback with the fields, preserving empty-format request defaulting and - inspection/preparation equality. -- Remove unused private reachability endpoint state and the unused schema-helper - result, preserving actual report evidence and public single-file behavior. - Remove orphan `filecatalog.RelativePath`, `resourceNamespaceFromID`, and the - unused runner-test `stringPtr` helper with their obsolete self-tests. - -**Completion evidence:** Documentation agrees with the owning GoDoc/format/wire -contracts. Caller checks precede deletion; meaningful discovery, identity, -namespace, schema, privacy, and bundled behavior remain covered. Tests do not -merely reproduce the loader's output as their expected answer. - -## Implementation Plan - -The [companion plan](implementation.md) defines the ordered, prompt-sized -implementation units, dependencies, and validation for one end-to-end sprint. -Workstream letters group target outcomes, not delivery stages. Temporary -intermediate compilation gaps are acceptable; completion concerns the combined -end state. No significant product decisions remain open. Equivalent private -coding details remain flexible; new evidence requiring a scope or contract -change should be surfaced rather than silently changing the selected target. - -## Compatibility And Documentation Delivery - -The selected ownership, schema, and convenience-source changes are deliberate -pre-1.0 compatibility changes. Deliver them in a minor release under the existing -[release policy](../policy/release.md#versioning-and-compatibility), with migration -notes explaining independent artifact storage, self-contained schemas, and -ConfigKit-supported input types. Mention changed definition hashes so consumers -can account for comparisons or caches; do not establish a stable encoding. -Release selection and publication are not part of writing this roadmap. - -Update exported GoDoc, [consumer guidance](../consumers/pkg-promptkit.md), -[formats](../formats.md), [wire integration](../integrations/openai-compatible-chat.md), -and relevant [internal documentation](../internal/overview.md) when their owning -behavior lands. Clarify ordinary development toolchain selection in -[development guidance](../development.md), linking to existing release policy. -Correct standalone documentation defects without waiting for unrelated code. - -## Planning Evidence - -These are historical assessment results from unchanged production code at the -baseline above, using Go 1.27.3 on linux/amd64, Intel i5-7260U. Assessment records -were completed in commit `3ef2a9e`. Dependency/source inspection supplied the -ownership, caller, and interface evidence; temporary offline probes supplied the -schema results and measurements below. No replacement implementation was built -or benchmarked. Aggregate allocation is not peak live heap, and noisy timings -are not throughput promises. - -The temporary harness directory was `/tmp/promptkit-assessment-aqn3B8`. Its -continued existence is not a planning dependency: the reproduction recipes below -retain the workloads and invocation settings. Recreate targeted harnesses outside -the repository or carry equivalent cases into focused implementation benchmarks, -not a new permanent benchmark framework. Reuse unchanged evidence unless a -changed dependency, workload, or implementation makes a new measurement useful. - -### Schema Dependency Probe - -With jsonschema/v6 6.0.2, decode each document with `json.Decoder.UseNumber`, -create a fresh compiler with `DefaultDraft(Draft2020)`, register the document at -`https://example.invalid/retrieval`, and install a loader that counts requests -and returns an external-resource error. Compile the retrieval URI; for the -unused-definition case, additionally compile `#/$defs/unused` with the same -compiler. The harness used `go run -mod=readonly` with -`GOTOOLCHAIN=local GOWORK=off GOPROXY=off`; no network resources were fetched. - -| Document case | Root compilation | External loads | -| --- | --- | --- | -| `$defs.v` string schema referenced by `#/$defs/v` | Succeeds | 0 | -| `$anchor: v` in a definition, referenced by `#v` | Succeeds | 0 | -| Object definition with `next` referring recursively to that definition | Succeeds | 0 | -| Explicit absolute root ID and reference to its definition through that ID | Succeeds | 0 | -| Embedded definition with relative `$id: child`, referenced as `child` | Succeeds | 0 | -| Root `$ref: child.json` with no embedded resource | Fails | 1 | -| Root string schema with unused `$defs.unused: {$ref: child.json}` | Succeeds; explicitly compiling the definition fails | 0, then 1 | -| Example instance object containing literal `$ref: child.json` | Succeeds | 0 | -| Standard 2020-12 `$schema` declaration | Succeeds | 0 | - -Explicit-ID cases also worked when retrieval URI and document ID differed. A -review rerun reproduced these results. This supports B's rejecting loader plus -schema-location enumeration, not a lexical URI ban. Dynamic references, escaped -pointers, embedded dialect handling, and full location-table coverage remain -implementation regression cases, not claims established by this small probe. - -### Overlay Baseline - -Construct maps outside timing. Start with -`map[string]any{"leaf": 1, "array": []any{1, 2, "x"}}`; wrap it `depth` times in a map with -`child` pointing to the previous level and `width` scalar keys named `key0`, -`key1`, etc., each with its index value. Build lower and upper independently. -For the disjoint case, wrap upper once more under `new`; otherwise use matching -keys. Call unchanged `overlayProviderMap`, retaining its result and checking -errors. These are chains with scalar siblings, not exponentially branching trees. - -Run a temporary test overlay in `internal/configuration`, substituting its -absolute JSON mapping path: - -```sh -go test -mod=readonly -overlay=/absolute/path/overlay.json ./internal/configuration \ - -run '^$' -bench '^BenchmarkAssessmentOverlay$' \ - -cpu=1 -benchmem -benchtime=50x -count=3 -``` - -The overlay maps an absolute package-local test filename to an external harness -file declaring that package; it does not edit repository source. The same -`GOTOOLCHAIN=local GOWORK=off GOPROXY=off` settings apply to all measurements. - -| Depth / width / overlap | Observed ns/op range | B/op | Allocs/op | -| --- | --- | --- | --- | -| 2 / 2 / overlapping | 20,104–34,618 | 9,496 | 236 | -| 12 / 2 / overlapping | 263,387–287,814 | 117,869 | 2,609 | -| 40 / 2 / overlapping | 2,355,659–2,580,220 | about 1,241,620 | about 22,672 | -| 4 / 16 / overlapping | 168,483–178,889 | 65,434 | 1,591 | -| 12 / 2 / disjoint upper root | 49,437–53,966 | 32,496 | 686 | - -Source inspection shows lower descendants cloned again at each recursive -overlap. The selected owned-overlay algorithm removes that repeated work; -post-change measurements must establish actual savings. - -### Repair Feedback Baseline - -Call unchanged `formatRepairDiagnostics` with one-element string slices: -64 ASCII `x` bytes, 1 MiB of `x`, and 1 MiB of repeated `<&>` plus NUL (four bytes -per repetition). Build inputs outside timing, retain the output in a sink, and -record input/output lengths after stopping timing. Use the overlay method above -in `internal/usecase`, benchmark `BenchmarkAssessmentFeedback`, with -`-cpu=1 -benchmem -benchtime=20x -count=3`. - -| Input | Observed time per operation | Allocated bytes per operation | Output bytes | -| --- | --- | --- | --- | -| 64 ASCII bytes | 780–1,198 ns | 224 | 68 | -| 1 MiB ASCII | 5.83–7.99 ms | about 6.81 MB | 65,536 | -| 1 MiB escaped content | 27.7–39.1 ms | about 44.12 MB | 65,531 | - -Full-string normalization and repeated full marshaling account for the observed -allocation scale. D's bounded-prefix design must preserve the existing fitting -result and omission semantics; these timings do not measure the new design. - -### Listing Baseline - -For N = 32, 1,024, and 4,096, create sorted IDs `namespace/p%06d` outside timing. -Sparse mode assigns one ID to `target`; dense assigns half. Other IDs use `other`. -All mode uses the sparse inventory with the All filter; absent mode selects -`missing`. The baseline list callback returns `append([]string(nil), ids...)`, -matching configuration's full-inventory copy. Invoke unchanged -`filterIdentityList` with background context and empty view namespace. Use an -`internal/engine` overlay, `BenchmarkAssessmentListing`, and -`-cpu=1 -benchmem -benchtime=100x -count=3`. - -| Total IDs | Mode order | ns/op ranges in mode order | B/op | Allocs/op | -| --- | --- | --- | --- | --- | -| 32 | All / sparse / dense / absent | 1,176–1,332 / 759–937 / 1,053–1,875 / 797–819 | 1,472 / 528 / 960 / 512 | 5 / 2 / 4 / 1 | -| 1,024 | All / sparse / dense / absent | 41,851–48,965 / 13,860–19,334 / 38,416–45,031 / 15,258–19,282 | 78,144 / 18,448 / 37,184 / 18,432 | 11 / 2 / 9 / 1 | -| 4,096 | All / sparse / dense / absent | 194,479–202,439 / 92,265–95,215 / 148,209–151,180 / 93,157–94,526 | 305,472 / 65,552 / 166,208 / 65,536 | 14 / 2 / 12 / 1 | - -This measures filtering/copying, not full construction or public projection. -For post-change comparisons, exercise D's selection-before-copy path rather than -retaining an artificial full-copy callback. Repeated requests repeat the -baseline full-inventory allocation. Binary search over existing named ranges -has estimated comparison work proportional to log N plus output and introduces -no persistent index; no alternative layout was benchmarked. - -### Requirement Evidence Work Count - -For a left-heavy recognized condition with N distinct missing names, the current -merge copies growing sets: `2 + 3 + ... + N` insertions, excluding leaf creation. -That is 9 at N=4, 135 at N=16, and 8,255 at N=128; the ordinary two-name case -needs two. These are source-derived counts, not elapsed-time measurements. The -selected small-to-large owned merge addresses that specific repeated copying; -it does not establish a universal inspection complexity contract. - -## Validation And Completion - -Follow the [testing policy](../policy/testing.md) and existing -[maintainer validation workflow](../development.md#maintainer-validation). -Use focused behavioral regressions for demonstrated defects, race testing for -changed concurrent ownership/scheduling paths, and offline wire tests where -local success does not prove the emitted request is correct. Reuse successful -checks for unchanged content; release validation remains the existing workflow. -No live inference, provider credentials, generalized benchmark suite, or new -recurring release gate is introduced. - -The workstream catalog lists partition all 34 entries. During implementation, -record for each workstream its completed units, evidence, and any explicitly -justified deferral. A measured deferral is not an implemented fix; optional -cleanup does not hold unrelated fixes hostage. Full implementation completion -requires the selected changes, their relevant validation, migration guidance, -and removal of obsolete documentation/tests. Any remaining work must be named -and retained in an appropriate roadmap rather than lost during consolidation. - -When this roadmap's work is complete or otherwise explicitly disposed of, move -still-useful contract information to its canonical owners and retire temporary -planning documents under the [documentation lifecycle](../policy/documentation.md#document-lifecycle). -Keep audit evidence and decision provenance discoverable while they remain -useful; do not maintain competing current-behavior references. diff --git a/docs/roadmap/audit/01-correctness.md b/docs/roadmap/audit/01-correctness.md deleted file mode 100644 index 3795005..0000000 --- a/docs/roadmap/audit/01-correctness.md +++ /dev/null @@ -1,131 +0,0 @@ -# Functional Correctness Audit - -Reviewed 2026-11-04 against commit `d9f6aad`. - -## Scope And Evidence - -Reviewed the public facade and conversions; ConfigKit handoff, normalization, -presence-aware target assembly, collections and namespace selection; artifact -loading and template rendering/requirement discovery; ordinary and single-use -prepared execution; output validation and repair; backend admission and active -permits; HTTP request/response handling and transient retries; execution -observations; and smoke-test selection and reporting. Contracts were checked -against GoDoc, the format and integration references, internal documentation, -and relevant existing tests. Source loading before the ConfigKit handoff remains -ConfigKit's responsibility. - -Validation: - -- `go test -race -vet=off ./...` passed, including the maintained examples. -- `go vet ./...` passed. -- An offline, temporary public-API reproduction outside the repository confirmed - the number conversion and captured schema payload described below. It used - `ProfileSource`, `Prepare`, a runtime override, and `WithSchemaFS` with - `fstest.MapFS`; no provider was contacted. - -No production code or tests were changed. This audit does not establish provider -availability or acceptance of arbitrary JSON Schema features. - -## 1. Provider Source Helpers Silently Change Exact JSON Numbers - -**Priority:** P2. Confirmed data-conversion defect. - -**Code:** [providerValueForMapping](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/d9f6aad/provider_values.go#L51), used by -[ProfileSource](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/d9f6aad/profiles.go#L33) and -[BackendSource](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/d9f6aad/backends.go#L100). - -**Circumstances:** A caller supplies a valid `json.Number` in provider -`ExtraParams` that cannot be represented exactly by the helper's chosen native -number type. After signed and unsigned integer parsing fail, the helper converts -the value to `float64` without checking whether that conversion loses precision. -The same problem affects sufficiently precise fractional values. - -**Expected:** A source helper preserves the meaning of a supported provider -value, or reports that the value cannot be represented. It must not silently -substitute a different number. This concerns PromptKit's own convenience-source -conversion, not additional validation of ConfigKit's raw sources. - -**Actual / reproduction:** With -`ExtraParams: map[string]any{"number": json.Number("18446744073709551617")}` -on a profile, `NewEngine` succeeds. JSON encoding of -`Prepare(...).EffectiveModelParams.ExtraParams` produces: - -```json -{"number":18446744073709552000} -``` - -Supplying the identical value through `RunRequest.Execution.ExtraParams` -instead produces: - -```json -{"number":18446744073709551617} -``` - -The built-in client serializes those effective values directly into the request, -so configuration-source selection changes the actual provider parameter. -`BackendSource` shares the same converter. No high-precision numeric preservation -claim needs to be added for arbitrary source formats to correct this discrepancy. - -**Potential remediation:** Preserve exact numbers through a numeric -representation supported by ConfigKit, if available. If that contract cannot -represent a particular `json.Number` exactly, surface a deferred source error -rather than returning a rounded float or a numeric-looking string. Check numeric -value equivalence rather than textual spelling: alternate exponent spellings -need not be preserved. Add focused regression coverage for both source helpers -and contrast with the already-preserving runtime path. - -## 2. External Schema Resources Do Not Reach The Provider Constraint - -**Priority:** P2. Confirmed schema-packaging gap; remediation requires choosing -how provider-facing schemas accommodate the existing local-reference feature. - -**Code:** [preparedValidation.SchemaDocument](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/d9f6aad/internal/validate/standard_validator.go#L63), -[StandardValidator.PrepareValidation](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/d9f6aad/internal/validate/standard_validator.go#L74), -[FSValidator.PrepareValidation](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/d9f6aad/internal/validate/standard_validator.go#L126), -and [structuredOutputFromValidationPlan](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/d9f6aad/internal/usecase/runner.go#L419). - -**Circumstances:** A JSON Schema output contract uses a root schema referring to -another file within the configured schema source. The -[schema format contract](../../formats.md#schemas) supports such references. -Preparation compiles the transitive resources locally, but `SchemaDocument` -exposes only the unchanged root document. The runner always uses that document -for the native provider constraint in JSON Schema mode. - -**Expected:** The provider-facing constraint should carry the schema resources -needed to interpret the supported local references, or there should be an -explicitly chosen way to retain local validation without submitting an -unresolvable native constraint. This is not a promise that every provider accepts -every schema dialect feature. - -**Actual / reproduction:** With these files under `WithSchemaFS`: - -```text -root.json: {"type":"object","properties":{"x":{"$ref":"child.json"}},"required":["x"]} -child.json: {"type":"integer"} -``` - -`Prepare` succeeds, but `PreparedRun.StructuredOutput.JSONSchema.Schema` contains -only the root object, including the literal `$ref: "child.json"`. No child -document, bundled definition, or reference rewrite is present. The HTTP adapter -passes that object unchanged to `response_format.json_schema.schema`. An upstream -provider has no access to the application filesystem or private `fs.FS`, so it -cannot resolve that reference from the request. A provider requiring resolvable -native schemas can reject generation before local validation or repair is -reached. No live-provider rejection was tested. - -The internal source document accurately describes the existing root-only -projection; the gap is between the supported local schema-resource graph and -its mandatory projection into an outbound constraint, rather than a failure of -local compilation or prepared-resource freezing. Self-contained schemas are -unaffected. - -**Potential remediation:** Produce a self-contained provider schema from the -operation's captured resources, preserving reference semantics, identifiers, -anchors, and recursion. Avoid naive recursive textual substitution. An -alternative is an explicit choice to use local-only schema validation when -native submission cannot represent the resource graph. Any alternative that -restricts existing schema sources or silently disables native structured output -requires a deliberate compatibility/policy decision. Protect the selected -behavior with an offline test combining transitive schema preparation and the -actual outgoing request, while retaining frozen local validation for -`RunPrepared` and repair. diff --git a/docs/roadmap/audit/02-efficiency.md b/docs/roadmap/audit/02-efficiency.md deleted file mode 100644 index 72d8479..0000000 --- a/docs/roadmap/audit/02-efficiency.md +++ /dev/null @@ -1,147 +0,0 @@ -# Algorithmic Efficiency Audit - -Reviewed 2026-11-04 against commit `0d27252`. - -## Scope And Method - -Reviewed PromptKit-owned configuration assembly, provider-value conversion and -copying, rendering and requirement analysis, inventory selection, prepared-run -ownership, validation orchestration, repair formatting, retry coordination, -observations, and capacity management. Findings below follow from implementation -and operation counts; no benchmarks or live-provider calls were needed. - -The public facade delegates to internal components. ConfigKit supplies the final -configuration; PromptKit constructs immutable runtime indexes and captured -templates. Execution uses operation-local validation plans, shared backend -capacity, and a generation coordinator. Inspection reads the captured snapshot. -ConfigKit and ReleaseKit internals are outside this repository audit. - -### Deliberate Choices Preserved - -Checked current internal documentation and relevant retired plans in Git history, -including the performance plan at `8a5c0de`, configuration adoption at `5f7b194`, -and collections at `89af2a1`. This audit does not reopen: - -- lexical inventory ordering, recursive provider-map overlay semantics, or - conservative requirement-discovery semantics; -- captured parsing, per-render artifact-text reuse, operation-local schema - compilation, frozen retry payloads, and linked-list FIFO permits with - constant-time cancellation removal; -- the existing JSON depth/work bounds, output/diagnostic limits, or copies - deliberately retained to protect public and injected-collaborator ownership; -- synchronous filesystem/template/schema operations or the selected retry and - repair budgets. - -The opportunities below concern implementation work that those choices do not -require. Complexity statements use expected constant-time Go map operations and -treat identifier comparison/hashing cost separately. No new public performance -guarantee, cache, caller limit, or release gate is proposed. - -## 1. Condition Evidence Unions Can Require Quadratic Work - -**Priority:** P2. Requirement-inspection path; most relevant to deeply nested -conditions rather than ordinary small templates. - -**Code:** [evaluateCommand](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/0d27252/internal/prompt/requirements.go#L142) and -[mergeConditionEvaluations](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/0d27252/internal/prompt/requirements.go#L318). - -Each comparison recursively evaluates its operands, then allocates fresh -`missingVariables` and `contentInputs` maps and copies both operands' accumulated -evidence. A left-heavy expression with distinct missing caller keys, such as -successively nesting `eq (eq .v0 .v1) .v2`, carries an increasingly large missing -key set through each ancestor. Template parsing accepts these expressions; -inspection can encounter them even when their eventual execution would fail or -the inspection result is partial. - -For an expression of size n, successive unions copy approximately -2 + 3 + ... + n entries: **Theta(n²) total set work and cumulative allocation**. -The final unique evidence has only O(n) entries. This is separate from the -previously fixed repeated syntax-subtree scan: visiting each syntax node once -does not eliminate repeated copies of the evidence produced by those nodes. - -**Opportunity:** Transfer ownership of temporary sets and merge the smaller set -into the larger one, rather than copying both at every comparison. For this -expression family that is O(n); small-to-large merging gives O(n log n) aggregate -set-insertion work across general expression trees. Another option is an -operation-local evidence accumulator that records each discovery directly, -while keeping only condition truth/limitation state on recursive return values. -Choose the simpler approach after checking temporary-set ownership. Preserve -current grammar, classifications, deduplication, sorted output, cancellation, -and partial-result behavior; interpreting more expressions is not necessary. - -## 2. Recursive Provider Overlay Repeatedly Clones Descendant Subtrees - -**Priority:** P3. Construction-time improvement with an existing depth bound. - -**Code:** [overlayProviderMap](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/0d27252/internal/configuration/targets.go#L131). - -The helper deep-copies the entire lower map before considering overrides. When -both values at a key are maps, it recursively invokes itself on a descendant -that was already copied, and copies that whole descendant again. The first copy -is then replaced and discarded. All descendants on an overlapping path can be -copied once per ancestor. - -Let N be the lower tree's node count and D its overlapping map depth. Node-copy -work can reach **O(ND)**. For a narrow chain it is the triangular sum -D + (D-1) + ... + 1. A wide K-entry leaf below D overlapping ancestors is cloned -roughly D times instead of once. `jsonvalue.CopyMap` also sorts map keys, so the -same wide leaf's sorting work is repeated at those ancestors. - -The existing maximum container depth of 100 is important: this is **not an -unbounded quadratic-in-N claim under today's fixed limits**. It is an avoidable -depth-dependent multiplier in traversal and cumulative allocations within those -limits, multiplied again when multiple profiles overlay the same backend. That -bound and the recursive-map/value-replacement semantics remain selected choices. - -**Opportunity:** Clone the lower tree once, then overlay into the owned clone -without re-cloning existing descendant maps. Copy incoming replacement values -where needed to preserve isolation from configuration inputs. Alternatively, -construct the merged output in one traversal. Either removes the repeated -ancestor copies: expected O(N + M) merge traversal for lower/upper trees of sizes -N and M, apart from the retained validation/sorting costs. Preserve null and -array replacement, empty-map behavior, input ownership, and existing JSON -validation/bounds; do not replace ConfigKit's configuration machinery. - -## 3. Filtered Listings Copy And Scan The Complete Inventory - -**Priority:** P3. Scalability opportunity for engines with many namespaces. - -**Code:** [filterIdentityList](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/0d27252/internal/engine/engine.go#L306), -[ListPrompts](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/0d27252/internal/engine/engine.go#L270), and the -[configuration inventory methods](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/0d27252/internal/configuration/listing.go#L14). - -A named-namespace request first obtains a detached copy of all N identities and -then scans it to build its k matching results. Even an absent namespace costs -**Theta(N) time and O(N) temporary storage**, rather than work proportional to -the selected inventory. Listing C namespaces separately costs Theta(CN); when -each has a bounded number of entries, that becomes quadratic in total inventory -size despite only O(N) total returned entries. - -The collection implementation plan at `89af2a1` expressly allowed namespace -buckets or sorted ranges to avoid repeated full scans; it did not require the -present whole-inventory-copy algorithm. The lexical ordering and caller-owned -result contracts do not prevent indexed selection. - -**Opportunity:** Capture per-kind namespace buckets or ranges alongside existing -construction-time inventories, then copy only the selected entries. Buckets -allow expected O(1 + k) selection/copy; binary search over sorted full IDs can -support named-prefix ranges in O(log N + k). Keep unqualified and all-ID modes -correct without assuming unqualified entries form one contiguous lexical range. -Resolve filters before requesting the inventory, and retain exact namespace -matching, full IDs, ordering, context behavior, and independent returned slices. -An all-ID request must still spend O(N) to return N owned entries; that is not a -defect. No public pagination API or mutable cache is needed. - -## Review Outcome - -These three opportunities are supported by direct operation-count analysis; -they are not measured latency claims. The largest asymptotic issue is growing -condition-evidence copying. The overlay finding is explicitly bounded, and the -listing finding matters as independent namespace count grows. - -No additional ordinary correctness defect was established in this pass. The -findings in [the correctness audit](01-correctness.md) remain separate and were -not duplicated here. No production code, tests, dependencies, or established -behavior were changed, so the successful race-suite/vet evidence recorded there -still applies to the unchanged implementation; those checks were not rerun just -for this documentation change. diff --git a/docs/roadmap/audit/03-complexity.md b/docs/roadmap/audit/03-complexity.md deleted file mode 100644 index a0ed855..0000000 --- a/docs/roadmap/audit/03-complexity.md +++ /dev/null @@ -1,216 +0,0 @@ -# Complexity And Maintainability Audit - -Reviewed 2026-11-04 against commit `5adc1c1`. - -## Method And Architectural Assessment - -Used the local codebase-memory graph's cyclomatic/cognitive complexity, -function-length, and call-relationship tools to locate candidates. Reviewed -functions and methods, including test hotspots, rather than relying on -function-only rankings. Inspected implementation, callers, owning interfaces, -and current architecture/internal documentation before selecting findings. -Metrics are analyzer-specific indicators, not correctness proofs, acceptance -thresholds, or requirements to split every large switch. - -PromptKit's overall boundaries remain useful: a public facade adapts values and -errors; configuration interprets ConfigKit's final handoff and captures runtime -indexes; the runner coordinates preparation and execution; generation owns -attempts/retries; capacity owns admission and permits; transport and schema -adapters own external protocols. No repository-wide architectural rewrite or new -public abstraction is warranted by this review. - -Representative hotspot metrics from the current graph: - -| Function | Cyclomatic | Cognitive | -| --- | ---: | ---: | -| `requirementWalker.visit` | 50 | 137 | -| `validateEffective` | 39 | 132 | -| `requirementWalker.evaluateCommand` | 26 | 103 | -| `providerValueForMapping` | 33 | 84 | -| `jsonvalue.copyValue` | 29 | 66 | -| `loaded.buildExecutionProfiles` | 27 | 58 | -| `loaded.materializePrompt` | 30 | 52 | -| `Coordinator.generate` | 24 | 49 | - -The findings below are behavior-preserving refactoring opportunities, not -confirmed bugs or automatic release blockers. They cover both package-local -complexity and interactions between packages. No production code, tests, or -dependencies were changed, and no empirical testing or benchmarks were needed. - -## 1. Consolidate Schema Compilation And Express The Runner's Actual Interface - -**Code:** [validator interfaces](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5adc1c1/internal/validate/validator.go#L10), -[schema implementations](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5adc1c1/internal/validate/standard_validator.go#L45), -[runner preparation](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5adc1c1/internal/usecase/runner.go#L398), and -[engine assembly](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5adc1c1/internal/engine/engine.go#L133). - -**Complexity:** The validator maintains direct validation and prepared -validation, with separate filesystem and `fs.FS` compilation paths. Compiler -construction, resource registration, error wrapping, and cancellation checks -are repeated across direct schema-validation methods and `PrepareValidation`. -The package's constructors return `Validator`, although the runner actually -requires `ValidationPreparer` and discovers it through a runtime type assertion. -An implementation satisfying the declared constructor dependency can therefore -still be unsuitable for the execution path. - -**Opportunity:** Make the runner's preparation dependency explicit in its -internal field/constructor types, updating assembly and focused test -collaborators together. Retain the direct validation interface where callers -actually need it. Consolidate compilation behind one internal operation, with -small source-specific loading/URL/loader adapters; direct and prepared callers -can reuse that operation rather than separately implementing registration and -compilation. No public validator API or generic source framework is needed. - -**Preserve:** Direct validation currently rejects malformed JSON before trying -to compile its schema; simply implementing direct `Validate` as eager -`PrepareValidation` followed by plan validation would change that error/I/O -ordering. Keep that short-circuit, operation-local compiled resources, prepared -freezing, schema dialect and containment rules, synchronous cancellation -precedence, and the runner's existing nil-validator behavior where used. -Coordinate schema representation changes with correctness-audit finding 2, -without treating this refactor alone as its fix. - -## 2. Decompose Configuration Validation And Resource Materialization - -**Code:** [validateEffective](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5adc1c1/internal/configuration/load.go#L410), -[materializePrompt](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5adc1c1/internal/configuration/prompts.go#L58), -[buildExecutionProfiles](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5adc1c1/internal/configuration/targets.go#L14), and -[resolution callbacks](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5adc1c1/internal/configuration/resolve_prompts.go#L10). - -**Complexity:** `validateEffective` combines namespace checks, reference indexes, -backend settings, capacity arithmetic, smoke metadata, profile bounds, and prompt -references in one nested traversal. Prompt materialization then combines input -validation, presence projection, content-file I/O, message validation, template -compilation, output-contract construction, hashing, and inspection assembly. -Target assembly is another long presence-state decision chain. Reviewing a -single field change requires following several distinct responsibilities inside -large functions. - -**Opportunity:** Keep the top-level traversal/index preparation visible, but -delegate coherent record-level work to backend, profile, and prompt helpers -with explicit record/path/index arguments. Isolate prompt input projection, -message materialization/compilation, and output-contract construction into -small functions returning the values the caller needs. Keep target presence -interpretation in one identifiable helper rather than distributing it between -new callers. Organize files by those responsibilities only where that helps -navigation; moving unchanged code between files is not itself the improvement. - -**Preserve:** Validate only final ConfigKit-provided values. Keep recorded -normalization, field presence/null distinctions, exact references, error -identities, source-relative resource contexts, lazy metadata capture, and -construction-time template/hash capture. Materialization legitimately combines -validation with resource access; this finding does not require moving every -check into a new validation pass or aggregating formerly fail-fast errors. -Avoid a reflection-driven field schema that obscures the domain rules. - -## 3. Untangle Requirement Traversal, Expression Evaluation, And Evidence - -**Code:** [requirement walker](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5adc1c1/internal/prompt/requirements.go#L356), -[syntax dispatch](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5adc1c1/internal/prompt/requirements.go#L451), -[condition evaluation](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5adc1c1/internal/prompt/requirements.go#L93), and -[command traversal](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5adc1c1/internal/prompt/requirements.go#L652). - -**Complexity:** The approximately thousand-line file contains mutually recursive -syntax walking and partial expression evaluation, branch reachability, caller -scope tracking, template-call handling, evidence promotion, issue construction, -and result sorting. `if` and `with` repeat branch orchestration. `input`, `index`, -root variables, and skipped-path behavior appear in both evaluation and -conservative traversal. Correctness depends on the interaction of return-value -flags and mutable walker state such as `rootKnown`, `dependencies`, and the -unknown-effect flags, not just the local switch case. - -**Opportunity:** Separate the existing roles into small internal units: syntax -recognition/evaluation, branch/scope traversal, and evidence/result accumulation. -Factor shared `if`/`with` orchestration while making their different body-dot -rules explicit. Reuse recognition of supported command shapes instead of -duplicating it between evaluative and inventory-only paths. Make existing -template scope entry/exit state handling explicit. Keep a small enum/value model -rather than adding a general expression interpreter or pluggable AST framework. - -**Preserve:** The deliberately conservative recognition scope, skipped-path -inventory, optional `index` behavior, named-template limitations, incomplete -results, and cancellation semantics. Traversal and evaluation are not -interchangeable: inactive paths must retain recognizable names without -evaluating caller-derived names. Any shared helper must express that distinction. -The evidence-union efficiency finding in [audit 02](02-efficiency.md) is related -but separate; a mechanical file split would not fix it. - -## 4. Clarify Ownership And Policies Across The JSON Value Walkers - -**Code:** [source mapping conversion](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5adc1c1/provider_values.go#L31), -[bounded JSON copying](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5adc1c1/internal/jsonvalue/jsonvalue.go#L67), and -[public-value copying](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5adc1c1/convert.go#L406). - -**Complexity:** Three recursive reflection-based paths independently handle -interfaces/pointers, maps, sequences, scalars, and nil values. Each adds its own -policy: source conversion translates into ConfigKit-compatible containers and -defers unsupported-value errors; `jsonvalue` validates and bounds owned copies; -facade copying detaches already-valid values while preserving their types. -Similar-looking switches conceal different assumptions and failure behavior. -The numeric conversion defect in correctness-audit finding 1 demonstrates a -concrete risk at one of these boundaries, but is not a new finding here. - -**Opportunity:** First make the three responsibilities explicit at their call -boundaries. Isolate source-number encoding from structural traversal, and -identify reusable scalar/indirection/container-copy primitives only where their -semantics truly match. Prefer removing a redundant traversal path or reusing an -existing trusted-value copy operation over maintaining another policy-rich -reflection switch. Keep ConfigKit-specific mapping encoding in its adapter; -do not teach the source-neutral domain about ConfigKit. - -**Preserve:** Supported concrete types, nil behavior, cycle/depth/work handling, -caller ownership, and deferred source errors. In particular, passing source -builders through rejecting `CopyMap` would introduce validation before the -ConfigKit handoff and is not an acceptable consolidation. Nor should an -infallible facade projection acquire new rejection behavior merely to reuse a -validator. A universal callback-configured reflection framework could increase -complexity; retain separate small adapters where their policies differ. - -## 5. Extract Attempt Accounting From Generation Control Flow - -**Code:** [Coordinator.generate](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5adc1c1/internal/generation/coordinator.go#L98), -[generateInjected](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5adc1c1/internal/generation/coordinator.go#L203), -[execute](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5adc1c1/internal/generation/coordinator.go#L303), and -[diagnostic evidence consumers](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5adc1c1/internal/usecase/smoke_tester.go#L246). - -**Complexity:** The coordinator interleaves permit acquisition, dispatch, -cancellation, retry decisions, observer events, diagnostic evidence, and -presence-aware usage accumulation. The built-in and injected paths repeat -generation-number setup, evidence finishing, attempt projection, and usage -projection. The injected path also repeats its non-dispatched cancellation -reporting. `execute` returns five parallel values, increasing the amount of -state the caller must keep aligned while constructing two reporting forms. - -**Opportunity:** Return one small internal attempt-result value containing -outcome and timing/dispatch facts, and extract common projection/accounting -operations where evidence is genuinely shared. Keep the retry loop primarily -about eligibility, budget, waits, and terminal outcomes. Keep injected and -built-in dispatch strategies explicit rather than manufacturing HTTP facts for -injected clients or introducing an extensible retry framework. - -**Preserve:** Release active permits before callbacks/backoff, one-shot injected -dispatch, frozen built-in requests, usage-presence distinctions, private -diagnostic evidence, ordinary observer boundaries, and retention of decoded -facts before cancellation discards ordinary results. The HTTP adapter should -continue owning transport facts; accounting extraction must not push provider -protocol knowledge into the runner or observation package. - -## Prioritization And Non-Findings - -Start with the schema interface/compilation consolidation and configuration -helpers: their responsibilities provide comparatively clear refactoring seams. -The walker and generation coordinator need narrower, behavior-preserving units -because their state transitions are more subtle. Coordinate source conversion -cleanup with the independently required number-preservation fix. - -High scores alone did not justify findings for explicit public error-category -switches, presence-aware source-field encoding, regular-file/schema read loops, -or table-driven tests. Those branches often directly express selected contracts. -Large facade files also include substantial GoDoc and straightforward value -projection, which is not evidence that public/internal separation has failed. -Keep useful test scenarios; do not rewrite tests solely to reduce their metrics. - -No additional ordinary correctness defect was established in this pass. -[Audit 01](01-correctness.md) and [audit 02](02-efficiency.md) retain their own -findings. No implementation changes or additional consumer guarantees are -implied by this refactoring catalog. diff --git a/docs/roadmap/audit/05-resources.md b/docs/roadmap/audit/05-resources.md deleted file mode 100644 index ae27cd3..0000000 --- a/docs/roadmap/audit/05-resources.md +++ /dev/null @@ -1,215 +0,0 @@ -# Resource Management Audit - -Reviewed 2026-11-04 against commit `fb87fab`. - -## Scope And Evidence - -Reviewed file and response-body ownership, bounded reads, HTTP client/transport -lifetimes, timeout contexts and retry timers, backend admission and permits, -prepared-handle cleanup, retained snapshots, JSON conversion, rendering buffers, -schema resources, and repair feedback. The architecture uses immutable -engine-lifetime configuration, invocation-local execution/validation resources, -and caller-owned public results. Production code launches no PromptKit-owned -goroutines or background workers requiring a shutdown protocol. - -Evidence consists of source/call-graph review, relevant existing tests, and a -small temporary offline diagnostic outside the repository. The diagnostic used -a controlled HTTP transport, counted body reads/closure, inspected the existing -prepared-handle owner reference, and compared string backing addresses. No -benchmark, heap-size measurement, deliberate exhaustion, live network call, -production change, or new repository test was needed. - -Focused existing tests passed for content-file closure, HTTP error-body limits -and closure, independent attempt contexts/body closure, read cancellation, -prepared discard/ownership, diagnostic formatting, synchronous schema -cancellation, and repeated-value JSON work bounds. These tests establish the -existing protected behavior; they do not cover all findings below. - -## 1. Authentication Diagnostics Read An Unbounded Upstream Body - -**Priority:** P2. Unbounded remote-response consumption; a finite limit would be -a new policy decision, not enforcement of an existing documented byte limit. - -**Code:** [runOpenRouterKey](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/fb87fab/internal/usecase/smoke_authentication.go#L66), -[recognizedOpenRouterKeyResponse](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/fb87fab/internal/usecase/smoke_authentication.go#L139), -and [diagnostic HTTP execution](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/fb87fab/internal/llm/http_evidence.go#L129). - -**Circumstances:** An authentication endpoint returns HTTP 200 with a large or -continuously growing body. Recognition calls unrestricted `io.ReadAll` and then -unmarshals the whole document into raw-message maps. Neither declared length nor -streamed bytes are bounded on this path. Generation's 16 MiB success-body limit -and 64 KiB error-body limit do not apply to diagnostic GETs. - -**Consequences / evidence:** The offline diagnostic supplied a valid small -identifying `data` object plus 4 MiB of irrelevant padding. The probe read all -4,194,375 bytes, closed the body, and reported credential acceptance. Increasing -the padding has no corresponding byte-budget check. A fast misconfigured or -unexpected endpoint can consume substantial memory before its timeout; concurrent -authentication probes multiply consumption because HTTP-only diagnostics bypass -generation admission. A timeout bounds elapsed time, not bytes allocated within -that time. There is no demonstrated descriptor leak: the body is closed. - -**Potential remediation:** Select a finite body budget appropriate to this small -diagnostic protocol, enforce it for both declared and streamed/underreported -lengths, and close without unbounded draining on overflow. Preserve single-JSON -framing and the distinction between unknown acceptance and observed rejection; -an oversized response must not be reported as credential rejection. Decide the -budget and overflow report classification explicitly, then update the owning -contract and tests. This need not impose new limits on application prompts, -artifacts, or generated output. - -## 2. Convenience Source Conversion Expands Shared Graphs Before Bounded Loading - -**Priority:** P2. Potentially exponential allocation before upstream resource -checks can run. - -**Code:** [providerMapForMapping](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/fb87fab/provider_values.go#L19), -[providerValueForMapping](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/fb87fab/provider_values.go#L31), -[ProfileSource](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/fb87fab/profiles.go#L33), and -[BackendSource](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/fb87fab/backends.go#L100). - -**Circumstances:** A caller builds a small acyclic Go value graph containing -repeated map/slice references. Each level can point twice to the same child: - -```go -value := map[string]any{"value": true} -for range depth { - value = map[string]any{"left": value, "right": value} -} -``` - -The source converter tracks only active ancestors to identify cycles. Once one -branch finishes, its visit is removed, so the sibling independently converts -the same entire child. It allocates a fresh expanded tree before calling -`configkit.Mapping`. For d levels, O(d) distinct input maps become -Theta(2^d) output maps. The depth guard does not constrain width or total work; -this expansion can become impractical well below depth 100. - -**Consequences:** Programmatic source creation can exhaust memory before -`NewEngine` or ConfigKit's deferred diagnostics are reached. This is not a claim -that ConfigKit lacks bounds: the selected ConfigKit v1.1.0's mapping builder -checks node, byte, and depth budgets during its own snapshot traversal -(`internal/engine/value.go`, `valueBuilder.build`). It is called only after -PromptKit has already expanded the graph. PromptKit's later `jsonvalue` node -budget is also too late to protect this conversion. - -**Potential remediation:** Memoize completed structural conversions so repeated -input containers reuse already-converted containers until ConfigKit snapshots -them with its existing bounds. Keep active-ancestor tracking distinct from the -completed memo so cycles still become deferred source errors. Account correctly -for container identity, including distinct slice views. Alternatively, seek -upstream support for the additional Go value forms so this conversion can be -removed. Preserve immediate source ownership and deferred failure behavior; -do not introduce a PromptKit raw-source validation pass or duplicate ConfigKit's -configuration limits to fix this pre-handoff allocation problem. - -## 3. Inert Prepared Handles Retain Their Entire Execution Owner - -**Priority:** P3. Avoidable retained-memory dependency after use/discard. - -**Code:** [PreparedExecution.owner](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/fb87fab/internal/usecase/prepared_execution.go#L24), -[Discard](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/fb87fab/internal/usecase/prepared_execution.go#L96), -[claim](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/fb87fab/internal/usecase/prepared_execution.go#L172), and -[payload clearing](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/fb87fab/internal/usecase/prepared_execution.go#L188). - -**Circumstances:** A consumer retains a completed or discarded handle for its -supported `Details` access, while replacing or dropping the originating engine. -Payload clearing releases the per-execution plan, credential, observer, and -execution snapshot, but the handle's `owner *Runner` remains unchanged. - -**Consequences / evidence:** The reference path is -`handle -> Runner -> captured repositories/clients/validator/capacity manager`. -Thus a small inert handle keeps the originating engine's complete runtime -snapshot and collaborators reachable, not just its deliberately retained details. -The offline diagnostic confirmed the same nonnil runner pointer after both -discard and successful execution. Retaining handles from successive engine -reconstructions can retain successive large snapshots. Keeping details itself -is required behavior, and this has no additional owner-retention cost while the -application still retains the same engine anyway. - -**Potential remediation:** Clear the owner reference when the handle becomes -inert, checking ownership under the same mutex to avoid racing the current -unlocked owner read. Another option is a small standalone identity token shared -by a runner and its handles, which establishes ownership without retaining the -runner graph. Such a token must have reliable distinct identity; do not rely on -distinct zero-sized allocations comparing unequal. Preserve related namespace -views, foreign-owner rejection, single claim, discard/run races, and details -availability. No engine `Close` method or automatic closing of caller-owned -collaborators is implied. - -## 4. Repair Feedback's Output Bound Does Not Bound Its Temporary Serialization - -**Priority:** P3. Avoidable memory amplification on validation failures. - -**Code:** [formatRepairDiagnostics](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/fb87fab/internal/usecase/repairer.go#L119). - -**Circumstances:** A failed schema validation produces a large diagnostic or many -diagnostics and repair is enabled. The formatter normalizes every string, marshals -the complete array, then discovers that it exceeds 64 KiB. It subsequently -marshals individual full entries again before truncating one of them. - -**Consequences:** Although the returned feedback is bounded correctly, temporary -storage scales with the complete diagnostic text, including text omitted from -the request. JSON escaping can amplify its size, and normalization adds another -copy for invalid UTF-8. Large validation messages and concurrent repairs can -therefore create large additional buffers despite producing tiny feedback. -The complete diagnostics already exist for the validation result; these extra -full encodings do not help preserve them. Existing formatting tests check output -bounds/data preservation, not scratch-space use. - -**Potential remediation:** Encode feedback incrementally with bounded buffering -and lookahead sufficient to distinguish a complete array from an omitted suffix. -Stop encoding omitted text rather than allocating its full JSON representation. -Keep the existing 64 KiB result bound, ordered entries, rune-safe truncation, -valid UTF-8, JSON escaping, and in-bound omission notice. Leave the original -validation diagnostics and full previous candidate unchanged. This is an -implementation opportunity, not a claim that the existing contract promises a -64 KiB peak heap footprint or permission to add a diagnostic-input limit. - -## 5. Short Normalized Session IDs Can Pin Large Input Strings - -**Priority:** P3. Avoidable backing-buffer retention in retained metadata. - -**Code:** [NormalizeSessionID](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/fb87fab/internal/domain/session.go#L10), -[preparation](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/fb87fab/internal/usecase/runner.go#L312), and -[prepared metadata construction](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/fb87fab/internal/usecase/runner.go#L491). - -**Circumstances:** A direct or rendered session ID contains substantial leading -or trailing whitespace around a short valid identifier. `strings.TrimSpace` -returns a substring; normalization validates that substring and returns it -without detaching its backing storage. Public conversion preserves the string. - -**Consequences / evidence:** An offline public-API probe prepared a request with -1 MiB of spaces followed by `s`. The returned session ID had length one but its -backing address still pointed inside the large original allocation. Retaining a -prepared value, handle details, or run result can therefore retain that buffer -after the original request/rendering buffer would otherwise be collectible. -The 256-code-point semantic limit does not prevent this retention. This does not -require changing which padded identifiers are accepted. - -**Potential remediation:** Detach accepted normalized session metadata with a -small owned string, for example `strings.Clone` after validation. Preserve empty -omission, Unicode trimming/counting, hashes, and wire content. Review other -short canonical metadata returned by trimming for the same pattern when making -the change; do not indiscriminately clone large prompt/message payloads or add a -new raw-session-size restriction. - -## Cleanup Checks And Non-Findings - -No additional descriptor, timer, connection-ownership, or goroutine leak was -established. Artifact/schema handles use deferred closure; prompt content handles -close after reading; generation response bodies close on success and decoding, -status, and size failures. Diagnostic bodies close, and their timeout cancellation -is attached to body closure. Retry timers stop on exit. Admission/active permits -release on errors, cancellation, and panic unwinding. Prepared payloads otherwise -clear as intended. - -Caller-selected artifact/schema/prompt sizes, intentionally unlimited backends, -and caller-retained ready handles are not classified as bugs merely because -their total memory can grow. Synchronous dependency calls may delay cancellation -by design, rather than being detached into leaking goroutines. Borrowed HTTP -transports/filesystems/clients remain subject to their ownership contracts; this -audit proposes no blanket shutdown or disposal API. - -Previously recorded repeated-copying issues remain in -[the efficiency audit](02-efficiency.md); they are not duplicated here. diff --git a/docs/roadmap/audit/06-error-handling.md b/docs/roadmap/audit/06-error-handling.md deleted file mode 100644 index 1038a55..0000000 --- a/docs/roadmap/audit/06-error-handling.md +++ /dev/null @@ -1,197 +0,0 @@ -# Error-Handling Audit - -Reviewed 2026-11-04 against commit `6b1323d`. - -## Scope And Evidence - -Reviewed error detection, wrapping, public translation, diagnostic reporting, -cancellation and timeout handling, retries, validation/repair failures, and -cleanup across construction, configuration inspection, prompt/profile selection, -artifact reading, rendering, ordinary and prepared execution, generation, and -smoke diagnostics, plus the small release-entrypoint wrappers. Contracts were -checked against GoDoc, integration references, the architecture policy, ADR 0001, -and existing tests. - -Configuration composition belongs to ConfigKit; PromptKit validates and -interprets its final handoff and preserves upstream load errors. The runner adds -operation context, generation coordinates attempts and status retries, the HTTP -adapter classifies/redacts provider failures, and the public facade translates -internal categories. Content-validation failures and concluded diagnostic -failures are generally data rather than Go errors under their respective -contracts. Prepared handles permit one execution claim; failure after a -successful claim does not make the handle reusable. - -Evidence includes source/call-graph review and a temporary public-API diagnostic -outside the repository. It used injected collaborators and a controlled HTTP -transport; it made no live network calls. No production code or repository -tests were changed. - -Existing error, failure, cancellation, timeout, panic-cleanup, and retry tests -passed using: - -```sh -go test -count=1 -run 'Error|Failure|Fail|Cancel|Timeout|Panic|Release|Retry|Retries' \ - . ./internal/llm ./internal/generation ./internal/usecase \ - ./internal/capacity ./internal/validate ./internal/artifact \ - ./internal/configuration -``` - -These tests establish protected behavior but do not cover the three reproductions -below. Findings from the earlier correctness and resource-management audits are -not repeated here. - -## 1. Nested Collaborator Errors Can Hide The Owning Operation's Public Category - -**Priority:** P2. Confirmed public error-classification defect. - -**Code:** [publicErrorFor](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/6b1323d/errors.go#L63), -[mapPublicError](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/6b1323d/errors.go#L16), and -[wrapGenerationError](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/6b1323d/internal/usecase/runner.go#L271). - -**Failure condition:** An injected `LLMClient` or `ArtifactReader` returns an -error obtained from another PromptKit operation. Ordinary mapped errors retain -their internal chain. `publicErrorFor` searches that complete chain with -`errors.Is` and chooses the first matching category in a fixed priority order, -regardless of which operation the outer runner classified. - -**Expected:** A model-client failure matches `ErrLLMGenerate`; an artifact-reader -failure matches `ErrArtifactLoad`. The injected cause remains available through -`errors.Is`, as their [public GoDoc](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/6b1323d/engine.go#L49) specifies. Preserving -the nested cause may also preserve its public categories; it must not replace -the category for the operation that just failed. - -**Actual / reproduction:** A child engine's `Run` returns a missing-prompt error. -A parent engine selects a valid prompt, then its injected client returns that -child error. The outer error contains the text `failed to generate output`, but: - -```text -errors.Is(err, ErrLLMGenerate) = false -errors.Is(err, ErrPromptNotFound) = true -errors.Is(err, childError) = true -``` - -Returning the same child error from an injected artifact reader similarly -produces `errors.Is(err, ErrArtifactLoad) == false`. This is reproducible using -only public APIs, without importing internal sentinels. Consumers can therefore -miss the documented failure category when composing PromptKit-backed adapters. - -**Potential remediation:** Translate the owning operation's classification -independently of an arbitrary collaborator's nested chain, while retaining the -original injected cause. Explicit boundary/stage information or mapping at the -operation boundary can do this. Merely reordering the global switch risks -breaking intentional direct-selection behavior: a genuinely missing prompt or -profile must retain its documented distinction from other load failures. -Protect nested-client and nested-reader cases through public-boundary regression -coverage; preserve existing capacity, retry-exhaustion, provider-error redaction, -and context identities. - -## 2. Successful-Status Body Read Failures Are Misreported As Malformed JSON - -**Priority:** P2. Confirmed loss of failure provenance and cause. - -**Code:** [decodeOpenAIChatResponse](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/6b1323d/internal/llm/openai_compatible_client.go#L373). - -**Failure condition:** A 2xx response body returns a non-EOF I/O error other than -`context.Canceled` or `context.DeadlineExceeded`, either during the first decode -or while checking the required final EOF. For example, a connection can fail -while reading the response stream after headers were received. - -**Expected:** Distinguish an underlying read/transport failure from evidence of -invalid JSON or trailing data. Retain the read cause through a redacting error -so callers can inspect it with `errors.Is`/`errors.As`, consistent with the -[transport category](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/6b1323d/engine.go#L65). Do not report trailing data when the -reader failed without supplying any. This does not change the established -classification of invalid/truncated JSON or introduce transport retries. - -**Actual / reproduction:** The controlled transport returned HTTP 200 and a body -that supplied either a partial JSON prefix or a complete valid chat envelope, -then returned a distinct read-error sentinel. Both executions returned -`ErrLLMGenerate` plus `ErrMalformedResponse`, did not match `ErrTransport`, and -discarded the sentinel. For the complete envelope, the error said `response -contains trailing data`, although no trailing bytes were supplied. Inference -smoke diagnostics likewise reported `malformed_response` with nil Go error -rather than a transport failure. Thus the issue affects both public execution -errors and factual diagnostic classification. - -The integration contract explicitly guarantees exact causes for -[`http.Client.Do` failures](../../integrations/openai-compatible-chat.md#response-handling), -not every decoder error. The finding concerns the separate loss and -misclassification of an actual body-read failure, rather than asserting a new -requirement to expose JSON parser errors or provider content. - -**Potential remediation:** Track underlying reader failures separately from -decoder/framing failures, or reliably distinguish those classes before replacing -the decode error. Reuse the redacting transport-error path for genuine read -failures. Keep syntax/type errors, EOF-truncated JSON, extra JSON values, and -size overflow in the existing malformed-response category; preserve cancellation -precedence and bounded reads/closure. Add focused coverage for interrupted -initial decoding and interrupted EOF verification, including diagnostic failure -classification. Default formatting must remain free of raw response content and -private transport details. - -## 3. Inference Diagnostics Swallow Some Invalid Request-Construction Errors - -**Priority:** P2. Confirmed mismatch with the diagnostic error contract. - -**Code:** [runInference](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/6b1323d/internal/usecase/smoke_tester.go#L207), -[generation preparation](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/6b1323d/internal/generation/coordinator.go#L96), and -[request payload validation](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/6b1323d/internal/llm/openai_compatible_client.go#L511). - -**Failure condition:** An inference diagnostic's effective settings are invalid -in a way detected by built-in request construction rather than target selection. -A runtime `Execution.ExtraParams` override containing the reserved key `model` -is one public example. The coordinator returns `llm.ErrInvalidRequest` before -dispatch with no attempt evidence. `runInference` treats every generation error -as a concluded diagnostic failure. - -**Expected:** The [SmokeTest contract](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/6b1323d/smoke_diagnostics.go#L238) says invalid -kinds, settings, and selections return an ordinary Go error before diagnostic -work. A locally invalid request should follow that path and expose the public -invalid-request category. This is distinct from a provider rejecting a validly -constructed request or a diagnostic concluding with an observed failure. - -**Actual / reproduction:** Calling text-inference `SmokeTest` with -`Execution.ExtraParams: map[string]any{"model": "override"}` returned nil Go -error and a report with `Completed == true`, `Failure == invalid_request`, no -attempts, and zero transport calls. The request-construction error and its -explanation were discarded. The caller cannot use the documented ordinary-error -path to identify this configuration mistake. - -**Potential remediation:** Preserve the distinction between local built-in -request-preparation failures and concluded execution failures. Return preparation -errors through the use-case/public invalid-request mapping rather than marking -the diagnostic complete; preparation can remain in the generation adapter, with -enough provenance returned to the caller to make that distinction. Do not infer -that every zero-attempt diagnostic is an error: missing required credentials, -capacity rejection, unsupported operations, and inapplicability have intentional -report semantics. Preserve concluded provider failures and the separate -caller-cancellation behavior. Add one public-boundary regression for the -reserved-field override, retaining existing tests for those intentional reports. - -## Reviewed Behavior Not Raised As Defects - -- ConfigKit owns raw-source composition and errors before the accepted handoff; - no additional PromptKit raw-source validation is proposed. -- Non-2xx provider detail parsing is deliberately best-effort. Unreadable, - oversized, or unrecognized error bodies retain the received status without - exposing the raw body. That differs intentionally from decoding a 2xx body. -- Retries are limited to selected HTTP statuses. Transport failures, malformed - responses, and arbitrary injected-client failures are not automatically - retried; no expansion is needed for these findings. -- Completed output-validation failures are returned as validation results; - operational schema failures are errors. Repair failure propagation and - exhaustion retain that distinction. -- Caller cancellation, configured request timeouts, cancellation during capacity - acquisition/backoff, synchronous schema operations, and partial diagnostic - evidence have explicit, tested handling. Callback panics propagate while - deferred permit/admission cleanup still runs. -- Ignored SHA-256 writer errors cannot occur with the selected implementation. - Best-effort closure of read-only files and HTTP bodies does not imply a - persistence guarantee or require converting a completed read into a failure. -- Release wrappers use fail-fast shell execution and `exec` to propagate the - selected tool's status. Publication and recovery implementation belongs to - ReleaseKit; no release or publication operation was run for this audit. - -No additional state-recovery or cancellation defect was substantiated in this -pass. The findings do not call for a new recovery subsystem, automatic replay of -possibly completed provider work, or changes to source ownership. diff --git a/docs/roadmap/audit/07-security.md b/docs/roadmap/audit/07-security.md deleted file mode 100644 index 38fd7d1..0000000 --- a/docs/roadmap/audit/07-security.md +++ /dev/null @@ -1,205 +0,0 @@ -# Security Audit - -Reviewed 2026-11-04 against commit `cf0bfd4` and the current worktree. Pending -relocations of earlier audit documents do not change the production baseline -and were left untouched. - -## Scope, Architecture, And Threat Boundary - -PromptKit is an in-process Go library with a public facade and internal -configuration, execution, transport, validation, and observation components. -ConfigKit owns source loading/composition; PromptKit interprets its final -handoff and captures immutable definitions. Execution reads caller-selected -artifacts, renders captured templates, resolves credentials, and contacts the -selected backend. Prepared handles privately retain one execution's settings. -Inspection and observation expose deliberately selected metadata. - -Reviewed configuration and request validation, filesystem/resource selection, -template and JSON handling, credential flow, public formatting and reports, -HTTP requests/redirect handling, provider responses, schema loading, repair, -backend capacity, prepared ownership, and release-entrypoint wiring. The review -included interactions with the selected ConfigKit and JSON Schema dependencies -and an advisory scan of runtime packages. - -The [architecture policy](../../policy/architecture.md#repository-and-consumer-boundary) -and [consumer security guidance](../../consumers/pkg-promptkit.md#protect-files-and-generated-data) -place application authorization, untrusted path/endpoint admission, payload-size -policy, and output disclosure with the consumer. Configuration sources, -templates, and injected Go collaborators are application-selected capabilities. -Upstream HTTP response bytes remain an external trust boundary. This review -does not treat ordinary file selection, runtime endpoint overrides, namespace -selection, or supported message roles as authorization mechanisms. - -No production code or repository tests were changed, no secrets were collected, -and no external provider or forge was probed. Network activity was limited to -obtaining the public Go vulnerability checker/database and reading official -advisories. This is a source review with focused diagnostics, not a claim that -every possible consumer deployment is secure. - -## Evidence - -- Source/call-graph review and the applicable public GoDoc, framework formats, - integration contract, policies, and ADR 0001. -- Existing focused tests passed for credential/report formatting, sensitive - configuration inspection/diffs, schema reference containment, provider-body - bounds, JSON value cycles/depth, prepared ownership, and capacity boundaries. -- A temporary offline public-API diagnostic used a controlled HTTP transport - and a synthetic credential. It verified the authentication-body behavior in - finding 2 and confirmed that credential was absent from the serialized report. -- `govulncheck` v1.9.0 scanned `./...` using Go 1.27.0 and the Go vulnerability - database last modified `2026-11-02T22:01:43Z`. The runtime scan covered the - module, its examples, ConfigKit v1.1.0, jsonschema/v6 v6.0.2, YAML v4.0.0-rc.6, - and x/text v0.14.0. It is not a complete audit of ReleaseKit's implementation - or every dependency used only by the release tool. - -The text-mode scan reported ten symbol-level standard-library advisories, plus -package/module-only matches. Those are candidate matches, not ten established -PromptKit vulnerabilities. Applicability was checked against the actual -operations described below rather than inferred from interface-call traces. - -## 1. The Audited Go Toolchain Predates Applicable Networking Security Fixes - -**Priority:** P2. Build/deployment security finding, not a newly discovered -PromptKit algorithm defect. - -**Relevant code:** [declared Go minimum](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/cf0bfd4/go.mod#L3), -[default HTTP-client construction](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/cf0bfd4/internal/llm/openai_compatible_client.go#L58), -[generation dispatch](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/cf0bfd4/internal/llm/openai_compatible_client.go#L211), and -[diagnostic dispatch](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/cf0bfd4/internal/llm/http_evidence.go#L129). - -**Condition and impact:** The local toolchain used for this audit is Go 1.27.0. -Applications built with that patch can exercise affected standard-library -networking through PromptKit's normal outbound client: - -- [GO-2026-6611](https://pkg.go.dev/vuln/GO-2026-6611) describes excessive CPU - consumption caused by a malicious HTTP/2 peer repeatedly changing initial - stream windows. An application using HTTP/2 against a hostile or compromised - backend, with enough active streams for amplification, can suffer availability - loss. The Go 1.27 fix is in 1.27.2. Body-size limits do not address protocol - processing below response decoding. -- [GO-2026-6809](https://pkg.go.dev/vuln/GO-2026-6809) adds explicit rejection of - compromised intermediate certificates. The affected conditions depend on the - platform and trust-store state. When the old trust path remains accepted, an - attacker able to present a matching certificate chain and intercept the - connection could compromise confidentiality/integrity of prompts, responses, - or bearer credentials. The Go 1.27 fix is in 1.27.3. This is not a claim that - every default system trust store remains affected. - -**Evidence and limits:** The advisories establish affected versions and attack -conditions; source review establishes use of the standard outbound HTTP/TLS -path. No certificate forgery, HTTP/2 attack, or running consumer binary was -tested. The scanner's surprising interface-based paths from unrelated methods -are not relied upon as proof of exploitation. PromptKit is source-only: a -consumer's actual compiled toolchain and transport configuration determine its -exposure. The `go 1.27.0` directive is a compatibility minimum, not a record of -which patch built an application. - -**Potential remediation:** Use an appropriate current patched Go toolchain, -at least 1.27.3 within this series for these two fixes, for release validation -and ReleaseKit execution; advise consumers to rebuild affected applications -with patched Go. Recheck advisories against that selected toolchain. This fits -the existing distinction between consumer minimum and maintainer toolchain in -the [release procedure](../../release.md#prepare-the-candidate). -Whether to raise the consumer minimum for the identified fixes is a separate -compatibility decision under the existing policy, not an automatic consequence -of this audit. These fixes do not require custom TLS verification, a blanket -HTTP/2 ban, or a new recurring release gate. - -## 2. Authentication Probes Permit Unbounded Remote-Body Memory Consumption - -**Priority:** P2. Existing availability finding, confirmed at the external -response boundary. This is the same mechanism as resource-management audit -finding 1, not a second independent remediation item. - -**Relevant code:** [authentication recognition](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/cf0bfd4/internal/usecase/smoke_authentication.go#L139), -[authentication probe execution](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/cf0bfd4/internal/usecase/smoke_authentication.go#L66), -and [diagnostic HTTP execution](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/cf0bfd4/internal/llm/http_evidence.go#L129). - -**Condition:** A consumer explicitly invokes an `openrouter_key` authentication -diagnostic. The selected endpoint, a compromised provider, or an intermediary -on an operator-selected plaintext connection returns HTTP 200 with a large or -continuously growing body. Recognition uses unrestricted `io.ReadAll` followed -by whole-document JSON unmarshalling. Unlike generation response decoding, -this path has no byte budget. HTTP-only probes also do not use generation -capacity admission. - -**Impact:** The remote peer controls memory consumption before the diagnostic -returns. A fast response can cause large allocations or process termination -before a deadline, and concurrent probes multiply the exposure. A configured -timeout bounds time, not bytes allocated during that interval. This requires a -diagnostic call; it is not an unauthenticated inbound attack on a PromptKit -server, and ordinary `Run` does not execute the probe. - -**Evidence:** The offline diagnostic returned a minimal recognized account -object and 4 MiB of irrelevant padding. PromptKit read all 4,194,374 bytes, -closed the body, and concluded with credential acceptance. The body used an -unknown content length. No deliberate memory exhaustion was attempted. The -synthetic bearer credential was supplied to the controlled transport and was -absent from the serialized diagnostic report. - -**Potential remediation:** Select a finite body budget for this small diagnostic -protocol and enforce it for declared and streamed lengths, closing on overflow -without unbounded draining. Preserve factual reporting: overflow establishes -neither credential rejection nor acceptance. Agree the limit and overflow -classification explicitly; no existing authentication-body byte guarantee is -being enforced. The existing resource-audit remediation can own this fix; -there is no need for separate global limits on prompts or artifacts. - -## Other Candidates And Boundary Checks - -- **Scanner applicability:** Server-only advisories GO-2026-6603, 6612, 6613, - and 6617 do not establish an attack on PromptKit's outbound client. ECH - server parsing (6607), multipart-form parsing (6608), and Range-serving - behavior (6609) likewise lack a matching operation here. The CONNECT-with-body - condition in 6605 and raw HTTP/2-to-HTTP/1 header forwarding in 6610 are absent - from built-in POST/GET generation and factual diagnostic reporting. Windows - junction-directory creation (6604) is not established by read-only resource - opens. These exclusions concern PromptKit, not all possible code in a host - application. -- **Module-only matches:** x/text advisories GO-2026-5970 and 6629 were not - reported as called vulnerable symbols; no matching use of `norm.Iter` or - `precis.Nickname` was established. html/template advisories 6599 and 6600 do - not apply to the text/template renderer. No separate runtime third-party - vulnerability was substantiated by this scan; that is not a general safety - certification of those modules. -- **Files and schemas:** Ordinary artifacts intentionally allow unrestricted - caller-selected regular files. Host-backed `content_file` contexts use - ConfigKit's rooted filesystem, while the standard schema loader checks - canonical containment and rejects remote reference schemes. Generic `fs.FS` - inputs remain caller-supplied capabilities; logical path checks do not turn an - arbitrary filesystem implementation into a host sandbox. No additional - confinement guarantee is proposed. -- **Injection:** Variables and artifact text are rendered as data rather than - reparsed as Go templates. Captured templates have the input helper and normal - text/template functionality, not a shell or process-execution helper. - Provider payloads use JSON encoding and reserved-field checks. Model output - is data for validation/repair, not executable code or an authorization - decision. Prompt injection and domain-aware interpretation remain consumer - concerns. -- **Credentials and disclosure:** Direct keys are request-scoped, excluded from - supported JSON and public request summaries, and removed from diagnostic - settings and prepared details. Provider error defaults redact details; - explicit provider accessors and arbitrary application content retain the - documented consumer disclosure responsibility. Inspection/diff tests protect - configured sensitive provider paths. No blanket redaction of arbitrary - prompt, schema, variable, error, or provider content is promised. -- **Ownership and availability:** Engine snapshots and copied public values - prevent ordinary cross-call mutation. Prepared handles enforce owner binding - and single claim. Capacity and retry cleanup are tested. Intentionally - unlimited backends, synchronous template/schema dependency calls, and - application-selected large files are not newly classified as vulnerabilities. - The other resource-audit findings remain maintenance items rather than - independently demonstrated remote attacks. -- **Release tooling:** Local launchers pass quoted arguments through `exec`; - there is no PromptKit runtime command-execution surface. The delivery token - path is ignored and selected for committed-file protection. Publication, - token-file permission enforcement, and recovery implementation belong to - ReleaseKit; no publication command was run during this audit. - -## Recommended Follow-Up - -Use a patched Go toolchain before release preparation and address the existing -authentication-body availability finding. Neither requires a new application -authorization subsystem or stronger filesystem, endpoint, or semantic-output -contracts. No additional PromptKit-specific confidentiality, integrity, or -authorization defect was substantiated within the documented boundary. diff --git a/docs/roadmap/audit/08-testing.md b/docs/roadmap/audit/08-testing.md deleted file mode 100644 index 1247f1d..0000000 --- a/docs/roadmap/audit/08-testing.md +++ /dev/null @@ -1,284 +0,0 @@ -# Test Coverage And Effectiveness Audit - -Reviewed 2026-11-04 against commit `f7085d7` and the current worktree. Existing -audit-document moves were preserved. No production code or tests were changed. - -## Assessment And Method - -The suite broadly follows the [testing policy](../../policy/testing.md): real -ConfigKit composition, filesystems, rendering, validation, and engine assembly -are exercised alongside controlled model clients and HTTP transports. Internal -tests own focused rules and coordination, external-package root tests protect -the public facade, and the three maintained examples have Go workflow tests. -Capacity, prepared-handle state transitions, ownership, cancellation, retries, -repair, observations, and configuration inspection have substantial behavioral -coverage. This is a targeted improvement list, not a recommendation to replace -the test architecture or pursue a percentage threshold. - -Both required analyses were performed: review of missing behavioral protection, -and review of assertion/fixture cost and coupling. Evidence includes source and -call-graph review, existing GoDoc and integration contracts, full-suite coverage, -repeated race-enabled execution, and reuse of the earlier error audit's temporary -offline diagnostic. No live provider, paid API, new repository test, production -mutation, or external mutation-testing service was needed. - -Validation used the local Go 1.27.0 toolchain: - -```sh -go test -count=1 -covermode=atomic -coverpkg=./... -coverprofile=/tmp/promptkit-testing-audit-LHAyNm/coverage.out ./... -go tool cover -func=/tmp/promptkit-testing-audit-LHAyNm/coverage.out -go test -race -vet=off -shuffle=42 -count=2 ./... -``` - -Both suite runs passed, including the examples. The race run exercised two -shuffled passes; this is evidence against readily reproducible order/race -problems, not proof that no flakes exist. Coverage output remains outside the -repository. The commands are audit evidence, not additional release gates. - -Combined coverage is **86.4%**, or 4,782 of 5,536 instrumented statements. -Selected package totals below combine hits across all test binaries. The -per-binary percentages printed by `-coverpkg=./...` are not individual-package -coverage percentages; repeated profile spans were combined before calculating -these totals. - -| Component | Combined statement coverage | -| --- | ---: | -| Public facade | 81.0% | -| Configuration | 91.0% | -| Domain | 98.0% | -| Capacity | 98.0% | -| Generation coordination | 91.2% | -| HTTP model client | 94.2% | -| Execution use cases | 90.5% | -| Rendering and requirement inspection | 78.4% | -| Output/schema validation | 75.8% | - -Coverage located candidates; the findings below concern specific risks and -assertions, not the percentages themselves. Testing recommendations associated -with already recorded implementation defects should accompany those fixes, -rather than becoming duplicate remediation projects. - -## 1. RunRequest's Privacy Formatting Contract Is Not Exercised - -**Priority:** P2. Missing protection for a documented confidentiality boundary. - -**Relevant code:** [RunRequest formatting](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/formatting.go#L5) and the -analogous existing [SmokeTestRequest privacy test](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/smoke_test_contract_test.go#L135). - -**Deficiency:** `RunRequest.String`, `GoString`, and their shared formatter all -have 0% combined statement coverage. The suite protects formatting of generated -client requests, smoke-test requests, and opaque prepared handles, but does not -exercise the ordinary run request's own formatting contract. - -**Unverified behavior:** A change that prints the direct key, input bodies, or -variable values from `RunRequest` can pass the complete suite. This is an -assertion gap, not evidence that current formatting actually leaks those values. - -**Potential improvement:** Add one small public-boundary test with distinct -synthetic private markers and exercise the supported String/GoString formatting -paths on values and pointers. Assert absence of the data covered by the GoDoc; -do not snapshot the complete summary or freeze incidental field order/count -formatting. If adding a JSON check, assert only the documented direct-key -exclusion, not a stable JSON shape or omission of ordinary request data. -Reflection-based disclosure is explicitly outside this formatter guarantee. - -## 2. The Public Single-Schema-File Option Has No Execution Coverage - -**Priority:** P2. An exported source-selection path is entirely unprotected. - -**Relevant code:** [WithSchemaFile](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/engine.go#L212), -[SetSchemaFile](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/internal/engine/engine.go#L75), and -[fileSource](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/internal/engine/engine.go#L211). - -**Deficiency:** All three functions have 0% combined coverage. The existing -[single-file validator test](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/internal/validate/standard_validator_test.go#L596) -constructs `FSValidator` directly with `fstest.MapFS`; it does not test the public -option, host-file selection, or its engine wiring. - -**Unverified behavior:** The public option could ignore its argument, fail to -replace `SchemaDir`, select the wrong root/base name, or mishandle invalid files -without this validator test noticing. Direct validator success does not establish -that a consumer can reach that behavior through the supported API. - -**Potential improvement:** Use `t.TempDir` and a real schema file in a small -public integration test. Exercise construction plus preparation/validation -through that file's base name, and representative invalid selections such as a -missing file or directory. Include source-option replacement where it protects -the documented last-valid-option behavior. Assert public error identity and -observable schema selection, not private helper calls. This does not require a -helper-by-helper test matrix or model-provider access. - -## 3. Programmatic Provider-Value Conversion Lacks Representative Value Cases - -**Priority:** P2. Conversion and ownership coverage misses known data-integrity -and deferred-error risks. - -**Relevant tests:** [programmatic source ownership/order](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/sources_contract_test.go#L14) -and [programmatic construction](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/construction_contract_test.go#L226). -**Implementation:** [providerValueForMapping](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/provider_values.go#L31). - -**Deficiency:** The converter has only 11.9% combined coverage. The principal -source-ownership test mutates a flat string-valued map. It does not establish -preservation and ownership for representative nested/typed containers or the -`json.Number` branch, nor deferred failure for unsupported provider values. -The more extensive `internal/jsonvalue` tests protect a different traversal and -do not execute this pre-ConfigKit convenience conversion. - -**Unverified behavior:** The suite passes with the exact-number rounding defect -already recorded in [the correctness audit](01-correctness.md#1-provider-source-helpers-silently-change-exact-json-numbers). -It also would not detect many regressions in nested source capture or the -documented source-creation-versus-loading failure boundary. - -**Potential improvement:** Exercise both `ProfileSource` and `BackendSource` -through `NewEngine` with a compact set of representative supported nested -values, mutations after source creation, and unsupported/cyclic values that -should fail during loading. Add the numeric regression when addressing the -existing precision finding. Compare meaningful final values or encoding, not Go -container identity, numeric spelling, or a required float representation. -Keep tests at the final handoff/public source-helper boundary; do not duplicate -ConfigKit's raw-source validation or its entire value-limit suite. Work-bound -coverage for the separately identified shared-graph issue belongs with that fix. - -## 4. Error Tests Miss Composed Causes And Pre-Dispatch Diagnostic Failures - -**Priority:** P2. Covered code paths lack important boundary-case assertions. - -**Relevant tests:** [public generation failures](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/generation_error_contract_test.go#L111), -[direct mapper cases](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/errors_internal_test.go#L13), -[body cancellation](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/internal/llm/prepared_generation_test.go#L161), and -[inference diagnostic cases](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/internal/usecase/smoke_inference_test.go#L14). - -**Deficiency and demonstrated blind spots:** - -- Injected failures are mostly plain marker errors or a single explicit - category. They do not check a real nested PromptKit error returned through a - different operation. The suite passes although an injected client/reader can - lose the outer public generation/artifact category. -- Tests cover `http.Client.Do` failures and cancellation during successful-body - reading, but not an ordinary non-EOF read error during initial decoding or - final EOF verification. Such errors currently lose their cause and become - malformed-response failures. -- Diagnostic tests cover invalid selection/settings and concluded generation - failures separately, but miss a runtime setting rejected by built-in request - construction before dispatch. A reserved `ExtraParams["model"]` override - currently returns a completed report with nil Go error. - -These are the three implementation findings in the error-handling audit, not -new duplicate defects. Re-running its temporary offline public-API diagnostic -against the current tree confirmed all three while the complete suite passed. - -**Potential improvement:** Add narrowly owned regressions with those fixes: -real nested public causes through the injection boundary; a controlled body -reader failing at each decode/framing phase; and a public `SmokeTest` call with -a reserved runtime field and counted transport. Assert relevant error identities, -cause preservation, factual failure classification, and whether dispatch occurred. -Avoid reproducing the internal error topology by hand or matching complete -messages. Preserve intentional nil-error diagnostic findings, cancellation -precedence, best-effort non-2xx detail parsing, and status-only retry policy. - -## 5. Multi-Resource Schema Tests Do Not Establish A Usable Provider Constraint - -**Priority:** P2. Missing assembled validation-to-wire assertion. - -**Relevant tests:** [prepared execution source freezing](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/prepared_execution_contract_test.go#L19), -its [referenced schema fixture](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/prepared_execution_contract_test.go#L809), -and [local reference validation](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/internal/validate/standard_validator_test.go#L627). - -**Deficiency:** Local tests establish that referenced schemas compile and -validate. The prepared-execution test checks that the injected client's schema -matches frozen details. Its opaque model client accepts that schema without -interpreting references. Equality between those two values does not establish -that the provider-facing constraint contains the resources it needs. - -**Unverified behavior:** Both sides can contain the same unresolved `value.json` -reference while the test passes. This is the packaging gap already identified -in [the correctness audit](01-correctness.md#2-external-schema-resources-do-not-reach-the-provider-constraint). -The freezing test's fake is appropriate for its stated ownership purpose; the -missing protection is a separate built-in-path integration case. - -**Potential improvement:** Once the schema-packaging policy is selected, add one -offline assembled test with a root and child schema, capture the actual built-in -HTTP payload, and assess the chosen provider-facing representation independently -of `Details`. For a bundled native schema, compile/evaluate the captured -constraint without granting access to PromptKit's original local files; if the -chosen policy supports a local-only fallback, assert that behavior instead. -Keep the existing freezing test. Do not require live provider acceptance or -promise support for arbitrary provider schema dialect features. - -## 6. Artifact Tests And Benchmarks Pin An Explicitly Opaque Hash Algorithm - -**Priority:** P3. Remove or rewrite change-detector assertions, retaining the -substantive file-reading tests. - -**Relevant assertions:** [size-hint test](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/internal/artifact/reader_test.go#L287) -and [benchmark correctness check](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/internal/artifact/reader_benchmark_test.go#L156). - -**Deficiency:** Both independently calculate a hex SHA-256 digest and require -that exact value. The [public Artifact.Hash contract](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/types.go#L290) -expressly leaves its format and algorithm unspecified. A legitimate algorithm -or encoding change would fail tests unrelated to the requested behavior. - -**Potential improvement:** Replace those algorithm-specific checks with the -opaque equality relationships needed for each test. For the under/overreported -size cases, comparing with the inline reader for the same complete body still -detects hashing only a reported-size prefix. The existing -[source-parity test](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/internal/artifact/reader_test.go#L27) already owns -repeatability, same-content equality, and changed-content inequality. Retain -body, actual-size, metadata, and read-boundary assertions. The benchmark may -reuse an opaque expected hash prepared outside measurement or rely on the -owning behavioral test; it need not maintain its own SHA-256 contract. - -## 7. Bundled Catalog Expectations Have Multiple Test Owners - -**Priority:** P3. Consolidation/removal candidate for duplicated assertions. - -**Relevant tests:** [internal complete catalog table](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/internal/configuration/bundled_test.go#L12), -[public option/inventory table](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/bundled_openrouter_contract_test.go#L54), -[HTTP payload matrix](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/bundled_execution_contract_test.go#L51), and -[bundled smoke defaults](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/f7085d7/bundled_smoke_contract_test.go#L16). - -**Deficiency:** The current count of 17, complete ID inventory, selected -ID-to-model mappings, tier-plugin data, and diagnostic default model recur -across configuration, facade, wire, and diagnostic tests. A catalog update can -require expectation edits in several layers even when loading, option handling, -routing, and serialization are unchanged. The internal plugin assertion also -searches `fmt.Sprint` output instead of comparing the plugin's structure. - -**Potential improvement:** Choose one owner for the complete maintained asset -inventory and expected model/plugin/default data. Remove duplicate full-catalog -counts/lists from option permutations and keep those tests focused on enable, -disable, last-option selection, isolation, and override behavior. Retain a small -assembled wire/diagnostic set for distinct integration risks; controlled -overrides can isolate routing assertions from a changing catalog default. -Use semantic structural assertions for plugin data. Keep independent expected -asset values in the selected owner rather than deriving every expectation -from the production loader, which could conceal an omitted or incorrect asset. -This is consolidation, not a proposal to stop checking bundled configuration. - -## Findings Not Supported And Work To Preserve - -- No general test-flakiness, default-suite live-provider dependency, or race - failure was reproduced. Bounded timeout guards and channel synchronization - are not classified as flaky merely because a test involves concurrency. -- The public API/type-boundary checks and their checker fixtures protect an - explicit architectural rule; they are not automatically obsolete because - they inspect structure. Stable JSON shape tests protect selected contracts. -- Fakes at model/HTTP boundaries, counted paid-effect calls, and no-work-after- - rejection checks are appropriate. A fake is not deficient merely because it - does not model unrelated provider behavior. -- Exhaustive domain-rule tables plus representative boundary wiring tests are - generally sound. No wholesale deletion of error, cancellation, capacity, - ownership, or prepared-lifecycle coverage is recommended. -- Low coverage of thin wrappers, example `main` functions, defensive impossible - branches, and unexercised reflection variants is not independently a finding. - Interface-only packages do not need artificial tests to obtain a package - percentage. Conservative requirement inspection is not obliged to fully - evaluate unsupported template expressions just to cover every AST branch. -- The standalone release-wiring fixture remains scoped to local disposable - repositories under the documented testing model. This audit does not propose - moving it into every normal test run or retesting ReleaseKit's algorithms. - -Prioritize the missing public privacy/source-option protection and regressions -for the already identified defects. Make the two assertion-cleanup changes as -focused rewrites, retaining their useful behavioral coverage. None of these -recommendations requires new public guarantees or a coverage-percentage gate. diff --git a/docs/roadmap/audit/09-interfaces.md b/docs/roadmap/audit/09-interfaces.md deleted file mode 100644 index 3c67e02..0000000 --- a/docs/roadmap/audit/09-interfaces.md +++ /dev/null @@ -1,199 +0,0 @@ -# API And Interface Contract Audit - -Reviewed 2026-11-04 against commit `31f1e6a` and the current worktree. Existing -audit-document moves were preserved. No production code or repository tests -were changed. - -## Scope And Assessment - -The public root package remains a facade over configuration, execution, -generation, capacity, rendering, validation, and observation components. -ConfigKit owns composition before the final configuration handoff; PromptKit -validates that handoff and captures its runtime indexes and templates. Public -adapters translate owned request/result values and errors. Namespace views -share execution ownership and capacity, while separately constructed engines -remain independent. - -The review compared exported GoDoc, configuration and integration contracts, -internal collaborator interfaces, call paths, and existing behavioral tests. -It covered preparation/preflight/execution parity, prepared-handle lifecycle, -optional values and numeric presence, configuration inspection and listing, -injected collaborator ownership, output repair, diagnostics, observations, -serialization, cancellation, and public error categories. It did not treat -different intentionally documented workflows as defects or infer additional -admission, filesystem, provider, or semantic-validation guarantees. - -Four findings warrant remediation. The configuration behaviors in findings 2 -and 3 are consistent with the current format reference and implementation; -their older contradictory descriptions should be corrected rather than used -to change those behaviors. - -## 1. ArtifactReader Permits Buffer Reuse Before PromptKit Can Copy It - -**Priority:** P2. A documented ownership permission permits data races and -inconsistent input snapshots. - -**Relevant interfaces/code:** [ArtifactReader GoDoc](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/31f1e6a/types.go#L294), -[public reader adapter](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/31f1e6a/artifact_reader.go#L19), and -[copyBytes](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/31f1e6a/convert.go#L490). - -**Contract defect:** The interface says that `Read` may be called concurrently -and that the reader may reuse or mutate the returned artifact and body after -`Read` returns. The adapter can only copy those values after that return. -There is no synchronization or acknowledgement between return and completion -of the copy. “Immediately copies” does not make concurrent reuse safe. - -**Circumstances and consequence:** A reader serializes its own `Read` calls -with a mutex and returns a reusable buffer. Another invocation can overwrite -that buffer as soon as the first call returns, while the first adapter is still -copying it. Both calls follow the published reuse permission, yet preparation -can race, mix contents, or associate metadata with the wrong bytes. This is -not a demonstrated defect in the default file reader or in readers that -return independent snapshots. - -**Evidence:** A temporary, offline public-API diagnostic used two concurrent -`Prepare` calls and a mutex-protected reader returning a shared 1 MiB buffer. -`go run -race` reported a write in the next `Read` racing with `copyBytes` -through `publicArtifactReaderAdapter.Read`. The adapter's existing copy test -establishes detachment after copying, not safety during that copy. - -**Potential remediation:** Resolve the return-value ownership contract before -v1.0. The straightforward approach is for a successful `Read` to return a -per-invocation snapshot that remains stable while PromptKit consumes it, with -reuse permitted after the enclosing preparation/execution method returns. -Returning fresh owned data is the simplest implementation for concurrent -readers. If reuse immediately after `Read` itself returns must remain -supported, it requires an explicit borrowing/release arrangement; another -unsynchronized copy cannot provide that permission. Add focused coverage for -the selected lifetime, without globally serializing unrelated engine work. - -## 2. Profile Inspection Misstates Required-Credential Semantics - -**Priority:** P2. The public inspection contract describes valid results as -impossible and incorrectly limits the sources that can satisfy a requirement. - -**Relevant interfaces/code:** [Engine.InspectProfile](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/31f1e6a/engine.go#L313), -[ProfileInspection.APIKeyRequired](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/31f1e6a/types.go#L372), -[profile-inspection consumer example](../../consumers/pkg-promptkit.md#inspect-a-profile-before-prompt-work), -[effective-target construction](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/31f1e6a/internal/configuration/targets.go#L99), -and [credential availability check](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/31f1e6a/internal/usecase/runner.go#L611). - -**Contract defect:** Both method and field GoDoc claim that `APIKeyRequired` -is mutually exclusive with a nonblank `EffectiveModelParams.APIKeyEnv`. They -also describe the flag as requiring a direct credential or a request-level -environment override. In fact, `api_key_required: true` suppresses a -backend-derived environment source only when no explicit profile source was -provided. An explicit profile environment source remains usable, as the -[credential format reference](../../formats.md#credentials) already states. -Inspection can therefore legitimately return both the flag and a nonblank -environment name. The consumer example also calls every reported environment -source optional, regardless of the flag. - -**Circumstances and consequence:** A profile sets `api_key_required: true` and -`api_key_env: PROMPTKIT_INTERFACE_AUDIT_KEY`. Inspection returns `true` and -that name; preparation succeeds using its populated environment variable -without a direct key or request override. Consumers following the GoDoc may -reject this valid combination or report credential readiness incorrectly. - -**Evidence:** A temporary public-API diagnostic demonstrated that combination -and successful preparation. The existing -[effective-target test](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/31f1e6a/internal/configuration/targets_test.go#L88) -also includes a required profile with an explicit environment source. - -**Potential remediation:** Describe the flag as a local credential-availability -requirement, distinguish source selection from required availability, and -remove the mutual-exclusion/direct-only claims. Align the consumer explanation -with the format reference and retain inspection's intentional lack of an -environment-value lookup. Do not reject the valid configuration or change -the public target's stable JSON shape to make the old wording true. - -## 3. The Wire Contract Incorrectly Says Configured Provider Maps Replace - -**Priority:** P2. Conflicting published contracts predict different outbound -provider parameters. - -**Relevant interfaces/code:** [integration request-body contract](../../integrations/openai-compatible-chat.md#request-body), -[profile defaulting reference](../../formats.md#defaults-and-overrides), -[overlayProviderMap](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/31f1e6a/internal/configuration/targets.go#L133), and -[runtime override merge](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/31f1e6a/internal/usecase/runner.go#L562). - -**Contract defect:** The integration document says backend, profile, and request -extra-parameter maps all use whole-map replacement. Actual configured profile -maps recursively overlay backend defaults, including nested maps. A nonempty -runtime `ExecutionTargetOverride.ExtraParams` instead replaces the entire -configured map. The format reference and runtime-override GoDoc describe -these distinct behaviors correctly. - -**Circumstances and consequence:** A backend supplies `backend_only: true` and -`nested: {backend: true}`; its profile supplies `profile_only: true` and -`nested: {profile: true}`. All entries survive into the effective target and -built-in HTTP body. A reader of the integration contract would expect the -backend entries to disappear, potentially leading to surprising provider -options or ineffective attempts to clear defaults. A runtime map containing -only `request_only: true` does remove the configured entries. - -**Evidence:** A temporary public-API diagnostic observed both forms through an -injected client and confirmed the configured merged fields through a controlled -built-in HTTP transport. No external request was made. The existing -[effective-target test](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/31f1e6a/internal/configuration/targets_test.go#L70) -also exercises recursive configured overlay. - -**Potential remediation:** Replace the stale configuration-precedence claim -with a short link to the format reference. Keep the integration contract -focused on flattening the resulting effective map into the HTTP body and on -reserved request fields. Preserve the distinct runtime replacement behavior; -do not add another merge policy or duplicate its full configuration reference. - -## 4. Identity GoDoc Still Describes Removed Normalization And Version Fields - -**Priority:** P3. Small but concrete public-contract inconsistencies. - -**Relevant interfaces/code:** [ProfileInspection.ProfileID](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/31f1e6a/types.go#L364), -[ListPrompts GoDoc](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/31f1e6a/listing.go#L90), and -[literal profile identity test](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/31f1e6a/profile_inspection_contract_test.go#L13). - -**Contract defect:** `ProfileInspection.ProfileID` is described as trimmed, -but configured IDs and lookups preserve meaningful whitespace exactly. -`ListPrompts` promises ownership of each `Version` pointer even though -`PromptIdentity` contains only `ID`; collection-level `ReleaseVersion` belongs -to the separate collection inventory. - -**Circumstances and consequence:** Inspecting the valid profile `" padded "` -returns that exact ID, while looking up `"padded"` fails. Callers expecting the -documented trimmed identity can fail later lookups. The nonexistent version -field creates an erroneous expectation about available prompt metadata. - -**Evidence:** The existing literal-identity test explicitly protects preserved -whitespace and failure of the trimmed lookup; a temporary public-API diagnostic -confirmed it. The current `PromptIdentity` declaration and facade conversion -have no version field. - -**Potential remediation:** State that the profile ID is the exact selected -identity after any namespace qualification, and remove the nonexistent version -pointer language. Keep literal identity behavior and collection metadata -unchanged; no new normalization, version field, or compatibility guarantee is -needed. - -## Existing Findings And Validation - -The earlier error-handling audit already records loss of owning public error -categories for nested injected causes, loss of successful-body I/O causes, and -pre-dispatch smoke failures incorrectly becoming concluded reports. Those are -also interface-contract concerns, not new duplicate findings here. The hidden -`ValidationPreparer` requirement behind the runner's `Validator` parameter is -already recorded in the [complexity audit](03-complexity.md). No additional -independent defect was established in the reviewed prepared-handle lifecycle, -optional-value handling, inspection ownership, or observation/result mapping. - -Focused existing tests passed with the local Go 1.27.0 toolchain: - -```sh -go test -race -count=1 -run 'Test(InspectProfile|ListIdentities|ScopedIdentityListings|PublicArtifactReaderAdapter|PublicLLMClientAdapter|ConfigurationInspection|LoadBuildsEffectiveTargetsAfterInheritance)' . ./internal/configuration -``` - -The separate temporary reader-reuse diagnostic intentionally produced a race -report and exited unsuccessfully, substantiating finding 1 despite the green -existing tests. Temporary programs and outputs remain outside the repository. -All diagnostics were offline with synthetic configuration and credentials; -no paid provider calls, new repository tests, or production changes were used. -These checks are review evidence, not additional release gates. diff --git a/docs/roadmap/audit/10-equivalence.md b/docs/roadmap/audit/10-equivalence.md deleted file mode 100644 index 75ec7e0..0000000 --- a/docs/roadmap/audit/10-equivalence.md +++ /dev/null @@ -1,249 +0,0 @@ -# Behavioral Equivalence And Shared Abstraction Audit - -Reviewed 2026-11-04 against commit `5177db0` and the current worktree. Existing -audit-document moves were preserved. No production code or repository tests -were changed. - -## Assessment And Method - -The public package adapts consumer values into internal configuration and -execution components. ConfigKit owns source composition; PromptKit interprets -its final handoff and captures indexes and templates. The runner shares -preparation and execution machinery across public workflows, generation owns -attempts and retries, and capacity owns admission and permits. Domain rules, -validation, JSON-value ownership, and observation have existing shared owners. -The architecture does not need another public abstraction or a general-purpose -utility package to address the opportunities below. - -This review started from tests: direct cross-path comparisons, common test -matrices, and separately asserted matching behavior. It then examined the -corresponding implementation and callers to distinguish a shared responsibility -from similar-looking code. Matching syntax, similar coverage, or duplicated -test setup alone did not establish an extraction opportunity. - -Three bounded refactoring opportunities warrant consideration. They provide -test-backed scope for findings 1, 4, and 3 respectively in the -[complexity audit](03-complexity.md), rather than three additional independent -bugs or duplicate remediation projects. None is a newly demonstrated behavioral -failure or an automatic release blocker. Existing behavior, not a stronger -equivalence promise, is the preservation target. - -## 1. Share The Schema Compilation Lifecycle Across Source And Execution Paths - -### Evidence - -- [TestJSONSchemaNumericConstraintsRetainJSONPrecision](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5177db0/internal/validate/standard_validator_test.go#L721) - runs the same exact-number constraints through operating-system and `fs.FS` - validators using `jsonSchemaValidatorSources`. Both must distinguish adjacent - large integers, preserve precise decimal boundaries, and accept the same - valid instances. -- [TestJSONSchemaDecodingRequiresOneCompleteDocument](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5177db0/internal/validate/standard_validator_test.go#L805) - uses that same source matrix to require equivalent schema/instance framing - and the distinction between operational schema errors and rejected output. -- [TestSchemaReferencesPreserveEscapedFilenames](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5177db0/internal/validate/standard_validator_test.go#L492) - applies the same reference cases to both source implementations, with an - explicit platform exception for filenames illegal on Windows. -- [TestStandardValidatorPreparedSchemaSurvivesSourceRemoval](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5177db0/internal/validate/standard_validator_test.go#L228) - and [TestFSValidatorPreparedSchemaSurvivesSourceMutation](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5177db0/internal/validate/standard_validator_test.go#L416) - separately require the same captured schema to accept an integer-valued - object and reject a string-valued one after its source changes. Their source - operations differ, but the prepared validation responsibility matches. - -These tests support common schema semantics and compilation ownership; they do -not require identical filesystem representations or diagnostic strings. - -### Implementation - -[StandardValidator and FSValidator](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5177db0/internal/validate/standard_validator.go#L74) -each implement `PrepareValidation` and direct `validateJSONSchema`. The four -paths repeat compiler setup, root-resource handling, cancellation boundaries, -compilation/error wrapping, and construction or use of executable schema -state. The direct host-file path lets its loader fetch the root, whereas the -other paths explicitly load/register it. Decoding, compiler construction, -schema execution, and content-result classification already have shared helpers; -the remaining duplication is the lifecycle around them. - -### Opportunity And Potential Remediation - -Keep source-specific resolution/loading adapters, but introduce one private -compilation operation in `internal/validate` that produces the root document -and compiled executor for the caller's selected source. Reuse it from prepared -validation and from the direct schema-validation callback. Retain the already -shared artifact-validation/result logic. This makes future dialect, framing, -number-preservation, and cancellation changes less likely to land in only some -of the four paths. - -The existing package is the appropriate owner; neither a public compiler API -nor a generic filesystem framework is needed. The related runner dependency -clarification remains part of complexity-audit finding 1. - -### Tradeoffs And Test Independence - -Direct validation currently rejects malformed output JSON before accessing its -schema. Eagerly implementing it as `PrepareValidation` followed by plan -validation would change I/O and error ordering. Share compilation when that -work is reached, not necessarily the entire entry point. Preserve contained -host paths and symlink handling, logical `fs.FS` paths, escaped resource URLs, -single-file sources, synchronous cancellation precedence, and operation-local -prepared freezing. These source differences belong in their adapters. - -Keep the source matrix and independently specified valid/invalid instances. -Do not compute expected results using the new compilation helper or delete -source-specific containment/cancellation tests merely because compilation is -shared. Provider-facing packaging of external schema references is a separate -correctness finding, not solved by this extraction. - -## 2. Centralize Structural Copying Of Already-Normalized JSON Values - -### Evidence - -- [TestPublicLLMClientAdapterGivesClientOwnedNestedValues](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5177db0/llm_adapter_internal_test.go#L14) - requires nested provider maps, schema values, messages, and cache metadata to - remain independent when an injected client mutates and retains its request. -- [TestRunnerPreparedExecutionDetachesTransferredSnapshotSources](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5177db0/internal/usecase/prepared_execution_test.go#L634) - separately requires nested provider maps and schema values to survive source - mutation, repeated `Details` calls, and execution without aliasing. -- [TestCopyHandlesIndirectionAndIsolatesNestedMutations](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5177db0/internal/jsonvalue/jsonvalue_test.go#L156) - exercises the underlying JSON-value ownership responsibility with typed - collections and repeated acyclic children. The adjacent scalar/collection - tests protect compatible concrete types and exact `json.Number` values. -- [TestCopyAndCopyMapApplyDistinctEmptyKeyRules](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5177db0/internal/jsonvalue/jsonvalue_test.go#L204) - explicitly demonstrates a necessary policy distinction: general schema - objects can have empty keys, while provider maps cannot. - -The common requirement is an independently owned copy of accepted nested -values. These tests do not establish universal equivalence of validation, -normalization, and copying on arbitrary Go values. - -### Implementation - -The facade's [copyAny/copyReflectValue](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5177db0/convert.go#L406) implements a -separate recursive reflection copier for values projected into client requests, -prepared details, results, and profile inspections. Internal prepared-state -cloning uses [clonePreparedRun](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5177db0/internal/usecase/prepared_execution.go#L226) -and [jsonvalue.Copy/CopyMap](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5177db0/internal/jsonvalue/jsonvalue.go#L30). -The latter package already owns compatible JSON-tree copying and validation. - -Ordinary runtime provider values reach facade projection after JSON-value -validation/copying. The runner also copies its schema document through -[structuredOutputFromValidationPlan](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5177db0/internal/usecase/runner.go#L419) -before projecting it. Thus these normal runtime paths have a common accepted -value domain, even though their entry-point policies are different. - -### Opportunity And Potential Remediation - -Move ownership of trusted JSON-tree cloning into `internal/jsonvalue`, reusing -its compatible structural-copy machinery where semantics match, so the facade -does not maintain another general reflection traversal. Keep public/domain DTO -field projection in the facade. A small internal trusted-value clone operation -is a possible approach; reuse of existing fallible operations is another where -the caller can preserve the existing error boundary. Select the simpler option -after inventorying the concrete callers and values, rather than introducing a -configurable reflection framework. - -The meaningful payoff is removing an independent traversal implementation, -not wrapping the existing facade copier under another name. This does not -require consolidating every pointer or slice helper in the repository. - -### Tradeoffs And Test Independence - -`Copy` validates and bounds values and normalizes null/indirection; the facade -copier assumes prior acceptance and can preserve typed nils and pointers. -Neither is a drop-in replacement for the other on all possible inputs. -Preserve observed types, nil behavior, exact numbers, schema empty keys, and -caller ownership at the relevant boundary. An infallible projection must not -silently turn a new copy error into a zero value, a panic, or an additional -consumer restriction. - -The [configuration-boundary ADR](../../adr/0001-configuration-boundary.md) -also remains authoritative. `providerValueForMapping` serves a different, -pre-handoff source-encoding purpose and defers unsupported-value errors to -ConfigKit. It must not be routed through a rejecting runtime copier as part of -this work. Source number conversion is separately recorded in the correctness -audit; visual similarity between its reflection switch and a copier is not -enough to equate their contracts. - -Keep literal expected values and mutation-based assertions in both facade and -internal tests. A shared production copier must not also manufacture their -expected snapshots. Retain focused tests for policy differences rather than -replacing them with one “all copy functions are equivalent” test. - -## 3. Share If/With Requirement-Branch Orchestration, Not Template Execution - -### Evidence - -- [TestAnalyzeRequirementsClassifiesKnownConditions](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5177db0/internal/prompt/requirements_test.go#L268) - verifies required/optional classification for selected and skipped `if` - branches, and directly compares the recognized branch choice with the real - Go renderer. This anchors recognized condition semantics in actual rendering. -- [TestAnalyzeRequirementsHandlesWithDotAndNestedReachability](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5177db0/internal/prompt/requirements_test.go#L332) - requires the same selected/skipped classification for `with` and nested - `if`, while separately protecting `with`'s changed dot and preserved root - caller-map access. -- [TestAnalyzeRequirementsReportsUnresolvedDecisionAndContentConditions](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5177db0/internal/prompt/requirements_test.go#L369) - protects conditional dependencies and meaningful incompleteness for unknown - decisions. It also establishes that an unresolved condition around plain - text alone does not make discovery incomplete. -- [TestAnalyzeRequirementsDoesNotHideCallerMapScopes](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5177db0/internal/prompt/requirements_test.go#L14) - compares several scope forms against actual template execution while allowing - incomplete analysis for constructs outside recognition scope. - -The tests support one reachability/evidence policy for recognizable branches, -with explicit scope differences. They do not require the inspector to execute -or completely interpret arbitrary templates. - -### Implementation - -The `IfNode` and `WithNode` cases in -[requirementWalker.visit](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/5177db0/internal/prompt/requirements.go#L451) -independently perform inactive-path traversal, condition evaluation, then/else -reachability selection, dependency-counter comparison, and limitation recording. -`with` adds a different body-dot calculation. `branchReach`, condition -evaluation, and `template.IsTrue` already provide shared lower-level behavior, -but their surrounding branch protocol is repeated. - -### Opportunity And Potential Remediation - -Extract a small package-private branch traversal operation that owns the common -reachability and evidence sequence, with explicit body/else scope handling. -Leave node recognition and `withDotIdentity` visible at the node-specific -boundary. Keeping this in `internal/prompt` would allow a change to skipped-path -handling or incompleteness reporting to be reviewed and implemented once. - -This is the narrowly supported part of complexity-audit finding 3. It does not -justify sharing the requirement walker with the real template executor or -building an interpreter abstraction. - -### Tradeoffs And Test Independence - -An abstraction that treats `if` and `with` identically would lose caller-map -scope information. Preserve `with`'s body-dot behavior, else scope, inactive -computed-name handling, and the placement of dependency bookkeeping used to -decide whether an unresolved condition matters. A helper with numerous mode -flags or callbacks would obscure those differences; prefer a small explicit -operation over a generalized AST visitor framework. - -Retain separate node-specific assertions and the independent standard-library -rendering comparisons. Extracting production traversal is not a reason to -calculate expected inventories using the production branch helper or to -strengthen `Complete` into an execution-readiness guarantee. - -## Recommendation And Validation - -Start with schema compilation, which has the clearest shared lifecycle and -cross-source test matrix. The branch helper is a smaller local follow-up. -Treat JSON-copy consolidation as a more careful ownership refactor: remove a -traversal only after its caller/policy differences are accounted for. No new -package boundary is necessary solely to carry out these changes, and retaining -small source/boundary-specific adapters is appropriate. - -Existing offline tests passed with the local Go 1.27.0 toolchain: - -```sh -go test -race -count=1 ./internal/validate ./internal/prompt ./internal/jsonvalue -go test -race -count=1 -run 'Test(PublicLLMClientAdapter|PreparedExecutionFreezesSourcesAndReturnsIndependentDetails|RunnerPreparedExecutionDetachesTransferredSnapshotSources|RunnerRunAndPrepareResolveSameProfileAndEffectiveSettings|RunnerPreparationNormalizesOutputContractConsistently|ValidateRequestMatchesPrepareContract)' . ./internal/usecase -``` - -Passing tests substantiate the reviewed current behavior, not equivalence of a -future refactor. No new tests, production changes, live provider requests, -benchmarks, or additional release gates were introduced by this audit. diff --git a/docs/roadmap/audit/11-boundaries.md b/docs/roadmap/audit/11-boundaries.md deleted file mode 100644 index 2ead0c9..0000000 --- a/docs/roadmap/audit/11-boundaries.md +++ /dev/null @@ -1,229 +0,0 @@ -# Package Cohesion And Architectural Boundary Audit - -Reviewed 2026-11-04 against commit `b39460a` and the current worktree. Existing -audit-document moves were preserved. No production code, tests, dependencies, -or functionality were changed. - -## Scope, Method, And Architectural Assessment - -Reviewed the 17 repository-owned internal Go packages, their production import -relationships, engine assembly, relevant callers, and tests. The public facade -was checked where it establishes the boundary around those packages. Third-party -package implementations were outside the audit's scope. - -The [current component inventory](../../internal/overview.md) broadly matches -the implementation. Configuration owns the final ConfigKit handoff and captured -resource indexes; the engine assembles collaborators; use cases coordinate -selection, preparation, execution, and diagnostics. Generation owns dispatch -and retries, capacity owns admission and permits, and outbound transport, -rendering, artifacts, schema validation, observations, and shared values have -identifiable owners. No repository-wide package reorganization is warranted. - -The public facade does not leak internal types or receive reverse production -imports from internal packages in the checks run below. Deliberate boundaries -remain useful: configuration completion differs from request overrides; -diagnostic orchestration differs from ordinary execution; domain invariants -differ from source-specific presence and error handling. A broad package, a -small interface-only package, or an import of another internal component is not -itself a cohesion defect. - -Three focused ownership improvements warrant consideration. They are -maintainability opportunities, not newly demonstrated runtime bugs or automatic -release blockers. - -## 1. Retire The Test-Only Capacity Client As A Production Scheduling Surface - -**Relevant code:** [capacity.NewClient and client.Generate](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/b39460a/internal/capacity/client.go#L17), -[generation constructors and dispatch](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/b39460a/internal/generation/coordinator.go#L37), -[engine assembly](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/b39460a/internal/engine/engine.go#L157), and -[runner scheduling test](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/b39460a/internal/usecase/runner_test.go#L2391). - -**Basis:** The engine selects `generation.NewBuiltIn` or `NewInjected` for every -production engine. The coordinator owns active-permit acquisition around each -attempt, retry waits, observation, and diagnostic evidence. The earlier -`capacity.NewClient` still provides a separate model-client wrapper that -acquires/releases a permit around an opaque call. Both use the same manager; -there are not two independent FIFO algorithms or pools. - -Caller analysis found only tests invoking the older constructor: capacity tests -and a runner test that assembles initial/repair scheduling with this wrapper. -The [capacity document](../../internal/capacity.md#fifo-generation-permits) -acknowledges its retention for focused callers/tests, but no production caller -was found. Its presence leaves capacity owning a model-client adapter after -production dispatch responsibility moved to generation, and makes one runner -assembly test exercise a different scheduling composition than the engine. - -**Consequences:** The extra adapter and its documented role remain maintenance -surface without a production need. Future scheduling changes can require -reasoning about a legacy assembly alongside the real one. Green tests of the -wrapper alone do not demonstrate that coordinator dispatch applies the same -permit lifetime. Existing coordinator and facade tests do protect that actual -path, so this is not a claim that production scheduling is untested or broken. - -**Potential remediation:** Keep the manager and its admission/FIFO primitives -in `internal/capacity`, and generation dispatch integration in -`internal/generation`. Replace the runner test's wrapper with the production -injected coordinator. For focused capacity scenarios, test `Manager.Acquire` -directly or retain the small one-shot adapter as test-owned scaffolding rather -than a production constructor. Once callers are adjusted, the production -wrapper can be retired and its documentation updated. - -Preserve the useful FIFO, cancellation-race, independent-pool, and panic-release -assertions when adjusting test ownership. This finding does not call for moving -queue internals into generation, deleting those scenarios, or wrapping the -coordinator in another capacity client, which could acquire the same limit -twice. Retaining the wrapper is viable if a real internal production need is -identified; current test reachability alone does not establish one. - -## 2. Put OpenRouter Authentication Protocol Details Behind The Outbound Adapter - -**Relevant code:** [authenticationProbe.runOpenRouterKey](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/b39460a/internal/usecase/smoke_authentication.go#L66), -[recognizedOpenRouterKeyResponse](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/b39460a/internal/usecase/smoke_authentication.go#L139), -[diagnostic endpoint derivation](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/b39460a/internal/usecase/smoke_selection.go#L166), -and [HTTP authentication transport](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/b39460a/internal/llm/http_evidence.go#L111). - -**Basis:** `internal/usecase` currently interprets OpenRouter-specific HTTP -statuses and the account-response keys `data`, `label`, `is_free_tier`, and -`usage`. It owns JSON envelope parsing, identifying-field type checks, and body -consumption. The same package derives the OpenRouter-relative `/key` endpoint. -Meanwhile `internal/llm` owns authenticated HTTP execution, credential -resolution, evidence collection, and response ownership mechanics. One provider -integration is therefore divided between the application coordinator and its -outbound adapter. - -This division is different from a use case consuming factual HTTP evidence: -the precise external account document and provider endpoint are protocol -knowledge. The [architecture policy](../../policy/architecture.md#architectural-approach) -calls for encapsulating external integration details in internal adapters. -The parser is already functionally isolated, making the ownership seam fairly -small. The authentication tests in -[smoke_authentication_test.go](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/b39460a/internal/usecase/smoke_authentication_test.go#L64) -also show that this is provider protocol behavior rather than application -selection policy. - -**Consequences:** A change to OpenRouter's account endpoint or identifying -response fields requires changing application-layer code as well as the -transport integration. Adding another authentication adapter would extend -provider-specific branching inside the use-case package. This complicates the -otherwise clear distinction between choosing a diagnostic and implementing -the upstream protocol. - -**Potential remediation:** Move the OpenRouter request/response adapter, -including its default destination and response recognition, into -`internal/llm` alongside the existing diagnostic transport. Have the use case -consume a small factual outcome/evidence value. Keep target selection, -configured applicability, unsupported-operation reporting, caller cancellation, -overall completion, and report assembly in the use case. The existing -consumer-owned diagnostic interface is a useful seam; engine assembly can -supply the concrete outbound implementation without requiring protocol parsing -inside the coordinator. - -An alternative is a focused internal diagnostic-adapter package if separation -from inference transport becomes useful. Another package or a provider plugin -framework is not necessary for the current single protocol adapter. Do not -introduce a new dependency from the outbound adapter back into `usecase`, or -move application report/completion policy into transport merely to relocate -the parser. The diagnostic feature legitimately deals with HTTP facts; a -blanket prohibition on HTTP types or status observations in use cases is not -implied by this finding. - -Preserve anonymous authentication probing, explicit endpoint overrides, -neutral tri-state facts, private body/credential handling, and the absence of -ordinary observer events. The previously recorded unbounded authentication-body -read is a separate resource/security finding; relocation alone does not fix it. - -## 3. Narrow Smoke-Value Copy Helpers To Their Actual Ownership Boundaries - -**Relevant code:** [domain.CloneSmokeTestRequest](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/b39460a/internal/domain/smoke_diagnostics.go#L157), -[domain.CloneSmokeTestReport](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/b39460a/internal/domain/smoke_diagnostics.go#L179), -[smokeReportTarget](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/b39460a/internal/usecase/smoke_tester.go#L432), and -[public smoke-value conversion](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/b39460a/smoke_diagnostics.go#L279). - -**Basis:** `CloneSmokeTestRequest` is called only by its own domain test; -actual request ownership is established by public-to-domain conversion. -`CloneSmokeTestReport` has one production caller: `smokeReportTarget` creates -an otherwise empty report containing a target, clones the full report, and -extracts just that target. Whole-report facts, timing, and attempt cloning have -no runtime use through that caller. Public report conversion independently -copies the actual facts, timings, attempts, and provider map into public types. - -Thus the domain package maintains a broad request/report-copy API, but the -real internal consumer needs only a detached target/provider map. The -whole-report ownership functionality is exercised by its domain test rather -than by actual whole-report production use. Public DTO conversion is a -legitimate separate responsibility, not something that should disappear into -domain types or public aliases. - -**Consequences:** Smoke-value changes have additional copy lists and tests to -maintain even where those copies are not needed in production. A temporary -report container obscures the narrow target-detachment operation. Keeping a -general cloner solely because its own test calls it reverses the relationship -between production responsibility and supporting tests. - -**Potential remediation:** Express the target-copy operation directly at its -use-case boundary using existing JSON-value ownership, or a small target clone -helper if actual callers justify one. Retain public request/report projection -in the facade and shared JSON-tree copying in its existing owner. Retire the -unused request cloner and the unnecessary whole-report clone surface after -confirming no additional callers need them. Remove only clone-specific helper -code that is then unused. The backend-cloning helpers have real registry -consumers and are outside the proposed retirement; retain that metadata -ownership behavior. - -Preserve provider-map error propagation, caller-owned result values, and -credential/endpoint redaction. Keep the meaningful -[public conversion ownership test](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/b39460a/smoke_conversion_internal_test.go#L13) -and relevant use-case report tests. If genuine internal whole-report snapshots -are needed, retaining an explicitly used domain cloner is a reasonable -alternative. This finding does not require sharing public/internal DTO types -or routing public conversion through a full internal clone followed by another -copy. Coordinate any JSON traversal consolidation with -[audit 10](10-equivalence.md#2-centralize-structural-copying-of-already-normalized-json-values). - -## Related Existing Findings - -The runner's declared `Validator` dependency and actual `ValidationPreparer` -requirement remain the interface-boundary clarification in -[complexity-audit finding 1](03-complexity.md#1-consolidate-schema-compilation-and-express-the-runners-actual-interface). -The facade's generic JSON reflection copier has an existing internal owner -identified in audit 10. These are relevant ownership concerns, not additional -independent findings to count again. Large orchestration functions and repeated -schema compilation within an already coherent package likewise retain their -existing complexity/equivalence remediation scope. - -## Reachability And Validation Evidence - -Production imports were enumerated with `go list` and compared with the graph -and source callers. Rapid Type Analysis used the locally available pinned -`golang.org/x/tools` v0.52.0 command, with reporting restricted to -repository-owned internal packages: - -```sh -go list -f '{{.ImportPath}}|{{join .Imports " "}}' ./internal/... -go run golang.org/x/tools/cmd/deadcode@v0.52.0 -test -filter='^go.maximumdirect.net/promptkit/internal/' ./... -go run golang.org/x/tools/cmd/deadcode@v0.52.0 -filter='^go.maximumdirect.net/promptkit/internal/' ./... -``` - -With tests included, the tool reported only the unused test helper -`internal/usecase/runner_test.go:stringPtr`. That small test cleanup does not -justify a separate architectural finding. Test-only production helpers remain -reachable to this analysis when their tests call them. - -Without tests, the roots are the repository's example executables, not every -possible downstream library consumer. The output includes valid public -workflows that those examples do not exercise, so it is not a deletion list. -The capacity-wrapper and request-cloner observations above were additionally -checked against actual source references and assembly; graph in-degree or -example reachability alone was not treated as proof of obsolescence. The -analysis also covers only the selected local build configuration. - -The following existing offline checks passed with local Go 1.27.0: - -```sh -go test -race -count=1 -run 'Test(PublicFacadeDoesNotExposeInternalTypes|InternalProductionPackagesDoNotImportFacade|RepositoryDoesNotImportFormerModule|SmokeTestConcludedInferenceMismatchHasNilGoErrorAndDetachedValues)' . -go test -race -count=1 ./internal/capacity ./internal/generation ./internal/domain ./internal/usecase -``` - -No new release gate, consumer contract, provider call, production edit, or test -removal is proposed as part of this audit pass. The tools left module metadata -unchanged. diff --git a/docs/roadmap/audit/12-deadcode.md b/docs/roadmap/audit/12-deadcode.md deleted file mode 100644 index b8f1a57..0000000 --- a/docs/roadmap/audit/12-deadcode.md +++ /dev/null @@ -1,233 +0,0 @@ -# Dead Code And Obsolete Functionality Audit - -Reviewed 2026-11-04 against commit `3b76031` and the current worktree. Existing -audit-document moves were preserved. No production code, tests, dependencies, -or functionality were changed. - -## Scope And Assessment - -Reviewed repository-owned Go packages, public entry points, configuration and -runtime values, collaborator interfaces, default/optional paths, embedded -resources, build constraints, and the release launchers and their wiring test. -The architecture remains a public facade over captured configuration, use-case -orchestration, generation/capacity, and focused internal adapters. The former -standalone configuration repositories have already been retired; this review -found smaller residues rather than another abandoned subsystem. - -The review combined Rapid Type Analysis, Staticcheck unused/data-flow checks, -graph caller discovery, source inspection, and the format, internal, policy, -and roadmap documents. Tests count as real callers, but a helper's self-test -does not by itself establish a production need. Conversely, absence from the -examples or default execution path is not evidence that an optional public -feature is obsolete. - -Four cleanup areas warrant consideration. These are low-priority reductions in -state and maintenance surface, not newly demonstrated runtime bugs. None calls -for deleting a supported public feature or imposing a new release gate. - -## 1. Remove Backend Presence Bookkeeping With No Runtime Consumer - -**Relevant code:** [loaded.backends](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/3b76031/internal/configuration/load.go#L122), -[backendFields and presence capture](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/3b76031/internal/configuration/load.go#L215), -[effective backend construction](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/3b76031/internal/configuration/targets.go#L14), -and [Backend.QueueCapacitySet](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/3b76031/internal/domain/domain.go#L187). - -**Evidence:** Construction allocates a per-backend field-state map and fills it -through `captureFields(..., state.backends)`. No production code reads the -recorded backend states. The only lookup of the captured queue state is in -[TestLoadDefaultsAbsentQueueCapacity](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/3b76031/internal/configuration/load_test.go#L755). -Unlike profile presence, backend presence is not consumed while constructing -effective execution targets. - -`QueueCapacitySet` is also assigned during target construction and asserted in -tests, but never read to select a production behavior. The current assignment -is simply whether concurrency is positive; it no longer distinguishes an -explicit queue setting from a derived default. The registry's capacity-policy -projection uses `ConcurrencyLimit` and `QueueCapacity`, not this flag. - -**Why obsolete:** Queue defaulting and its provenance are already handled by -[resolveBackends](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/3b76031/internal/configuration/resolve_targets.go#L11) and -ConfigKit's derived-default record. Typed final values drive runtime capacity; -the retained side map and Boolean do not contribute to that behavior or to the -public inspection view. No documented compatibility or planned-feature need for -these extra records was found. - -**Potential remediation:** Remove the backend-only presence capture, its field -list and retained map, and the unused runtime Boolean/assignment. Adjust tests -to assert effective queue behavior and, where relevant, the existing ConfigKit -inspection/provenance result. Keep `captureFields`, profile field-state -tracking, and prompt-input metadata presence: those have actual consumers and -preserve required absent/null/explicit-value distinctions. Retain explicit -zero queue capacity, derived defaulting, and operator history unchanged. - -This removes unnecessary construction work and retained per-backend maps; it -does not justify dropping configuration provenance or changing its ownership -under the [configuration-boundary ADR](../../adr/0001-configuration-boundary.md). - -## 2. Retire Legacy Fields In The Captured Prompt Representation - -**Relevant code:** [PromptDefinition.OutputFormat](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/3b76031/internal/domain/domain.go#L148), -[PromptMessageTemplate.ContentFile](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/3b76031/internal/domain/domain.go#L173), -[prompt materialization](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/3b76031/internal/configuration/prompts.go#L58), -[resolveOutputContract](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/3b76031/internal/usecase/runner.go#L639), and -[definition hashing](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/3b76031/internal/promptdef/hash.go#L12). - -**Evidence:** Supported engine construction requires and validates a complete -output contract. Materialization sets `definition.Validation` to that contract -and duplicates its format into `definition.OutputFormat`. The runner reads -the duplicate only when `Validation.Format` is empty, a condition that the -captured definitions supplied by engine construction cannot have. The standard -runner test-definition helper fills both fields with the same value; no -supported public repository-injection path supplies an alternative definition. - -For messages, materialization resolves the real source `content_file` into -captured `Content` and does not populate the domain `ContentFile` field. The -renderer uses `Content`/captured templates, not that path field. Its one -nonempty assignment is a renderer fixture demonstrating that already-loaded -content is rendered without opening a file. The source-format -`promptMessageConfig.ContentFile` remains actively used and is a different -field. - -**Why obsolete:** The runtime representation still carries source-era message -path data and a second output-format source after configuration loading became -the authoritative materialization boundary. The empty-format fallback is -disconnected from supported engine definitions. No current documentation or -roadmap records a compatibility requirement for these internal duplicate fields. - -**Potential remediation:** Use `PromptDefinition.Validation.Format` as the one -captured format and remove the legacy duplicate/fallback. Remove only the -runtime message path field, adjusting its fixture to exercise captured text. -Preserve the actual YAML `content_file` feature, source-relative resource -resolution, ConfigKit origin inspection, and construction-time capture. Also -preserve the separate, supported request-override rule that an empty replacement -`OutputContract.Format` defaults to text. - -**Serialization caveat:** These fields are included indirectly by -`HashDefinition`'s `json.Marshal`, even when a message path is empty. They are -therefore not completely inert bytes: removing them changes current prompt -hash values. Account for that effect explicitly and protect the documented -opaque-equality relationships between inspection and preparation. Do not claim -bit-identical hashes after changing the hash input, or invent a permanent hash -encoding guarantee merely to perform this cleanup. The fields are internal -representation, not part of the public stable DTO shapes. - -## 3. Remove Unused Diagnostic State And A Discarded Helper Result - -**Relevant code:** [reachabilityProbeOutcome.initialEndpoint](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/3b76031/internal/usecase/smoke_reachability.go#L24), -[HTTP report assembly](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/3b76031/internal/usecase/smoke_tester.go#L376), -[fileSource](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/3b76031/internal/engine/engine.go#L211), and -[SetSchemaFile](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/3b76031/internal/engine/engine.go#L75). - -**Evidence:** The reachability probe assigns its private outcome's -`initialEndpoint`, but the smoke tester never reads that member. It records -the real public attempt's `InitialEndpoint` from the resolved report target. -Only `TestReachabilityProbeRemovesURLCredentials` inspects the private echo. -Actual sanitized request/evidence and report behavior are established through -other values. - -Separately, `fileSource` returns a filesystem, file name, source-directory -string, and error. Its only caller, `SetSchemaFile`, always discards the -directory result. The directory is still needed internally to construct -`os.DirFS`; exposing it as another return value serves no caller. - -**Potential remediation:** Remove the unused probe-outcome member and its -assignment; retain credential-removal assertions against the actual request, -evidence, or public diagnostic report. Reduce `fileSource`'s private signature -to the consumed results. Keep its directory calculation, file checks, selected -filesystem/root, and the public `WithSchemaFile` option intact. - -Neither cleanup removes `SmokeTestAttempt.InitialEndpoint`, destination -evidence, or single-schema-file support. Those public features are live even -though an extra private field/result is unnecessary. - -## 4. Remove Orphan Helpers Whose Only Callers Are Their Own Tests - -**Relevant code and evidence:** - -- [filecatalog.RelativePath](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/3b76031/internal/filecatalog/catalog.go#L71) is - called only by `TestRelativePathNested`. Current source discovery and - source-relative resource resolution use other helpers; no production or - public-facade caller uses this wrapper around `filepath.Rel`. -- [resourceNamespaceFromID](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/3b76031/internal/domain/resource_identity.go#L34) - is called only by `TestResourceNamespaceFromID`. Actual namespace inventory - and filtering paths do not call it. The exported identity-validation and - qualification operations remain active and are not candidates for removal. -- [runner_test.go:stringPtr](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/3b76031/internal/usecase/runner_test.go#L2592) - has no callers, including in tests. Both reachability analysis and - Staticcheck `U1000` report it. - -Caller searches found no reflection, registration, build-constrained caller, -or supported external API exposing these helpers. The first two belong to -internal packages and have only their direct self-tests; no documented future -or compatibility purpose was found. The third is already test-only and unused. - -**Potential remediation:** Remove the orphan helpers and their now-irrelevant -self-tests, keeping behavioral coverage of real discovery, namespace -inventory/filtering, and identity qualification. Remove the unused pointer -helper without replacing it. There is no reason to introduce a new abstraction -or move the self-tests to another layer solely to preserve these unused APIs. - -## Intentional Retention And Related Findings - -The following distinctions prevent removal based on misleading reachability: - -- `WithSchemaFile` and `InlineWithURI` are supported public helpers despite - being unreachable from the current test executables. The former's missing - public wiring coverage is already in [audit 08](08-testing.md); it is not - evidence that the option should be deleted. -- The older capacity client wrapper and no-repair runner constructor are - expressly retained for focused internal callers/tests in the current - [capacity](../../internal/capacity.md#fifo-generation-permits) and - [runner](../../internal/runner.md#collaborators) documentation. This audit - does not classify them as dead functionality or recommend deleting them. - [Audit 11](11-boundaries.md) discusses the wrapper's architectural placement; - altering that deliberate retention is a separate decision. -- Repository capability fallbacks, standalone rendering/validation, injected - clients' nil usage-presence compatibility, nil-validator/admitter behavior, - and optional provider settings have meaningful callers or documented - purposes. Normal engine assembly taking another path does not obsolete them. -- ConfigKit YAML fields and stable JSON DTO fields can be used by reflection; - formatter methods and interfaces can be invoked indirectly. Embedded - OpenRouter and smoke-schema assets, the Linux FIFO test, and release scripts - likewise have consumers outside a simple direct-call list. They were not - treated as dead. No unused public configuration option or abandoned release - launcher was established. - -The domain smoke-request/full-report cloners have a narrow-or-unused consumer -surface already described in [audit 11, finding 3](11-boundaries.md#3-narrow-smoke-value-copy-helpers-to-their-actual-ownership-boundaries). -That ownership/remediation scope is not counted again here. The schema and -JSON-copy consolidations in prior audits likewise remain their existing work, -not an excuse to delete intentional alternative implementations wholesale. - -## Tooling And Validation - -The checks used local Go 1.27.0 and pinned audit-tool versions: - -```sh -go run golang.org/x/tools/cmd/deadcode@v0.52.0 -test -filter='^go.maximumdirect.net/promptkit(/|$)' ./... -go run honnef.co/go/tools/cmd/staticcheck@v0.8.2 -checks 'U1000,SA4*' ./... -go list -f '{{.ImportPath}} {{join .GoFiles " "}} {{join .IgnoredGoFiles " "}}' ./... -``` - -`deadcode -test` reported the two public helpers and unused test pointer helper -listed above. Staticcheck reported only `stringPtr` (`U1000`) and exited with -status 1 because of that finding; it reported no additional selected `SA4*` -diagnostics. These are audit observations, not successful lint sign-offs. -Neither tool establishes that all tested or serialized state is necessary, -which is why the write-only and redundant fields required manual review. - -Rapid Type Analysis starts from executable roots; tests add roots, and example -programs do not exercise every library consumer path. The analysis is local-build -evidence, not proof about all platforms or downstream programs. In particular, -platform-specific schema handling and optional library operations were retained. - -Existing offline race-enabled tests passed: - -```sh -go test -race -count=1 ./internal/configuration ./internal/domain ./internal/filecatalog ./internal/prompt ./internal/engine ./internal/usecase -go test -race -count=1 -run 'Test(ConfigurationInspection|PreparedExecution|ValidateRequestMatchesPrepareContract|AppendedMessagesComposeFrozenEffectivePrompt)' . -``` - -Tool execution left `go.mod` and `go.sum` unchanged. No production changes, -test removals, provider calls, or new validation gates were made during this -review. diff --git a/docs/roadmap/audit/A1-responsibility.md b/docs/roadmap/audit/A1-responsibility.md deleted file mode 100644 index 18317f9..0000000 --- a/docs/roadmap/audit/A1-responsibility.md +++ /dev/null @@ -1,432 +0,0 @@ -# A1: Responsibility And Removable Machinery - -## Scope And Basis - -Architectural review of repository-owned code at -`1ff6d778af5209a91d60a9452847c2db20dd7932`, on 2026-11-05. This review follows -the [agent](../../policy/agents.md), [architecture](../../policy/architecture.md), -and [testing](../../policy/testing.md) policies. It is independent of the -completed [earlier remediation](../remediation.md). - -The review covered the public facade, internal package boundaries, engine -assembly, configuration capture and inspection, rendering and requirements, -artifact loading, generation/capacity, validation/repair, diagnostics, -observations/errors, examples, and release wiring. Graph caller searches were -checked against implementations and assembly; test-only callers were considered -separately. Dependency public contracts informed delegation choices; dependency -implementations were not audited. No production code or tests were changed. - -These findings are not a claim that every candidate is a bug or release blocker. -The evidence describes the audited baseline; subsequent review decisions are -recorded below. Findings 1–7 are accepted for remediation, and finding 8 is -accepted as a decision to preserve current runtime behavior. Implementation -has not begun under this document. All findings have an accepted disposition. - -Existing explicit decisions remain authoritative except where a later accepted -decision deliberately changes them. A test, private interface, or historical -implementation alone does not establish an additional consumer contract. - -## Current Architecture - -The root package is a public facade over an internal engine. ConfigKit owns -source composition, inheritance, and provenance; `internal/configuration` -interprets its final handoff and captures domain definitions, effective targets, -templates, inventories, and operator inspection. This follows -[ADR 0001](../../adr/0001-configuration-boundary.md). - -Use cases coordinate offline preparation, execution, bounded output repair, and -explicit diagnostics. Generation owns built-in retry orchestration and active -permits; capacity also owns logical-execution admission. The LLM adapter owns -HTTP/wire behavior. The facade owns public value projection, mutable-value -detachment, credential redaction, and public error translation. The -[component overview](../../internal/overview.md) is the maintained inventory. - -The overall separation is useful. The strongest simplification opportunities -are inside those boundaries: alternate private workflows, repeated projection, -and repeated acceptance checks on already captured values. - -## Findings: Unnecessary Implementation Complexity - -### 1. Remove Unused Client-Level Endpoint And Model Defaults - -**Status:** Accepted; implementation pending. - -**Confidence:** High. **Owner:** resolved execution settings belong to -configuration/request resolution; HTTP transport consumes the resolved target. - -**Evidence:** In the [LLM client](../../../internal/llm/openai_compatible_client.go), -`OpenAICompatibleConfig.BaseURL` and `.Model` become `baseURL` and `defaultModel`. -`PrepareGeneration` falls back to them when request fields are empty. The sole -non-test constructor caller, [engine assembly](../../../internal/engine/engine.go), -supplies only `Timeout` and `HTTPClient`. Normal runs and smoke inference already -provide a resolved target. The root configuration exposes no client-level -endpoint/model default independent of backend/profile/request settings. - -**Removal impact:** Internal fixtures that put routing settings on the client -instead of the request would need updating. Public endpoint precedence, default -profiles, and runtime overrides do not need to change. Keeping these fields -does not currently supply an additional reachable consumer capability. - -**Approach:** Delete those constructor fields, stored defaults, and fallback -branches; require the selected request target as production already does. -Remove the corresponding default-model argument from the wire builder. Keep -final endpoint/model checks and HTTP-client timeout/ownership behavior. -Update constructor-fallback tests to supply resolved targets, retaining useful -wire/invalid-target tests. The configured-base fallback prose in -[internal model-client documentation](../../internal/llm.md) should retire -with the behavior, rather than sustain a second settings-resolution layer. - -### 2. Build One Root Wire Payload Instead Of Mapping It Twice - -**Status:** Accepted; implementation pending. - -**Confidence:** High. **Owner:** the LLM adapter owns OpenAI-compatible encoding. - -**Evidence:** `PrepareGeneration` calls `openAIChatRequestFromGenerateRequest`, -which creates an `openAIChatRequest`, then `openAIChatRequestPayload`, which -copies its fields into a `map[string]any` and flattens extra parameters. Only -`PrepareGeneration` calls either helper in production. The intermediate root -struct has no separate lifetime or ownership boundary; the map, not that -struct, is actually marshaled. Optional-field selection is expressed in both -representations. - -**Removal impact:** The intermediate representation disappears, not provider -features. Explicit-zero presence, omissions, reserved-key rejection, session -IDs, structured output, and extension-field flattening still have to work. -Typed nested message/content-block/schema envelope values need not disappear. - -**Approach:** Construct the root payload directly from the resolved request -and marshal it once, using the existing narrow nested wire values where useful. -Compute structured-output-requested evidence from the same selected constraint. -Do not replace two mappings with a configurable serialization framework or -JSON round trip. Existing request-mapping and native-submission tests should -protect the wire contract rather than the deleted helper arrangement. - -### 3. Retire Alternate Uncaptured Lookup And Rendering Paths - -**Status:** Accepted; implementation pending. - -**Confidence:** High about reachability; exact private interface layout remains -an implementation choice. **Owner:** configuration captures definitions, -metadata, hashes, and templates; the runner consumes that captured state. - -**Evidence:** The engine always passes its configuration snapshot to the -runner and constructs `prompt.NewCompiledRenderer(snapshot)`. Nevertheless: - -- [Prompt selection](../../../internal/usecase/prompt_inspection.go) optionally - detects `GetPromptSnapshot`, falls back to `GetPromptDefinition`, and can hash - definitions at invocation time. Prompt inspection separately detects metadata - lookup, then falls back to loading/projecting a complete definition. -- [Profile identity lookup](../../../internal/usecase/profile_inspection.go) - detects an identity capability, otherwise loads a complete profile merely to - return its identity. -- [Requirement inspection](../../../internal/usecase/requirements_inspection.go) - already requires a captured snapshot capability; it has no equivalent - uncaptured implementation. -- [The renderer](../../../internal/prompt/go_renderer.go) supports both captured - templates and per-call parsing, with separate session paths and message - branches. `NewGoRenderer` has only test/benchmark callers. Production selects - the captured path. - -The captured repository implements the lookup capabilities above. Neither root -construction nor its extension options lets consumers inject alternative prompt -or profile repositories or this uncaptured renderer. - -**Removal impact:** Minimal private repositories and renderer fixtures would -need to supply captured data. The test -`TestRunnerInspectPromptUsesCapturedHashWhenMetadataCapabilityIsAbsent` -explicitly exercises an alternate private capability combination. Such tests -show that the accommodation exists, not a public extension requirement. - -**Approach:** Declare the captured operations actually consumed on the private -collaborator interfaces, retain the narrow metadata queries, and remove the -capability probes and full-definition/hash fallbacks. Make production rendering -use captured templates exclusively. Tests can compile small fixtures using the -existing template compiler or load real in-memory configuration; they should -exercise the same execution path without a replacement compatibility adapter. -Retain per-call template cloning and function binding for request isolation. - -This is not a recommendation to remove every convenient internal constructor -or useful test double. For example, a no-repair runner fixture can remain cheap -without maintaining a second rendering or repository-resolution workflow. - -### 4. Use Prepared Validation For Smoke Syntax And Schema Checks - -**Status:** Accepted; implementation pending. - -**Confidence:** High for consolidating the validation lifecycle. **Owner:** the -output-validation component owns syntax and schema checks; smoke diagnostics -orchestrate those checks and report their separate results. - -**Evidence:** [Smoke inference](../../../internal/usecase/smoke_inference_case.go) -constructs a `StandardValidator` with an empty schema directory solely for -`ValidationJSON`. It retains that object behind `validate.Validator` alongside -a separate prepared schema plan. The relevant branch in -[standard validation](../../../internal/validate/standard_validator.go) performs -`json.Valid` with context checks. It does not use the schema directory. - -Concrete host/`fs.FS` validators also expose direct `Validate(artifact, contract)` -methods and separate on-demand schema-loading callbacks. The runner now consumes -only `ValidationPreparer`. The identified production direct-validation call is -the smoke JSON-only call; direct schema workflows are exercised by internal -tests, not public injection. - -**Accepted decision:** Use the existing prepared-validation lifecycle for both -smoke checks: prepare a JSON-only plan for syntax validity and retain the schema -plan for conformance. A JSON-only plan needs no schema loading or compilation. -Run the schema check only after valid syntax, preserving cancellation handling -and the distinction between invalid JSON and schema-not-evaluated. JSON scalars -remain valid syntax even when they fail the diagnostic schema. - -After migrating that smoke caller, retire the standalone direct-validation -interface, methods, and redundant on-demand loading workflow. Move meaningful -source/schema tests to prepared-plan validation. Keep shared instance validation -and source-specific root loading; do not reproduce the removed convenience -interface in tests. - -**Rationale:** Direct `json.Valid` in smoke orchestration is a reasonable, slightly -simpler option. However, the output-validation component remains necessary for -execution and schema validation regardless of this choice. Using its existing -prepared plans keeps output-validation responsibility, cancellation handling, -and result semantics together. The modest difference in code complexity does -not outweigh that architectural benefit. This decision reuses the existing -abstraction; it does not call for a composite validator or a new diagnostic -framework. - -The previous roadmap explicitly retained direct validation and its -decode-before-schema-I/O behavior. This decision supersedes that selection for -the removed **private** workflow. It does not change the public `ValidateRequest` -API, which checks request variables and inputs, or make schema errors lazy again. - -**Alternatives considered:** Calling `json.Valid` directly in smoke diagnostics -would remove a plan and result projection, but would place syntax validation -and its cancellation handling outside the component that owns other output -validation. Retaining the standalone direct-validation lifecycle would avoid -migrating its tests and preserve its decode-before-schema-I/O behavior, but would -retain a second private workflow without an identified production need after -smoke migration. Neither alternative is selected. - -### 5. Separate Accepted-Value Detachment From Repeated Input Validation - -**Status:** Accepted; implementation pending. - -**Confidence:** High for repeated work; each replacement needs a clear -acceptance/ownership argument. **Owner:** acceptance checks belong where a new -value enters; copies protect the boundary at which mutable ownership changes. - -**Evidence:** `jsonvalue.CopyMap` validates supported types, keys, cycles, -depth, and produced-node limits while copying. It is used after acceptance in: - -- [Captured profile lookup](../../../internal/configuration/targets.go), then - again during profile inspection. -- [Backend cloning](../../../internal/domain/smoke_diagnostics.go), including - already-validated registry construction and subsequent lookup. -- [Executable preparation and details cloning](../../../internal/usecase/prepared_execution.go). -- [Smoke report target detachment](../../../internal/usecase/smoke_tester.go), - followed by another fallible provider-map copy in - [public report conversion](../../../smoke_diagnostics.go). - -The repository already has -[`jsonvalue.CloneTrusted`](../../../internal/jsonvalue/trusted.go), specifically -for detaching previously accepted values without revalidation. Captured data -does not become untrusted again merely because it crosses another internal -lookup or report-projection function. - -**Removal impact:** Deleting validation alone must not delete caller/client -isolation. Conversely, every clone is not necessary merely because a helper -currently promises an independently owned value. Infallible projection should -not keep impossible error branches or silently turn copying errors into empty -results. Public smoke conversion currently returns copy errors; it does not -silently swallow them. - -**Approach:** Identify the actual acceptance point for each path and keep -validation there. Use existing trusted cloning for already accepted snapshots -where detachment is still needed; transfer/share private immutable values where -ownership permits. Remove redundant error plumbing only after establishing -that invariant. Keep validation of new runtime overrides and final effective -configuration, and preserve accepted number/type/nil semantics. Do not move the -retired convenience-source compatibility converter back into the shared copier -or combine the distinct walkers into a configurable reflection framework. - -Findings 5–6 should be planned together. For each affected path, record a small -ownership inventory: where the final value is accepted, whether it can be mutated -afterward, and where it crosses into caller/client-owned mutable state. Check -prepared details, execution results, injected-client requests, repair requests, -and smoke reports together. Do not blanket-replace `CopyMap` with `CloneTrusted` -or remove a copy only to relocate it into another helper. This analysis does not -require a new ownership framework. - -Schema-document budgets are addressed by the explicit accepted decision in -finding 7, not an accidental side effect of replacing a copy operation. Runtime -extra-parameter acceptance and limits remain separate. - -### 6. Retain One Private Prepared Metadata Snapshot And One Outward Copy - -**Status:** Accepted; implementation pending. - -**Confidence:** High. **Owner:** prepared execution owns single claim/discard, -private execution settings, and independent public details. - -**Evidence:** In [prepared execution](../../../internal/usecase/prepared_execution.go), -`PrepareExecution` receives the prepared operation, recopies its provider map, -and deep-clones its metadata into a separate `details` snapshot. The original -snapshot remains in `payload.prepared`. `Details` then deep-clones `details` -again. The [public Details method](../../../prepared_execution.go) invokes -[`fromDomainPreparedRun`](../../../convert.go), which copies nested values -again while projecting public types. - -The prepared metadata is already credential-redacted. The direct key, observer, -retry settings, and validation plan are separate payload fields. -`executePreparedRun` reads prepared metadata and places the direct key on a -local target value. The existing payload `clear` does write an empty key into -the prepared metadata, so sharing that snapshot requires removing that -unnecessary mutation first; simply aliasing the current pointers is not enough. - -**Removal impact:** Consumers still need fresh, mutable, redacted details, and -details must remain available after execution or discard. Neither promise -requires two retained private metadata graphs or two deep copies per query. -Single-use state, owner identity, and synchronization remain necessary. - -**Approach:** Retain one immutable, redacted private metadata snapshot shared -by details and the execution payload. Keep execution-only settings separately -clearable. Remove the payload-cleanup mutation of metadata before sharing the -snapshot. Let the facade perform the one necessary public projection/copy -from that snapshot. Avoid an internal accessor that copies solely for another -copying accessor, including its currently hidden copy-failure-to-nil branch. -Preserve frozen schema/message/input state and the existing injected-client -copy boundary. No new snapshot manager, public type aliases, or replacement -lifecycle framework is needed. - -## Findings: Complexity Caused By Optional Contracts - -### 7. Decide Whether Prepared Schemas Own A JSON-Tree Resource Budget - -**Status:** Accepted: remove the incidental copier-derived schema budget; -implementation pending. - -**Evidence:** `structuredOutputFromValidationPlan` in -[the runner](../../../internal/usecase/runner.go) feeds a schema already decoded -and compiled by the validator through `jsonvalue.Copy`. Prepared details cloning -can repeat it. That helper applies the same depth-100/node-100,000 traversal -limits used for runtime JSON-compatible values. The test -`TestRunnerPreparationRejectsExcessivelyDeepPreparedSchema` supplies an oversized -schema through a fake private validation plan and protects that rejection. - -**Requirement/authority:** Detaching schema state for clients/public results is -established. The public output/schema contracts do not themselves specify those -numeric schema budgets. Internal comments/tests record the behavior, but this -review did not establish a separate user decision adopting a schema budget. -The deliberately selected runtime-extra-parameter limits are not automatically -authority for applying the same limits to loaded schema documents. - -**Decision:** Treat copying as copying and let the existing schema -decoder/compiler own schema acceptance, retaining detachment and PromptKit's selected -self-contained-schema rule. Remove the additional depth/node rejection inherited -from `jsonvalue.Copy` rather than adopt it as a schema-resource policy. -This could admit schemas previously rejected solely by the copier; it is not -a promise of unlimited dependency capacity, stack safety, or provider support. - -**Alternative considered, not selected:** Deliberately adopt a schema budget -and enforce it once at the schema acceptance boundary, with appropriate public documentation and focused -behavioral coverage. This keeps an explicit resource-control policy but adds a -consumer restriction and locally owned traversal responsibility. Do not choose -it solely to make an existing copier convenient or to keep a fake-plan test. - -**Follow-through:** Update tests and internal prose that pin the copier-derived -schema rejection. Preserve schema compilation, completeness checks, and outward -mutation isolation. Record the broadened acceptance of schemas previously -rejected only by that copier, without adding a new precision, unlimited-resource, -or provider-compatibility guarantee. Runtime provider-value budgets are unchanged. - -### 8. Broad Runtime Go-Value Support Is A Real Choice, Not ConfigKit Duplication - -**Status:** Accepted: preserve current runtime behavior in this work. - -**Evidence:** [`ExecutionTargetOverride.ExtraParams`](../../../types.go) accepts -JSON-compatible arrays/slices/maps and numbers. The -[JSON-value implementation](../../../internal/jsonvalue/jsonvalue.go) also -handles named/typed containers and scalar types, `json.Number`, and indirection, -preserving compatible concrete representations during detachment. Tests cover -these distinctions. This is more machinery than ConfigKit-native mapping -containers require, but runtime overrides do not pass through ConfigKit. - -**Requirement/authority:** Accepted runtime input forms and mutable-value -isolation are consumer-facing concerns. Exact preservation of every concrete -Go representation is broader than the public JSON-compatible wording; that -distinction should not be promoted into an additional promise solely because -the implementation/tests currently preserve it. The previous remediation -expressly left runtime override behavior unchanged. - -**Decision:** Preserve the current runtime behavior while addressing -revalidation and duplicate snapshots in findings 5–6. There is no demonstrated -need in this review to narrow the supported input forms. Do not delegate runtime -values through ConfigKit merely to reuse its source-input restrictions. - -**Alternative considered, not selected for this work:** If the remaining -reflection/type-preservation cost becomes a priority, deliberately standardize runtime inputs on a smaller JSON-native value -contract or normalize accepted typed containers to canonical containers. This -could reduce copier complexity, but callers using typed values may need explicit -conversion, and injected clients may observe changed dynamic types. Number -handling must remain explicit; marshal/unmarshal through `float64` is not a -semantics-preserving shortcut. This is a product/API decision, not an internal -cleanup implicitly authorized by the audit. - -## Machinery Worth Retaining - -These areas were considered and do not justify blanket deletion: - -| Area | Concrete responsibility and removal consequence | -| --- | --- | -| Public/internal value and error projection | Keeps implementation types private, redacts credentials, and gives callers/injected clients independent mutable values. Removing all conversion or aliasing internal types would change the intentionally selected facade. Findings 2 and 6 remove redundant steps, not this boundary. | -| Final ConfigKit handoff interpretation | ConfigKit owns composition; PromptKit owns LLM-specific legality, effective presence/null interpretation, referenced resources, and execution readiness. Runtime overrides are new inputs. No second raw-source validator or execution-time provenance audit is justified. | -| Read-only configuration inspection wrapper | Preserves the captured inspection view without exposing the mutable configuration result. Comparison delegates to ConfigKit; a separate PromptKit diff engine is unnecessary. | -| Directory/source discovery | Discovers YAML inputs and groups their resource roots. ConfigKit's explicit filesystem/file source API does not itself enumerate directories; discovery is distinct from reimplementing source composition. | -| Schema root-source adapters and completeness checks | Host-path containment and generic `fs.FS` loading have different source semantics. The selected self-contained submission rule also includes unused schema declarations and relocation away from the source name; root compilation alone is insufficient. Retain the focused traversal/compiler delegation rather than adding a resolver or bundler. | -| Artifact file checks and defensive copy | File-kind checks prevent reading unsupported resources; opened-file checks and context-checked reads serve actual artifact behavior. The earlier accepted ownership decision expressly retained the adapter copy. Removing it needs its own ownership analysis, not an assumption that all copying is redundant. | -| Admission versus active permits | Admission bounds logical execution occupancy across preparation/repair/backoff; active permits bound dispatched attempts. Replacing both with one semaphore would change the selected scheduling behavior. Cancellation-safe waiters and cleanup remain necessary. | -| Prepared-handle identity and state | Owner tokens, one-claim/discard synchronization, payload clearing, and details-after-use implement the opaque handle contract. Finding 6 does not remove that lifecycle. | -| Captured template clones and requirement traversal | Per-call template clones bind request input functions without mutating captured templates. Requirement analysis supplies selected offline required/conditional/optional facts that executing templates or listing AST fields alone cannot provide. Keep its documented conservative scope. | -| HTTP evidence and timeout/body wrappers | Invocation-local transport wrapping preserves caller client settings while observing redirects/status; timeout cancellation must remain active through body consumption. Removing these wrappers can lose facts or cancel responses at headers. | -| Operation-owner error metadata | Whole-chain sentinel priority cannot distinguish a parent operation from a retained child engine error. The private owner wrapper preserves that distinction through errors and observations without discarding child causes. | -| Release wiring | The local launcher and profile delegate validation/publication to ReleaseKit. The Python test checks repository wiring against disposable repositories rather than supplying a second release implementation. No substantial remaining release orchestration duplication was identified. | - -## Evidence And Recommended Disposition - -Focused existing tests passed with Go 1.27.3, `GOTOOLCHAIN=local`, `GOWORK=off`, -and `GOPROXY=off`: - -```sh -go test -race ./internal/usecase ./internal/llm ./internal/jsonvalue \ - -run 'Test(RunnerInspectPromptUsesCapturedHashWhenMetadataCapabilityIsAbsent|RunnerPreparedExecutionWithoutValidatorSkipsValidation|RunnerPreparationRejectsExcessivelyDeepPreparedSchema|OpenAICompatibleClientRequestMapping|CopyPreservesCompatibleCollectionsAndNilEmptyDistinctions)' - -go test -race . ./internal/prompt ./internal/configuration ./internal/validate \ - -run 'Test(PreparedExecution|SmokeTestConcludedInferenceMismatch|SmokeReportConversion|SchemaFile|GoRenderer|LoadBuildsEffectiveTargetsAfterInheritance|PromptInspectionLookupMatchesSnapshotSelectionAndOwnsInputs|StandardValidator|FSValidator|SchemaPreparationDoesNotDependOnSourceName|SchemaLocations)' -``` - -These tests substantiate current ownership, fallback, schema, and wire behavior; -they do not prove a proposed deletion correct or quantify its savings. No new -tests, benchmark implementation, live provider requests, or production changes -were made for this review. - -Begin with the clearly private transport defaults/root-payload simplification. -Then align private lookup/rendering collaborators with captured configuration. -Address copy validation and prepared snapshot ownership together so that deleting -one copy does not relocate it into another helper. Implement schema-budget -removal under finding 7's explicit decision, preserving runtime behavior under -finding 8. For finding 4, migrate smoke syntax validation to a prepared plan -before retiring the standalone direct-validation lifecycle and migrating its -meaningful tests. - -Any implementation plan should include retiring the corresponding private-path -tests and internal prose while retaining meaningful public/ownership regressions. -The accepted decisions above define the selected remediation scope; they do not -authorize additional contract changes, new limits, a general compatibility -framework, performance guarantees, or additional release gates. The retained -machinery table remains applicable, including artifact defensive copying, -prepared-handle lifecycle, and separate admission/active-permit responsibilities. - -## Decision Status - -All eight findings have an accepted disposition. No open decisions remain in -this audit; implementation planning can proceed on the selected scope. None is -marked implemented by this documentation update. diff --git a/docs/roadmap/audit/catalog.md b/docs/roadmap/audit/catalog.md deleted file mode 100644 index 3b7ad6d..0000000 --- a/docs/roadmap/audit/catalog.md +++ /dev/null @@ -1,671 +0,0 @@ -# Consolidated Pre-1.0 Audit Catalog - -Consolidated on 2026-11-04 from the 11 audit documents in this directory. -Their 41 numbered findings are represented by 34 entries below. Entries that -describe the same underlying issue share one remediation scope; missing -regression coverage for an identified defect is included with that defect. -Some broad testing findings consequently support more than one entry. - -The order reflects consequence, exposure, and the value of clarifying public -contracts before v1.0. Confirmed defects and conditional security exposure come -first, followed by contract/test gaps, resource efficiency, and maintainability. -This ordering does not upgrade the original severities or turn every cleanup -into a release gate. The source audits retain their evidence, qualifications, -and validation results. None of these entries is marked remediated by this -documentation-only consolidation. - -The [audit remediation roadmap](../audit-remediation.md) owns the consolidated -implementation-planning recommendations, evidence-dependent design choices, -sequencing, and workstream status. This catalog remains the finding/evidence -index and record of the six accepted decisions. - -## 1. Update The Audited Maintainer Toolchain For Applicable Security Fixes - -**Conditional security exposure.** The audited Go 1.27.0 toolchain predates -networking fixes applicable to HTTP clients, including HTTP/2 CPU exhaustion -and conditional certificate-trust exposure. Use a patched maintainer toolchain -and repeat the focused security assessment against it. Actual downstream -exposure depends on the toolchain that builds the consumer and its transport -and trust configuration; raising PromptKit's minimum Go requirement is a -separate compatibility concern, not an implied remedy. -[Decision 5](#decision-5-consumer-go-minimum-after-the-toolchain-update) reaffirms -independent maintainer-toolchain selection and the initial-release consumer -minimum. Advising consumers about generally applicable Go vulnerabilities is -outside PromptKit's responsibility. - -Sources: [Security 1](07-security.md#1-the-audited-go-toolchain-predates-applicable-networking-security-fixes). - -## 2. Bound Authentication-Probe Response Consumption - -**Confirmed resource/security defect.** Authentication diagnostics read a -successful upstream body with unbounded `io.ReadAll`. A deadline does not bound -the bytes a fast server can supply. -[Decision 3](#decision-3-authentication-response-budget-and-overflow-outcome) -selects a fixed 1 MiB internal constant and explicit overflow diagnostics, with -credential acceptance unknown. Implementation remains pending. Overflow should -not be interpreted as proof of rejected authentication. -Relocating the provider adapter in entry 26 would not itself fix this defect. - -Sources: [Resources 1](05-resources.md#1-authentication-diagnostics-read-an-unbounded-upstream-body), -[Security 2](07-security.md#2-authentication-probes-permit-unbounded-remote-body-memory-consumption). - -## 3. Avoid Exponential Expansion During Convenience Source Conversion - -**Confirmed resource defect.** `ProfileSource` and `BackendSource` conversion -expand repeated references before ConfigKit performs its bounded snapshot. -A small shared acyclic graph can therefore cause exponential work and -allocation. [Decision 4](#decision-4-exact-numbers-and-the-source-conversion-boundary) -selects removal of the recursive compatibility adapter, passing ConfigKit-supported -values directly to its bounded snapshot. Do not add conversion memoization, -PromptKit-side raw-source limits, or duplicate ConfigKit validation. Cover direct -handoff, source ownership, and deferred unsupported/cyclic-value diagnostics for -both helpers. Implementation remains pending. - -Sources: [Resources 2](05-resources.md#2-convenience-source-conversion-expands-shared-graphs-before-bounded-loading), -[Testing 3](08-testing.md#3-programmatic-provider-value-conversion-lacks-representative-value-cases). - -## 4. Resolve The ArtifactReader Buffer-Ownership Contract - -**Confirmed race under the documented collaborator contract.** The public -reader contract permits buffer reuse after `Read` returns, before PromptKit -can finish copying it. Concurrent preparation with a compliant custom reader -can race. [Decision 1](#decision-1-artifactreader-result-ownership-and-lifetime) -selects ownership transfer on successful return; implementation remains pending. -Align adapters, GoDoc, and concurrency tests with that rule. An immediate copy -alone cannot guarantee safety under the old immediate-reuse permission. No -default-reader defect was established. - -Sources: [Interfaces 1](09-interfaces.md#1-artifactreader-permits-buffer-reuse-before-promptkit-can-copy-it). - -## 5. Preserve Exact Numbers In Programmatic Provider Sources - -**Confirmed data-integrity defect.** Convenience source conversion falls back -from integer parsing to `float64` without checking precision, silently changing -some accepted `json.Number` values. -[Decision 4](#decision-4-exact-numbers-and-the-source-conversion-boundary) selects -removal of this conversion, not a new exact-number subsystem. Convenience-source -provider values must use ConfigKit-supported types; callers choose any necessary -conversion. Let ConfigKit report unsupported values during loading. Cover supported -value handoff and rejection of formerly converted unsupported types without adding -numeric precision guarantees. Runtime overrides remain a separate input path. - -Sources: [Correctness 1](01-correctness.md#1-provider-source-helpers-silently-change-exact-json-numbers), -[Testing 3](08-testing.md#3-programmatic-provider-value-conversion-lacks-representative-value-cases). - -## 6. Make Provider-Native Schema Constraints Self-Contained - -**Confirmed implementation gap; self-contained schemas selected.** Local -validation freezes referenced schema resources, but the provider receives only -the root document and cannot resolve its local external references. -[Decision 2](#decision-2-provider-submission-of-multi-resource-schemas) selects -self-contained schema documents and removal of external-resource loading. -Reject references requiring another document during preparation with an -actionable error; retain references within the selected document. Add coverage -for early rejection and for self-contained schemas reaching the provider intact. -Implementation remains pending. Entry 21 is a separate compilation refactor; -coordinate it with this simplification rather than preserving unnecessary -external-resource machinery. - -Sources: [Correctness 2](01-correctness.md#2-external-schema-resources-do-not-reach-the-provider-constraint), -[Testing 5](08-testing.md#5-multi-resource-schema-tests-do-not-establish-a-usable-provider-constraint). - -## 7. Preserve The Owning Operation's Public Error Category - -**Confirmed classification defect.** Public error conversion searches nested -causes in a global priority order, allowing a collaborator's sentinel to hide -the enclosing operation's category. Map the owning boundary independently -while retaining the original cause for `errors.Is`/`errors.As`. Add composed -cause regressions; simply reordering global checks can break legitimate direct -not-found versus load-failure distinctions. Carry explicit owning-operation -information through translation. Retaining nested causes may legitimately make -multiple public categories match through `errors.Is`; the remedy preserves the -owning category rather than requiring category exclusivity. - -Sources: [Error handling 1](06-error-handling.md#1-nested-collaborator-errors-can-hide-the-owning-operations-public-category), -[Testing 4](08-testing.md#4-error-tests-miss-composed-causes-and-pre-dispatch-diagnostic-failures). - -## 8. Distinguish Response Stream Failures From Malformed JSON - -**Confirmed error-handling defect.** Non-context I/O failures while decoding a -successful HTTP response or checking its end are reported as malformed JSON, -losing the transport cause; complete JSON followed by a read failure can also -be misreported as trailing data. Retain and classify reader failures separately -using the existing redacted transport boundary. Preserve cancellation -precedence and genuine malformed/truncated/oversized-content handling. Cover -both read locations without adding transport retries. - -Sources: [Error handling 2](06-error-handling.md#2-successful-status-body-read-failures-are-misreported-as-malformed-json), -[Testing 4](08-testing.md#4-error-tests-miss-composed-causes-and-pre-dispatch-diagnostic-failures). - -## 9. Return Invalid Diagnostic Request Construction As A Go Error - -**Confirmed diagnostic-contract defect.** Some invalid inference settings fail -before dispatch yet produce a completed report with a nil Go error. Propagate -local request-construction failures through `ErrInvalidRequest` and add a -no-dispatch regression. Preserve intentionally completed diagnostic outcomes -for missing credentials, capacity outcomes, and unsupported or inapplicable -operations; not every no-dispatch outcome is invalid configuration. -Use explicit preparation-stage failure information rather than inferring the -stage from a sentinel alone or from an empty attempt list. - -Sources: [Error handling 3](06-error-handling.md#3-inference-diagnostics-swallow-some-invalid-request-construction-errors), -[Testing 4](08-testing.md#4-error-tests-miss-composed-causes-and-pre-dispatch-diagnostic-failures). - -## 10. Remove Quadratic Requirement-Evidence Union Work - -**Confirmed algorithmic inefficiency.** A left-heavy recognized condition can -repeatedly copy growing missing-key evidence sets, producing quadratic work. -Use operation-local accumulation or ownership-aware merging while preserving -classification and conservative inspection coverage. Do not address this by -adding expression limits or expanding the recognized template language. Entry -23's traversal cleanup is related but is not a substitute for fixing this -specific accumulation algorithm. - -Sources: [Efficiency 1](02-efficiency.md#1-condition-evidence-unions-can-require-quadratic-work). - -## 11. Exercise RunRequest's Privacy Formatting Boundary - -**Missing protection, not a demonstrated leak.** Request formatting methods -have no execution coverage despite owning a privacy-sensitive public boundary. -Add focused tests with distinct synthetic private markers for the relevant -value/pointer formatting paths and JSON omission behavior. Assert privacy and -the documented boundary, not an incidental complete diagnostic or JSON shape. - -Sources: [Testing 1](08-testing.md#1-runrequests-privacy-formatting-contract-is-not-exercised). - -## 12. Correct Profile Inspection Credential Documentation - -**Public-contract documentation defect.** Inspection GoDoc incorrectly makes -required credentials and an environment variable mutually exclusive. Actual -behavior permits an explicit profile environment variable while suppressing -only the backend-derived default when required credentials are selected. -Correct the field/method GoDoc and consumer example, keeping the established -configuration behavior and DTO shape. - -Sources: [Interfaces 2](09-interfaces.md#2-profile-inspection-misstates-required-credential-semantics). - -## 13. Correct The Configured Provider-Map Wire Documentation - -**Integration-contract documentation defect.** The wire document says backend -and profile provider maps replace one another, but configuration recursively -overlays them; runtime replacement is a separate rule. Align the integration -summary with the canonical format reference rather than maintaining a second -conflicting precedence definition. Entry 19 concerns the efficiency of the -existing overlay, not a change to its semantics. - -Sources: [Interfaces 3](09-interfaces.md#3-the-wire-contract-incorrectly-says-configured-provider-maps-replace). - -## 14. Remove Stale Identity And Version GoDoc - -**Public-contract documentation defect.** Inspection comments still describe -trimmed profile IDs, and listing comments describe a removed prompt-version -field. Update them to the implemented literal identity and current collection -version model. Do not normalize meaningful whitespace or reintroduce a field -to match obsolete comments. - -Sources: [Interfaces 4](09-interfaces.md#4-identity-godoc-still-describes-removed-normalization-and-version-fields). - -## 15. Cover The Public Single-Schema-File Option - -**Public wiring coverage gap.** Direct filesystem-validator tests do not -exercise `WithSchemaFile` through engine construction. Add offline temporary -file cases through public preparation/validation, including option precedence -and relevant file errors. This is a supported feature, not dead code; entry -32 removes only an unused private helper result. - -Sources: [Testing 2](08-testing.md#2-the-public-single-schema-file-option-has-no-execution-coverage). - -## 16. Detach Inert Prepared Handles From The Execution Owner - -**Avoidable retention.** Used or discarded prepared handles retain their -runner, and therefore the engine's captured configuration and collaborators. -Use a narrower ownership identity or clear the unnecessary owner reference -safely. Preserve foreign-owner rejection, one-time claim, namespace-view -behavior, and available details. This is unnecessary retention when a handle -outlives its engine, not evidence of an ever-growing global leak. - -Prefer a small standalone ownership token that preserves identity after -use/discard without retaining the runner graph. Its distinct identity must not -depend on zero-sized allocations comparing unequal. If clearing the owner -pointer instead, synchronize the currently unlocked ownership check with that -mutation and preserve ownership/state behavior under concurrent claim/discard. - -Sources: [Resources 3](05-resources.md#3-inert-prepared-handles-retain-their-entire-execution-owner). - -## 17. Bound Repair Feedback's Temporary Serialization - -**Avoidable peak allocation.** Feedback formatting serializes potentially -large diagnostic text before truncating the final message, so its existing -output bound does not bound scratch memory. Produce bounded encoded feedback -incrementally, preserving the existing output semantics and complete stored -validation diagnostics and previous candidate. A new raw-diagnostic input -limit is not implied. - -Use representative allocation measurements for large and heavily escaped -diagnostics to assess the remedy, alongside the existing output-behavior tests. -Keep the encoder proportionate; the output bound does not establish an exact -peak-heap guarantee or justify a general-purpose encoding framework. - -Sources: [Resources 4](05-resources.md#4-repair-feedbacks-output-bound-does-not-bound-its-temporary-serialization). - -## 18. Detach Normalized Session IDs From Large Input Strings - -**Avoidable retention.** A short trimmed session ID can retain the allocation -of a much larger padded caller string. Clone the accepted normalized value at -its ownership boundary. Preserve trimming, accepted content, code-point -validation, and resulting wire/hash content rather than adding a raw-input -size restriction. - -Sources: [Resources 5](05-resources.md#5-short-normalized-session-ids-can-pin-large-input-strings). - -## 19. Avoid Repeated Descendant Cloning During Provider Overlay - -**Construction-time inefficiency.** Recursive overlays repeatedly clone and -sort overlapping descendant maps, giving work proportional to tree size times -depth. Existing depth bounds mean this is not unbounded quadratic growth. -Clone ownership once or use a single traversal while retaining recursive-map, -null, and array semantics. Coordinate with entry 24 without assuming all JSON -walkers have interchangeable policies. - -Compare representative nested overlays before and after the change for -allocation/work reduction. Balance the measured benefit against implementation -complexity, preserving input isolation instead of removing necessary copies. - -Sources: [Efficiency 2](02-efficiency.md#2-recursive-provider-overlay-repeatedly-clones-descendant-subtrees). - -## 20. Index Filtered Listings By Namespace - -**Repeated inspection overhead.** Filtered listing currently copies and scans -the complete inventory even when few or no entries match. Capture namespace -buckets or equivalent ranges so queries operate on the relevant subset. -Preserve ordering, exact identity matching, cancellation, and caller-owned -results. A pagination API or runtime cache is not needed for this improvement. - -Evaluate representative namespace counts and match sizes, including absent -namespaces, and account for additional construction cost and retained indexing -memory. Choose buckets or ranges based on that tradeoff. These measurements and -those for entries 17 and 19 inform the changes; they add no performance contract -or recurring release gate. - -Sources: [Efficiency 3](02-efficiency.md#3-filtered-listings-copy-and-scan-the-complete-inventory). - -## 21. Share Schema Compilation And Declare The Runner's Real Dependency - -**Maintainability and interface clarity.** Four host-file/`fs.FS`, -direct/prepared paths repeat the schema compilation lifecycle. The runner also -declares a validator interface while requiring preparation through a runtime -assertion. Share the lifecycle behind source-specific adapters and make that -dependency explicit. Preserve malformed-output short-circuiting before schema -I/O, source containment, exact numbers, cancellation, and frozen plans. Retain -independent cross-source tests; do not force whole entry points to be identical. - -Sources: [Complexity 1](03-complexity.md#1-consolidate-schema-compilation-and-express-the-runners-actual-interface), -[Equivalence 1](10-equivalence.md#1-share-the-schema-compilation-lifecycle-across-source-and-execution-paths). - -## 22. Decompose Configuration Validation And Materialization - -**Maintainability.** Large configuration operations mix record-specific -validation, resource capture, and effective-profile construction. Extract -focused helpers with visible inputs and ownership, rather than another generic -validation framework. Preserve final-handoff validation, presence/null rules, -origins, and meaningful fail-fast/error-precedence behavior. Do not turn -incidental ordering of independent validation failures into a new guarantee. -The configuration-boundary ADR remains authoritative; this is not a proposal -to inspect raw ConfigKit sources. - -Sources: [Complexity 2](03-complexity.md#2-decompose-configuration-validation-and-resource-materialization). - -## 23. Clarify Requirement Traversal And Share Branch Orchestration - -**Maintainability.** Separate recognition/evaluation from reachability and -evidence bookkeeping, particularly the repeated `if`/`with` branch protocol. -Keep their scope differences explicit and retain independent comparisons with -the actual Go renderer. Preserve conservative partial coverage and inactive -computed-name behavior. This does not justify a general template interpreter -or stronger execution-readiness promises from requirements inspection. - -Sources: [Complexity 3](03-complexity.md#3-untangle-requirement-traversal-expression-evaluation-and-evidence), -[Equivalence 3](10-equivalence.md#3-share-ifwith-requirement-branch-orchestration-not-template-execution). - -## 24. Centralize Compatible JSON Copying Without Conflating Policies - -**Maintainability and ownership clarity.** The facade maintains a separate -reflection traversal for already-normalized values despite an existing -`internal/jsonvalue` owner. Consolidate genuinely compatible structural -copying while retaining DTO projection at the facade and distinct source -encoding/runtime validation policies. Preserve concrete types, nils, exact -numbers, schema keys, and ownership. Infallible projection must not acquire -new errors, panics, or silent zero-value fallbacks. Entries 3 and 5 remain -separate source-conversion defects resolved by removing the adapter under -Decision 4; do not fold its compatibility conversions into the shared copier. - -Retain separate small traversals where ownership or validation policies differ -enough that sharing would require a configurable reflection framework. Reducing -duplicate code is useful only when it also simplifies the supported behavior. - -Sources: [Complexity 4](03-complexity.md#4-clarify-ownership-and-policies-across-the-json-value-walkers), -[Equivalence 2](10-equivalence.md#2-centralize-structural-copying-of-already-normalized-json-values). - -## 25. Separate Attempt Accounting From Generation Control Flow - -**Maintainability.** Generation coordination interleaves dispatch/retry policy -with parallel return values and repeated evidence/observation projection. -Introduce a small attempt result and focused accounting helpers. Preserve -built-in versus injected-client differences, permit release before callbacks -or backoff, frozen requests, existing retry classification, private evidence, -and known/unknown usage distinctions. Do not collapse legitimately different -dispatch strategies merely to reduce lines. - -Sources: [Complexity 5](03-complexity.md#5-extract-attempt-accounting-from-generation-control-flow). - -## 26. Move OpenRouter Authentication Protocol Details Into Its Adapter - -**Architectural ownership.** Use-case code currently interprets OpenRouter's -account-response fields and provider-relative endpoint while the outbound -adapter owns HTTP execution. Place protocol construction/recognition with the -outbound integration; keep diagnostic selection, applicability, cancellation, -completion, and neutral reporting in the use case. Preserve credential/body -privacy and anonymous probing. No plugin framework, reverse dependency, or -blanket prohibition on HTTP facts is needed. Coordinate the body-bound fix -in entry 2, which is a distinct behavioral change. - -Sources: [Boundaries 2](11-boundaries.md#2-put-openrouter-authentication-protocol-details-behind-the-outbound-adapter). - -## 27. Narrow Smoke-Value Copy Helpers To Real Consumers - -**Ownership cleanup.** A request cloner has only its self-test as a caller; -the full-report cloner is used to detach only a target. Express that actual -copy operation directly and retire unused broader cloning after checking -callers. Keep facade request/report projection, active backend cloning, -provider-map errors, ownership tests, and redaction. Do not introduce shared -public/internal DTOs or an additional full-report copy. - -Sources: [Boundaries 3](11-boundaries.md#3-narrow-smoke-value-copy-helpers-to-their-actual-ownership-boundaries). - -## 28. Give Bundled Catalog Expectations One Test Owner - -**Test duplication.** Complete bundled profile/model/provider expectations are -repeated across several test layers. Keep one independently specified asset -oracle, with smaller tests for distinct loading, wire, and diagnostic behavior. -Use controlled fixtures where bundled churn is irrelevant and structural -assertions rather than formatted-map strings. Retain meaningful bundled -coverage instead of deriving every expected value from the loader under test. - -Sources: [Testing 7](08-testing.md#7-bundled-catalog-expectations-have-multiple-test-owners). - -## 29. Stop Pinning The Opaque Artifact Hash Algorithm In Tests - -**Over-specified tests.** Artifact tests and a benchmark assert a literal hash -algorithm despite the public value being opaque. Assert the documented -equality/ownership relationships and actual byte/size behavior instead; keep -benchmark setup outside its timed work. Coordinate with entry 31, but do not -turn today's hash representation into a new compatibility guarantee. - -Sources: [Testing 6](08-testing.md#6-artifact-tests-and-benchmarks-pin-an-explicitly-opaque-hash-algorithm). - -## 30. Remove Unconsumed Backend Presence Bookkeeping - -**Low-priority state cleanup.** Backend field-state maps and -`QueueCapacitySet` have no runtime reader; effective values and ConfigKit -already own queue defaulting and provenance. Remove only these unused records -and adjust tests to assert effective behavior or operator inspection. Keep -profile/input presence tracking, explicit zero capacity, and derived defaults. - -Sources: [Dead code 1](12-deadcode.md#1-remove-backend-presence-bookkeeping-with-no-runtime-consumer). - -## 31. Retire Duplicate Fields In Captured Prompt Definitions - -**Low-priority representation cleanup.** Captured definitions duplicate the -output format and carry an unused runtime message-file field after content -materialization. Use the captured output contract and text as their respective -sources of truth. Preserve real YAML `content_file` loading, origins, and the -request-override default rule. Removing serialized internal fields changes -current definition hashes: retain documented opaque equality relationships, -not an invented bit-identical encoding promise. Record the observable hash -change in release notes so consumers can account for invalidated comparisons -or caches, without establishing a permanent hash algorithm or encoding contract. - -Sources: [Dead code 2](12-deadcode.md#2-retire-legacy-fields-in-the-captured-prompt-representation). - -## 32. Remove Unused Private Diagnostic State And Helper Results - -**Low-priority cleanup.** A private reachability outcome echoes an endpoint -that report assembly never reads; the schema-file helper returns a directory -that its only caller discards. Remove the unused state/result while preserving -actual sanitized request/report evidence, directory calculation, file checks, -and public single-file schema support. Keep privacy tests against the values -that real execution consumes. - -Sources: [Dead code 3](12-deadcode.md#3-remove-unused-diagnostic-state-and-a-discarded-helper-result). - -## 33. Remove Orphan Internal Helpers And Their Self-Tests - -**Low-priority cleanup.** `filecatalog.RelativePath` and -`resourceNamespaceFromID` have only their own tests as callers, and the runner -test's `stringPtr` helper is entirely unused. Remove those narrow surfaces -without replacement, preserving real discovery, identity qualification, and -namespace inventory/filtering coverage. Reachability from examples alone is -not a reason to remove optional public APIs. - -Sources: [Dead code 4](12-deadcode.md#4-remove-orphan-helpers-whose-only-callers-are-their-own-tests). - -## 34. Decide Whether To Retain The Legacy Capacity Client Wrapper - -**Accepted architectural cleanup; implementation pending.** The old capacity -client wrapper is used only by tests; production dispatch uses the generation -coordinator. [Decision 6](#decision-6-retention-of-the-legacy-capacity-client-wrapper) -selects removal of the production wrapper and its retention documentation. -Migrate the runner scheduling test to the production coordinator and preserve -useful FIFO, cancellation, independent-pool, and panic-release coverage with -the owning components. Keep any useful one-shot scaffolding test-only. This -is not a second scheduling algorithm or a confirmed concurrency defect. - -Sources: [Boundaries 1](11-boundaries.md#1-retire-the-test-only-capacity-client-as-a-production-scheduling-surface), -[Dead-code retention qualification](12-deadcode.md#intentional-retention-and-related-findings). - -## Resolved Decisions - -### Decision 1: ArtifactReader Result Ownership And Lifetime - -**Related finding:** [4](#4-resolve-the-artifactreader-buffer-ownership-contract). - -**Status:** Accepted; implementation pending. - -**Decision:** A successful `Read` transfers the returned artifact and its mutable -body storage to PromptKit. The reader must stop mutating the returned artifact -and body, including through retained aliases. Fresh independent results are the -simplest implementation. A reader may reuse private working buffers if it copies -into independent result storage before returning. - -Keep the existing interface signature and defensive adapter copy initially. -Removing redundant copies is a separate optimization, not part of this contract -correction. No borrowing lifetime or acknowledgement/release API is introduced. - -**Rationale:** The reader controls storage until return and can therefore make -that handoff safe. Copying only after return cannot make the current permission -to immediately reuse storage safe; a reader-local mutex also cannot protect the -caller's later copy. - -**Alternatives considered:** Borrowing until the enclosing operation returns -makes the lifetime difficult for a reader to observe and requires coordination -across concurrent calls. An explicit release protocol enables pooling but adds -lifecycle machinery. Caller-provided buffers or streaming would require a -broader interface redesign. None is needed for this correction. - -**Remediation follow-through:** Update GoDoc, adapter assumptions, and concurrency -regressions together. Explain the migration for readers that reuse returned -storage in release notes as a deliberate pre-1.0 contract correction. This -record does not itself implement the change. - -### Decision 2: Provider Submission Of Multi-Resource Schemas - -**Related findings:** [6](#6-make-provider-native-schema-constraints-self-contained) -and [21](#21-share-schema-compilation-and-declare-the-runners-real-dependency). - -**Status:** Accepted; implementation pending. - -**Decision:** Require self-contained schema documents. Continue loading the -selected schema from a file and supporting references within that document, -such as `#/$defs/address`. Reject references requiring another document during -preparation with an actionable error. Remove external-resource loading support -and update the owning documentation and tests as a deliberate compatibility -change. Do not introduce a local-only validation mode or implement bundling in -this remediation. - -**Rationale:** OpenRouter's documented structured-output request accepts one -schema object, with no accompanying collection of external schema files. -PromptKit currently supports local validation of resources it cannot faithfully -submit. Requiring a self-contained document resolves that mismatch without -adding a schema transformation subsystem. Provider-specific schema restrictions -still apply; self-containment does not guarantee universal provider acceptance. - -**Alternatives considered:** Bundling could preserve multi-file authoring and -reuse, but its value does not currently justify the machinery needed to preserve -reference semantics. Retain a [schema bundler](../future.md#schema-bundling) as a -possible future enhancement, not approved implementation work. An explicit -local-only mode would preserve external local validation but add a distinct -execution behavior that is not needed for the selected scope. - -**Remediation follow-through:** Cover external-reference rejection and retained -in-document references, including recursion where otherwise supported. Update -schema documentation and release notes, and simplify the shared compilation -work in finding 21 accordingly. This record does not itself implement the change. - -### Decision 3: Authentication Response Budget And Overflow Outcome - -**Related findings:** [2](#2-bound-authentication-probe-response-consumption) -and [26](#26-move-openrouter-authentication-protocol-details-into-its-adapter). - -**Status:** Accepted; implementation pending. - -**Decision:** Bound the authentication protocol's response body to 1 MiB -(1,048,576 bytes), defined as a named internal constant. This is a generous -practical processing allowance, not an upstream protocol limit. Do not add a -public configuration option now; keep the value centralized so a future option -can be introduced if needed. This limit does not apply to inference responses -or establish an application deadline. - -Enforce the budget against bytes actually read, regardless of declared content -length. Detect overflow, stop reading, and close without unbounded draining. -Report an explicit response-limit diagnostic with credential acceptance unknown, -retaining independently collected HTTP status and timing information. Overflow -is neither evidence of rejected credentials nor proof of malformed JSON. The -consuming application decides what to do with the diagnostic. - -**Alternative considered:** A caller-configurable budget with a finite default -would accommodate unusual deployments, but adds public configuration without a -current demonstrated need. Reconsider if such a need arises. - -**Remediation follow-through:** Add focused boundary and overflow coverage, -including missing or inaccurate content lengths. Represent overflow explicitly -in the diagnostic report and update its owning documentation. Keep this fix -independent of adapter relocation so architectural cleanup does not delay the -resource remedy. This record does not itself implement the change. - -### Decision 4: Exact Numbers And The Source-Conversion Boundary - -**Related findings:** [3](#3-avoid-exponential-expansion-during-convenience-source-conversion), -[5](#5-preserve-exact-numbers-in-programmatic-provider-sources), and -[24](#24-centralize-compatible-json-copying-without-conflating-policies). - -**Status:** Accepted; implementation pending. - -**Decision:** Remove the recursive provider-value compatibility adapter from -`ProfileSource` and `BackendSource`. Supply their provider values directly to -ConfigKit `Mapping`, using its supported input types: nil, built-in strings, -booleans, signed and unsigned integers, finite floats, `map[string]any`, and -`[]any`, recursively. Retain the helpers' ordinary profile/backend field mapping. -ConfigKit owns snapshotting and deferred diagnostics for unsupported values and -cycles. Do not introduce an upstream extension or a PromptKit numeric conversion, -precision-checking, or raw-source validation subsystem for this remediation. - -**Consumer impact:** Values formerly accommodated by the adapter, including -`json.Number`, named scalar types, typed maps/slices, arrays, and pointers nested -inside provider values, must be supplied in supported forms. Callers choose any -conversion and handle its errors or acceptable approximation; PromptKit does -not silently make that choice. Ordinary typed profile fields already use -supported forms. This decision does not narrow runtime request overrides. - -**Rationale:** The root cause is a mismatch between PromptKit's broader provider -value representation and ConfigKit's mapping inputs. PromptKit's compatibility -adapter introduces both lossy numeric interpretation and pre-handoff graph -expansion. Removing it simplifies the boundary and addresses both defects -without imposing new numeric guarantees on an LLM library. - -**Alternatives considered:** Repairing and memoizing the adapter would preserve -more input forms but retain unnecessary conversion machinery. Extending ConfigKit -could move that responsibility upstream but is not needed for the selected scope. - -**Remediation follow-through:** Update convenience-source GoDoc, migration/release -notes, and focused tests for direct supported-value handoff, source capture, and -ConfigKit's deferred rejection of unsupported inputs. Remove obsolete adapter -code and tests. Preserve distinct runtime JSON-copying behavior. This is a -deliberate compatibility change; this record does not itself implement it. - -### Decision 5: Consumer Go Minimum After The Toolchain Update - -**Related finding:** [1](#1-update-the-audited-maintainer-toolchain-for-applicable-security-fixes). - -**Status:** Accepted; maintainer-toolchain remediation pending. - -**Decision:** Select the maintainer toolchain independently of the consumer Go -minimum. Always use the latest locally available Go version for PromptKit builds -and tests, including ordinary development and release validation. Update the -local installation as needed to address the audit findings; local availability -alone does not establish that those fixes are present. - -Normally set the consumer minimum to the initial stable release (`1.x.0`) of -the current selected series. Advance series in minor releases; preserve the -minimum in patch releases. Retain the existing exception for a dependency or -identified correctness requirement that needs a higher patch minimum. This -reaffirms the [release policy](../../policy/release.md#consumer-minimum-and-release-toolchain) -and [release procedure](../../release.md#prepare-the-candidate), extending the -latest-local-toolchain instruction explicitly to ordinary builds and tests. - -Advising consumers to update Go for generally applicable vulnerabilities is not -PromptKit's responsibility. Do not add consumer advisories, upgrade notices, or -an ongoing vulnerability-notification obligation for those issues to this -remediation. Downstream toolchain maintenance belongs to the consuming project. - -**Alternative considered:** Raising the consumer patch minimum as a routine -response to general Go security advisories would conflate library compatibility -with downstream toolchain maintenance. That approach is not selected. - -**Remediation follow-through:** Use the selected local toolchain for the focused -reassessment and clarify ordinary development toolchain selection in contributor -guidance. No new recurring release gate or general consumer security advisory -work is introduced. This record does not itself update the installed toolchain. - -### Decision 6: Retention Of The Legacy Capacity Client Wrapper - -**Related finding:** [34](#34-decide-whether-to-retain-the-legacy-capacity-client-wrapper). - -**Status:** Accepted; implementation pending. - -**Decision:** Retire the legacy production capacity client wrapper. Move the -runner's initial-generation and repair scheduling test to the production -generation coordinator. Preserve useful FIFO, cancellation, independent-backend, -and panic-release coverage in tests of the components that own those behaviors. -Keep a small one-shot wrapper as test-only scaffolding if it improves focused -capacity tests; do not retain a production constructor solely for its self-tests. - -**Rationale:** All current wrapper callers are tests. Production already uses -the generation coordinator with the same capacity manager. Removal reduces -maintenance surface and makes the runner test reflect actual engine assembly, -without changing the public API or production scheduling behavior. - -**Alternative considered:** Retaining the small wrapper as an internal -convenience would avoid test migration, but no current production need justifies -that additional assembly surface. - -**Remediation follow-through:** Migrate the useful assertions before removing -the wrapper and update its retention documentation. Keep this cleanup narrowly -scoped; do not redesign the capacity manager or generation coordinator or add -another permit-acquisition layer around the coordinator. This is not a confirmed -concurrency defect or a release blocker. This record does not implement removal. - -## Open Decisions - -All six decisions above are resolved. Their implementation remains pending; -recording acceptance does not mark the audit findings remediated. No further -open decisions are currently identified in this catalog. diff --git a/docs/roadmap/future.md b/docs/roadmap/future.md index 138cd28..938c954 100644 --- a/docs/roadmap/future.md +++ b/docs/roadmap/future.md @@ -42,13 +42,13 @@ consumers. Allow consumers to author reusable schemas across multiple files and package them into a single self-contained schema for local validation and provider -submission. This could restore multi-file authoring convenience after the -[self-contained schema decision](audit/catalog.md#decision-2-provider-submission-of-multi-resource-schemas). +submission. This could restore multi-file authoring convenience while retaining +the [self-contained schema requirement](../formats.md#schemas). Bundling would need to preserve reference semantics, identifiers, anchors, and recursion. A complete bundle would not guarantee that a provider supports every schema feature. This is a possible future enhancement; implementation is not -part of the audit remediation and is not currently approved. +currently approved. ## Entry Format diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md deleted file mode 100644 index 50819c3..0000000 --- a/docs/roadmap/implementation.md +++ /dev/null @@ -1,504 +0,0 @@ -# Responsibility Simplification Implementation Plan - -## Scope And Execution - -Implement [responsibility-simplification.md](responsibility-simplification.md) -from baseline `c69703b`, using the accepted decisions and evidence in -[A1-responsibility.md](audit/A1-responsibility.md). Production code is unchanged -from the audit baseline `1ff6d77`. Findings 1–7 require changes; finding 8 requires -preserving current runtime behavior. Completion markers below record delivery. - -This replaces the completed 33-stage audit-remediation plan. Historical links -from the older audit/remediation documents refer to that earlier plan, available -with `git show c69703b:docs/roadmap/implementation.md`, not to the stages below. -Do not repeat the earlier sprint or treat its completion evidence as validation -of these changes. - -Execute stages in order as one end-to-end sprint. Each stage is a focused Luna -prompt. Intermediate stages may leave callers/tests unmigrated or fail to -compile; record deferred checks and run them once their affected paths are -coherent. Do not add compatibility wrappers or duplicate implementations just -to keep an intermediate commit green. No human-review or external-publication -gate is needed between stages. Completion concerns the combined end state. - -Read [development guidance](../development.md) and its applicable policies, -especially agents, architecture, testing, and documentation, plus -[ADR 0001](../adr/0001-configuration-boundary.md). Use graph tools for discovery -and verify actual callers before deletion. Preserve unrelated work. Follow the -local commit policy and record completion/evidence for finished stages without -claiming temporarily deferred checks passed. - -### Shared Rules - -- Keep ConfigKit composition/history outside runtime validation. Validate final - configuration and genuinely new runtime inputs at their existing boundaries. -- Preserve the public facade and substantive API/wire behavior, except removal - of incidental schema-copy budgets selected in finding 7. The removed direct - output-validator workflow is private, not public `ValidateRequest`. -- Retain runtime supported values, concrete type/nil behavior, limits, number - handling, and errors. Do not narrow runtime values to ConfigKit mapping forms. -- Keep needed ownership copies, artifact defensive copying, prepared-handle - identity/lifecycle, admission versus active permits, HTTP evidence/deadlines, - and operation-owner errors. Do not expand this into another subsystem audit. -- Use existing trusted cloning for accepted values; keep validating copying for - actual acceptance. Do not add a reflection framework, schema budget, numeric - converter, raw-source validator, or public internal-type alias. -- Private names are implementation seams, not new public promises. Test selected - behavior, not allocation counts, helper choreography, or incidental wording. -- Include relevant GoDoc/internal documentation with the owning change. Stage 16 - reconciles omissions, rather than authorizing stale documentation until then. -- Select the newest locally available Go installation as documented, with - `GOTOOLCHAIN=local` and `GOWORK=off` for isolated checks. The audit used Go - 1.27.3. This sprint does not select a release, change dependency versions or - the consumer Go minimum, or authorize pushing/tagging/publication. - -## Ownership Inventory For Stages 10–15 - -Use these decisions, checking the affected callers as they are migrated. Record -any discovered mutation edge and account for it in the same boundary; do not -introduce a new ownership subsystem. - -| Value/path | Acceptance and mutation | Selected ownership treatment | -| --- | --- | --- | -| Configured provider values | Final configuration validation and effective-target construction own acceptance; retained configuration/index state must not be mutated by runtime work. | Preserve acceptance/overlay semantics. Registry/profile lookup may continue returning detached values, but their copies use trusted cloning, not repeated acceptance checks. | -| Runtime provider override | `toDomainExecutionTargetOverride`, also used by smoke request conversion, validates/copies caller values. | Keep that boundary and its errors. Subsequent internal selection may transfer accepted private values without another deep validation pass. | -| Resolved target within one operation | Captured lookup returns detached values; runtime override conversion supplies owned values. Resolution changes scalar fields and selects whole provider maps, not nested content. | Transfer selected private maps through preparation/inspection; avoid repeated shallow/deep copying merely between private functions. Preserve current nil/empty projection behavior. | -| Loaded schema | The validator decodes/compiles a self-contained JSON document and retains immutable plan state. | Trusted-detach the provider-facing schema without provider-value depth/node validation. Keep the frozen compiled plan and existing acceptance checks. | -| Prepared metadata | Messages/input hashes are created during preparation; selected maps and schema belong to that operation. | Retain one immutable redacted snapshot for execution and details. Clear execution-only fields without modifying metadata. | -| Public prepared/details/result/inspection values | Root conversion crosses into caller-owned mutable state. | Keep one outward deep projection at that boundary. Never expose a borrowed internal map, schema, message pointer, or slice. | -| Injected generation and repair | The internal runner/repairer reads private settings; repair constructs a new message sequence. The root client adapter supplies a mutable public request. | Keep public request detachment for every dispatch, including repairs, and existing response copying. Do not add copies to compensate for removed internal copies. | -| Smoke reports | Selection and internal orchestration use accepted private targets; public report conversion creates caller-owned facts/settings. | Keep target redaction/endpoint sanitization and final projection. Remove validation-only copies and redundant private detachment, retaining any copy needed by an actual mutator. | - -## Stage 1: Remove Client-Level Routing Fallbacks - -**Completed:** Client routing now uses the resolved target; constructor fixtures -and integration guidance were migrated. With Go 1.27.3, `go test ./internal/llm -./internal/generation ./internal/configuration ./internal/usecase ./internal/engine` -passed, including loaded-target coverage. - -**Finding:** 1. **Files:** `internal/llm/openai_compatible_client.go`, client -fixtures/tests, `internal/engine/engine.go` caller verification, `docs/internal/llm.md`. - -1. Delete `OpenAICompatibleConfig.BaseURL`/`Model`, stored `baseURL`/`defaultModel`, - constructor endpoint normalization, and request fallback branches. Use the - selected target endpoint/model directly; retain final validity checks. -2. Remove the default-model parameter from the request builder and its caller. - Preserve HTTP-client cloning/timeouts and request authentication behavior. -3. Make the internal constructor infallible once its only failure path is gone; - update engine assembly and internal callers rather than retaining an always-nil - error return. Update constructor fixtures to put routing settings on request - targets. Delete tests solely asserting constructor-default precedence; - retain endpoint composition/rejection and supplied HTTP-client ownership cases. -4. Retire internal configured-base fallback prose. Do not add a new default - source or change backend/profile/request precedence. - -**Check:** Relevant llm, generation, and loaded-target tests; defer compilation -where fixture migration remains. No production construction supplies the removed fields. - -## Stage 2: Encode One Root Wire Representation - -**Completed:** Requests now build the provider payload directly, with structured -dispatch evidence derived from that payload. Go 1.27.3 `go test ./internal/llm -./internal/generation` passed, including mapping, presence, payload freezing, -session, structured-output, and encoding-failure tests. - -**Finding:** 2. **Files:** `internal/llm/openai_compatible_client.go` and request-mapping tests. - -1. Replace the root `domain request -> openAIChatRequest -> map` sequence with - one direct payload builder. Remove the intermediate root struct and obsolete - projection helper; retain useful nested typed wire structures. -2. Preserve explicit numeric zero versus omission, session normalization, - native schema envelopes, reserved/empty extension-key errors, and flattened - provider extensions. Derive structured-output evidence from the same selected - constraint, not a second independent mapping. -3. Keep successful payload encoding once and existing encoding-failure error - classification/privacy. Do not introduce JSON round trips or a custom serializer. - -**Check:** Run `go test ./internal/llm ./internal/generation`; reuse request-mapping, -presence, prepared-payload freezing, session, and structured-dispatch assertions. - -## Stage 3: Require Captured Repository Operations - -**Completed:** Repository interfaces require captured definitions, metadata, -requirements, and profile identity; the alternate lookups and hash wrapper -were removed. Go 1.27.3 `go build ./...` passed with the real configuration -snapshot. Use-case test fixture compilation is deferred to the next stage. - -**Finding:** 3. **Files:** `internal/promptdef/repository.go`, `internal/profile/repository.go`, -`internal/usecase/{runner,prompt_inspection,profile_inspection,requirements_inspection}.go`, -`internal/configuration/prompts.go` and `targets.go`. - -1. Make the runner's prompt repository explicitly provide `GetPromptSnapshot`, - `GetPromptInspection`, and `GetRequirementSnapshot` with their existing - signatures. Require `GetProfileIdentity` alongside `GetProfile` on its profile - repository. Reuse the existing domain/template values; no public interfaces change. -2. Call these captured operations directly. Remove capability assertions and - alternate full-definition/profile lookup/hash paths. Do not add new rejection - rules for hashes or introduce runtime hashing in another helper. -3. Preserve direct not-found versus load-error ownership, context checks, exact - IDs, and metadata-only inspection. Keep captured definition detachment. -4. Remove unused legacy repository methods and the use-case hash wrapper once - callers are migrated; retain the hash implementation used by configuration. - -**Check:** Inspect the real configuration snapshot against the new interfaces. -Use-case tests may remain uncompilable until Stage 4; no transitional fallback is needed. - -## Stage 4: Migrate Repository Fixtures And Inspection Tests - -**Completed:** Private repositories now supply captured lookup operations and -inspection tests cover captured hashes, metadata, identities, and errors. Go -1.27.3 `go test ./internal/configuration ./internal/usecase` passed. - -**Finding:** 3. **Scope:** private repository fixtures in use-case tests and affected configuration tests. - -1. Update small repository fixtures to supply captured hashes, metadata, and - requirement snapshots through Stage 3's operations. Prefer real in-memory - configuration where useful, rather than rebuilding configuration resolution. -2. Replace the missing-metadata-capability fallback case with captured-hash and - narrow-inspection behavior. Preserve useful tests for missing resources, - lookup errors, cancellation, independent returned metadata, and namespaces. -3. Remove fixture methods/helpers that solely support deleted alternatives. - Keep convenient no-repair runner construction and controlled external doubles. - -**Check:** Focused configuration/use-case prompt, profile, and requirement -inspection tests. Renderer-related failures may be deferred through Stage 6. - -## Stage 5: Keep Only Captured Rendering - -**Completed:** Rendering now requires captured templates and no longer parses -messages or sessions per call. Go 1.27.3 `go build ./...` passed; renderer and -use-case test migrations and their checks are deferred to the next stage. - -**Finding:** 3. **Files:** `internal/prompt/go_renderer.go`, template helpers, engine assembly. - -1. Remove `NewGoRenderer` and the nil-template lookup mode. Keep - `NewCompiledRenderer` consuming the existing `TemplateLookup`. -2. Remove per-call parsing branches and the uncompiled session-rendering helper. - Keep construction-time compilation, missing captured-template errors, - per-call template cloning/function binding, and direct-session suppression. -3. Preserve input requirements, roles/cache-control behavior, message order, - artifact text resolution, cancellation, and rendered/session hash inputs. - Do not expand recognized template syntax or add a template cache. - -**Check:** Engine assembly already provides captured templates. Test callers -are deliberately migrated in Stage 6 rather than accommodated in production. - -## Stage 6: Move Renderer Tests Onto Captured Templates - -**Completed:** Renderer fixtures and benchmarks now compile captured templates -before rendering; invalid syntax, missing captures, cancellation, session, and -artifact behavior remain covered. Go 1.27.3 `go test ./internal/prompt -./internal/configuration ./internal/usecase` passed, resolving the deferred -repository and renderer checks. - -**Finding:** 3. **Scope:** prompt rendering tests/benchmarks and their use-case/configuration callers. - -1. Use `CompileTemplate` and a small `TemplateLookup` fixture, or a loaded - configuration snapshot, to provide message/session parses. Keep parsing outside - benchmark timing where measuring execution rather than capture. -2. Preserve syntax-error protection at compilation/capture, runtime missing - variable/input cases, direct/rendered session behavior, artifact-text lifecycle, - cancellation, and independent requirement-versus-rendering comparisons. -3. Do not recreate the deleted parser mode inside a renderer fixture. Share only - simple fixture setup; avoid a general test configuration framework. - -**Check:** Run `go test ./internal/prompt ./internal/configuration ./internal/usecase`. -Revisit deferred Stage 3–5 checks; carry any still-blocked checks forward to -final validation without adding transitional code or an intermediate gate. - -## Stage 7: Use Prepared Plans For Both Smoke Output Checks - -**Completed:** Smoke inference prepares separate JSON and schema plans and -checks syntax before schema matching. Go 1.27.3 `go test ./internal/usecase --run 'Smoke|Diagnostic'` passed, including malformed, scalar, matching, -nonmatching, cancellation, and dispatch cases. - -**Finding:** 4. **Files:** `internal/usecase/smoke_inference_case.go` and diagnostic tests. - -1. Replace the smoke case's `jsonValidator validate.Validator` with a - `validate.PreparedValidation` JSON-only plan, prepared through the existing - validator. Keep the distinct compiled schema plan and provider schema. -2. Invoke JSON-only validation first and schema validation only after syntax - passes. Preserve nil/false/true facts, cancellation precedence, scalar JSON - syntax, construction errors, privacy, and current diagnostic completion rules. -3. Do not move `json.Valid` into orchestration or add a composite validation API. - JSON-only plan creation performs no schema loading/compilation. - -**Check:** Focused smoke tests for malformed JSON, valid nonmatching schema, -matching schema, cancellation, and native schema dispatch. Reuse existing coverage. - -## Stage 8: Remove Standalone Direct Output Validation - -**Completed:** Output validation now executes only prepared plans, with direct -on-demand schema callbacks removed. Go 1.27.3 `go build ./...` passed. Tests -using the retired private API are deferred to the next stage. - -**Finding:** 4. **Files:** `internal/validate/validator.go`, `standard_validator.go`. - -1. Remove the standalone `Validator` interface and concrete host/FS - `Validate(ctx, artifact, contract)` methods after Stage 7's migration. -2. Delete the direct on-demand schema callbacks. Keep `ValidationPreparer`, - `PreparedValidation`, shared instance-mode validation, and prepared schema - execution. Simplify callback plumbing tied only to the removed workflow - without writing a new validation mechanism. -3. Preserve host and `fs.FS` root-source handling, eager plan compilation, - synchronous context boundaries, self-containment/portability checks, numeric - decoding, and frozen plans. Keep nil-validator/no-op behavior in the runner. - -**Check:** Production no longer calls a direct validator. Old test calls may -temporarily fail to compile; migrate them in Stage 9, not through a wrapper. - -## Stage 9: Migrate Validator Tests To Prepared Execution - -**Completed:** Validator tests prepare plans before execution, distinguish -preparation failures from invalid instances, and retain host/FS, portability, -precision, freezing, and synchronous-cancellation coverage. The retired mock -interface was removed. Go 1.27.3 `go test ./internal/validate ./internal/usecase` -and `go test . -run 'Schema|Smoke'` passed. - -**Finding:** 4. **Scope:** `internal/validate` tests, particularly standard-validator and cancellation cases. - -1. Express meaningful direct-validation cases as plan preparation followed by - plan validation. Keep malformed instance outcomes separate from preparation - failures, rather than retaining decode-before-schema-I/O as an expectation. -2. Retain independent host/FS path and containment coverage, source mutation/removal - freezing, references/unused declarations/portability, JSON precision, basic/JSON - modes, and cancellation around reading/compilation/execution where already selected. -3. Delete redundant direct/prepared duplicates and mocks of the retired interface. - In-progress synchronous dependency work is still not detached on cancellation. - -**Check:** Run `go test ./internal/validate ./internal/usecase` and relevant root -schema/smoke tests. Resolve Stage 8's deferred checks. - -## Stage 10: Remove Copier-Derived Schema Budgets - -**Completed:** Schema documents are detached using trusted cloning rather than -provider-value validation. A real loaded schema with more than 100,000 annotation -values prepares successfully and its public details are independent; deep -prepared-plan isolation is also covered. Go 1.27.3 `go test ./internal/usecase -./internal/validate ./internal/jsonvalue . -run 'Schema|Prepared|Copy|CloneTrusted'` -passed. - -**Finding:** 7. **Files:** `internal/usecase/runner.go`, `prepared_execution.go`, schema/preparation tests. - -1. Replace schema-document validation/copying in - `structuredOutputFromValidationPlan` and prepared metadata cloning with trusted - detachment of the decoded schema. Preserve nil-plan/missing-document failures - and the existing frozen schema boundary. -2. Remove `cloneJSONValue` if no longer needed; do not substitute another limited - walker, JSON-number conversion, or new schema-resource check. -3. Replace the fake-plan excessive-depth rejection expectation with selected - acceptance/isolation behavior. Include a representative real loaded schema - exceeding the old copier depth or node budget in annotation data, which the - compiler accepts; verify preparation and outward detachment. Generate large - fixtures in test setup, not as bulky checked-in files. -4. Preserve invalid/external-schema rejection and runtime provider-value budgets. - Do not turn the regression into a promise of arbitrary depth or provider support. - -**Check:** Focused prepared/schema-file tests plus `internal/validate` and -`internal/jsonvalue` tests establish that only schema-copy acceptance changed. - -## Stage 11: Detach Captured Backend And Profile Values Without Revalidation - -**Completed:** Backend registry construction and lookups, and captured profile -lookups, use trusted detached copies while configured acceptance and overlay -validation remain intact. Go 1.27.3 `go test ./internal/backend -./internal/configuration ./internal/domain` passed, including ownership, -composition, presence, and construction-error coverage. - -**Finding:** 5; preserve 8. **Files:** `internal/domain/smoke_diagnostics.go`, -`internal/backend/registry.go`, `internal/configuration/targets.go`, associated tests. - -1. Apply the ownership inventory: final validation/effective construction remains - responsible for configured acceptance; later lookup does not repeat it. -2. Make `CloneBackend` an infallible trusted deep clone, retaining smoke metadata - detachment. Remove copy-only errors from effective registry construction and - update its callers; `GetBackend` still reports genuine not-found errors. -3. Keep `GetProfile` returning a detached target/map using trusted cloning, - retaining context/not-found failures and presence semantics. Keep required - construction ownership separation from the retained ConfigKit result. -4. Leave configured overlay/acceptance semantics intact in this stage; assembling - a new effective map is not the same as copying a captured lookup result. - Do not remove final `validateExtraParams` or runtime override validation. - -**Check:** Backend/configuration tests cover mutation isolation, composed targets, -null/empty values, and construction errors without pinning deleted copy-error branches. - -## Stage 12: Transfer Accepted Provider Maps Through Internal Resolution - -**Completed:** Internal selection transfers owned provider maps without repeat -copies, preserving empty-map inheritance and nil projections; public override -validation and outward detachment remain. Go 1.27.3 `go test ./internal/usecase -./internal/configuration . -run 'Target|Profile|Override|Smoke|Provider|ExtraParams'` -passed. - -**Finding:** 5; preserve 8. **Files:** `internal/usecase/runner.go`, -`profile_inspection.go`, `smoke_selection.go`, related ownership tests. - -1. With detached lookup values from Stage 11, pass selected maps through private - target resolution rather than copying them again. Runtime replacement maps - already belong to the converted request; scalar overrides need not mutate them. -2. Remove the extra validating map copy in profile inspection. Retain credential - redaction and the root facade's independent public projection. -3. Remove redundant `copyExtraParams` uses in target/override/smoke selection and - the helper if unused. Preserve existing nil/empty outcomes, including inheritance - for an empty runtime map; do not let sharing alter public JSON shapes. -4. Keep the facade's `toDomainExecutionTargetOverride` validation/deep copy and - actual client/result detachment. Do not expose internal request sharing as a - new mutable consumer interface. - -**Check:** Focused target precedence, profile inspection, runtime override, smoke -selection, and mutation-isolation tests. Retain typed/numeric runtime cases. - -## Stage 13: Share One Immutable Prepared Metadata Snapshot - -**Completed:** Prepared execution and internal details share one immutable, -redacted snapshot; cleanup clears only execution payload references and the -facade retains the outward copy. Obsolete copy-failure tests and the remaining -retired direct-validation mock were removed. Go 1.27.3 `go test -race -./internal/usecase . -run 'Prepared|Repair|SchemaFilePreparationDetachesLargeAnnotation'` -passed, including lifecycle, details, schema, and repair cases. - -**Findings:** 5–6. **Files:** `internal/usecase/prepared_execution.go`, -`prepared_execution.go`, `convert.go`, lifecycle/ownership tests. - -1. Remove payload cleanup's write into prepared metadata. Keep clearing private - credential, observer, retry, validation, and execution payload references under - the existing lifecycle. Owner identity and claim/discard synchronization stay. -2. Retain the prepared operation's one redacted metadata snapshot for both - internal execution and details. Remove the extra provider-map clone and - separate deep details snapshot created by `PrepareExecution`. -3. Make the internal details accessor return that immutable private snapshot; - its root caller performs the sole outward `fromDomainPreparedRun` copy. - Update private comments/tests to distinguish borrowed internal metadata from - mutable public details. Do not expose an internal type in a public signature. -4. Delete `clonePreparedRun` and hidden copy-error-to-nil fallback once unused. - Preserve nil/zero public handle behavior, details after use/discard, source - freezing, and private direct-key lifetime. Execution and repair must not mutate - shared metadata, including nested maps, cache controls, or schema values. - -**Check:** Race-enabled prepared lifecycle, concurrent claim/discard, engine/view -binding, independent details, mutation, credential, and repair tests. Public Run -results also remain detached from metadata retained by the handle. - -## Stage 14: Simplify Smoke Target And Public Report Detachment - -**Completed:** Smoke targets remain private through orchestration and public -reports use the trusted outward projection; endpoint sanitization and credential -redaction remain. Go 1.27.3 `go test ./internal/usecase . -run -'Smoke|Diagnostic'` passed, including report isolation, cancellation, and -diagnostic selection coverage. - -**Finding:** 5. **Files:** `internal/usecase/smoke_tester.go`, -`smoke_diagnostics.go`, smoke conversion/ownership tests. - -1. Keep accepted targets private and read-only through diagnostic orchestration; - remove their validation-only provider-map copy in `smokeReportTarget` where - no mutable ownership handoff occurs. Keep endpoint sanitization, credential - redaction, and genuine endpoint errors. -2. Use the existing trusted public execution-target projection in - `fromDomainSmokeTestReport`, rather than another `CopyMap`. Make report - conversion infallible and update facade callers, removing only the impossible - report-copy error path. Other operational/configuration errors remain. -3. Retain detached report facts, attempts, pointer fields, and nested provider - values. Delete the internal invalid-provider-map conversion-error test that - sustains revalidation of fabricated unaccepted reports; retain its meaningful - public acceptance/ownership protection at the real boundary. - -**Check:** Smoke conversion isolation, privacy, partial/canceled reports, and -configured-default/runtime-override diagnostics through the public facade. - -## Stage 15: Verify Cross-Path Ownership And Runtime Compatibility - -**Completed:** Cross-path checks cover mutations by an injected client on both -initial and repair requests, detached results and details, concurrent details -with claim/discard, and typed, numeric, null, cyclic, unsupported, and deep -runtime provider values. Remaining validating copies serve configuration -acceptance/overlay or public override acceptance; outward projections and schema -detachment remain ownership boundaries. Go 1.27.3 `go test -race -./internal/usecase ./internal/configuration ./internal/jsonvalue . -run -'Prepared|Provider|Extra|Smoke|Profile|Schema'` passed (the filtered jsonvalue -package selected no tests); the full `go test -race ./internal/jsonvalue` passed -separately. - -**Findings:** 5–8. **Scope:** focused existing public and package contract tests. - -1. Reconcile the ownership inventory with the delivered call paths. Verify that - prepared details/results, profile inspection, smoke reports, and injected - initial/repair requests cannot mutate retained engine/handle state. Extend a - representative existing test only where these changes leave a meaningful gap. -2. Keep valid typed/named runtime containers, numbers, nil/empty forms, map - replacement, and current invalid/cyclic/over-budget rejection at acceptance. - Distinguish those provider budgets from removed schema budgets. -3. Exercise a mutating injected client across initial and repair calls and - concurrent details/claim/discard where existing coverage supplies the seam. - Keep the built-in HTTP payload freeze and existing artifact defensive copy. -4. Review remaining copy/validation callers for purpose. Remove overlooked - repeated acceptance work on the selected paths, not all copies indiscriminately. - Do not preserve private copy-failure tests as new public requirements. - -**Check:** Run targeted race-enabled root/use-case/configuration/jsonvalue tests. -Record results and any reconciled ownership edge; no benchmark framework or exact -allocation threshold is needed. - -## Stage 16: Reconcile Canonical Documentation And Retire Obsolete Descriptions - -**Completed:** Canonical model-client, runner, source/validation, component, -and schema-format guidance now describes the delivered paths. The feature -roadmap records delivery with combined final checks pending; historical audit -evidence remains identified as baseline evidence. - -**Findings:** 1–8. **Scope:** GoDoc, `docs/internal/{overview,llm,runner,sources}.md`, -relevant consumer/format guidance, feature roadmap completion status. - -1. Remove descriptions of client routing defaults, uncaptured renderer/repository - fallbacks, standalone direct output validation, duplicate prepared metadata, - and prepared-schema copier limits. Describe the delivered ownership paths. -2. Record broadened schema acceptance only where consumer/format guidance needs - it. Keep the self-contained-schema rule, runtime provider budgets, and existing - number handling; add no resource/provider/performance promises. -3. Keep exact public contracts in GoDoc, formats/wire behavior with their owning - references, and private mechanics in internal docs. Update affected comments - throughout, not just the package inventory. -4. Update the feature roadmap with delivered outcomes and pending final checks; - do not mislabel the audit's original tests as replacement validation. Leave - unrelated completed-roadmap retirement and release notes to their own work. - -**Check:** Documentation links and whitespace, plus searches for the retired -names/claims to separate historical audit evidence from stale current guidance. - -## Stage 17: Validate And Close The Sprint - -**Completed:** Findings 1–7 are implemented and finding 8's runtime behavior is -preserved. With Go 1.27.3, `go test -race ./...` and `go vet ./...` passed, -including maintained examples. All tracked Go files are formatted; full diff -whitespace and repository hygiene checks passed. A local Markdown destination -and heading scan checked 580 links in 63 files; 153 historical code references -were corrected to their audit commits and verified against those Git objects. -Remaining validating copies are confined to acceptance/overlay boundaries; -trusted copies and transfers retain the documented ownership boundaries. No -release, push, or publication was performed. - -**Scope:** combined final state, all findings and deferred checks. - -1. Resolve remaining deferred compilation/tests; format changed Go files. Run - `go test -race ./...` and `go vet ./...` using the selected local toolchain. - The suite includes maintained examples; separate ordinary/full build/example - runs are not additional required checks for the same unchanged candidate. -2. Check documentation links, `git diff --check`, full intended/staged diff, - and repository status. Exclude secrets, generated artifacts, local workspace - files/replacements, and unrelated changes. No release publication check is needed. -3. Confirm removal of client defaults, duplicate root wire representation, - optional lookup fallbacks, uncaptured rendering, direct-validator lifecycle, - redundant copy-validation paths, duplicate prepared metadata, and schema-copy - budgets. Verify that remaining copies implement real ownership boundaries. -4. Record final evidence and completion for findings 1–7 and preservation of 8 - in the feature roadmap/plan. A deferred check or unimplemented selected change - is not completion. Preserve actual scope if a new blocker appears and surface - it rather than substituting a compatibility layer or narrowing the feature. -5. Commit the completed work under repository policy and report the result and - any remaining unrelated workspace changes. Do not push, tag, or publish. - -**End state:** The selected roadmap is implemented with fewer private workflows, -representations, and redundant traversals; public behavior and ownership remain -intact except for the explicitly removed incidental schema restriction. diff --git a/docs/roadmap/remediation.md b/docs/roadmap/remediation.md deleted file mode 100644 index 04f6511..0000000 --- a/docs/roadmap/remediation.md +++ /dev/null @@ -1,564 +0,0 @@ -# Implementation Review Findings - -## Responsibility Simplification Review - -Reviewed all 17 stages of the current [implementation plan](implementation.md) -against [responsibility-simplification.md](responsibility-simplification.md) -and its accepted audit decisions. The implementation spans `448e64d` through -`5ee602cc4b5a3afe4e7d361eb77181ef0940c7cd`, after planning commit `de46935`. - -The targeted architecture is implemented, and the ownership regression below -is now addressed. Its evidence and review table describe the pre-fix baseline; -the completion marker records the correction. No additional unintended -restriction or public guarantee was identified. Removing copier-derived schema -limits remains the selected acceptance change, while runtime provider-value -support remains in place. The earlier -33-stage review and its five completed findings are retained separately below. - -### R1. Isolate Custom JSON Encoders From Retained Metadata [P2] - -**Status:** Complete. **Stages:** 2, 13–15; particularly the shared prepared snapshot -and removal of private smoke-target detachment. - -Completion: built-in generation preparation uses trusted detachment before -payload construction, so whole-payload encoding and per-value encoding-failure -diagnostics cannot mutate retained provider values through their receivers. -The single prepared snapshot and named/typed custom encoding remain intact. -Public regressions failed before the fix and now protect prepared details, -result settings, smoke settings, caller input, custom wire output, and failure -classification without dispatch. Go 1.27.3 `go test -race ./...` and -`go vet ./...` passed, including the existing retry, repair, and concurrent -prepared-lifecycle coverage. Internal model-client documentation was updated. - -**Locations:** [provider payload construction](../../internal/llm/openai_compatible_client.go), -`openAIChatRequestPayload`, lines 468–475, and `PrepareGeneration`, lines 133–139; -[prepared metadata sharing](../../internal/usecase/prepared_execution.go), -`PrepareExecution`, lines 52–57; -[smoke report target](../../internal/usecase/smoke_tester.go), `smokeReportTarget`. - -Supported named map/slice values can implement `json.Marshaler`. The acceptance -copier preserves compatible concrete types, including their methods. The built-in -payload builder places provider values into the wire map by reference, and -`json.Marshal` can invoke those methods on retained values. For example: - -```go -type mutatingMap map[string]string - -func (m mutatingMap) MarshalJSON() ([]byte, error) { - m["value"] = "changed by encoder" - return json.Marshal(map[string]string(m)) -} -``` - -This is an accepted finite, acyclic string-keyed map. Its custom encoding method -mutates only its receiver, not external state. No concurrent caller mutation or -invalid injected collaborator is involved. - -**Verified reproduction:** An offline public-API probe constructs an ordinary -engine with the built-in LLM client and an in-memory HTTP transport returning a -valid response. It supplies `ExtraParams["options"] = mutatingMap{"value": -"original"}`. Preparation succeeds. It compares details before/after -`RunPrepared`, then separately inspects a text-inference smoke report using the -same kind of runtime override. Both operations return nil Go errors: - -| Observation | Current implementation | With pre-sprint detachment restored via Go overlay | -| --- | --- | --- | -| Prepared details before execution | `original` | `original` | -| Prepared details after execution | `changed by encoder` | `original` | -| Smoke report's effective provider setting | `changed by encoder` | `original` | -| Caller-owned input map | `original` | `original` | - -The comparison overlays only `internal/usecase/prepared_execution.go` and -`internal/usecase/smoke_tester.go` from `de46935` onto the current code. It -isolates the lost metadata ownership separation; it is not a test of the entire -old revision. The probe and overlays live outside the repository. - -**Cause and impact:** Removing the second prepared snapshot is correct only if -all consumers treat the shared snapshot as immutable. Injected clients receive -independent public requests, but built-in encoding is another callback boundary: -custom encoding code can mutate its input. The new sharing exposes that mutation -through retained details and diagnostic settings. It also creates an unsynchronized -mutation path alongside otherwise supported concurrent details reads. The -existing mutating-client regression exercises an injected client, so it does not -cover this built-in path. Encoder failure diagnostics are relevant too: -`PrepareGeneration` may separately marshal original extra-parameter values when -whole-payload encoding fails. - -**Required correction:** Keep the single immutable metadata snapshot, but detach -mutable accepted provider values at the built-in encoding boundary before any -custom encoder runs. Ensure both payload encoding and per-value encoding-failure -diagnostics operate on detached values. Reuse trusted copying where appropriate; -do not reinstate the duplicate details snapshot or reintroduce input validation -merely to copy accepted data. - -Retain current named/typed input and custom wire-encoding behavior, native retry -payload freezing, error classification, and the outward public ownership copies. -Rejecting these types, stripping their methods through JSON-native normalization, -or requiring nonmutating custom encoders would narrow the selected runtime -contract instead of repairing ownership. - -**Regression coverage:** Exercise a supported named provider map with a -receiver-mutating `MarshalJSON` through the built-in offline transport. Check -unchanged prepared details and smoke effective settings, independent caller -input, and the expected custom wire representation. Include the encoding-failure -path so its per-value diagnostic cannot mutate retained state. Retain existing -injected initial/repair mutation and concurrent prepared lifecycle protection. - -### Stage-By-Stage Review - -This is the pre-remediation review record; R1's completion above addresses its -identified gap without changing the other selected boundaries. - -| Stage | Current evidence and result | -| --- | --- | -| 1 | Constructor routing defaults and their only error path are removed. Resolved endpoint/model checks and HTTP-client timeout ownership remain. No additional finding. | -| 2 | One root payload builder preserves field omission/presence, reserved extensions, session/native schema handling, and encoded-body freezing. Its borrowed provider values participate in R1. | -| 3 | Private repositories explicitly require captured definitions, metadata, requirements, and profile identity; capability/hash fallbacks are removed. No additional finding. | -| 4 | Repository fixtures implement captured operations; metadata-only and selection/error coverage remain. No additional finding. | -| 5 | Rendering uses captured templates only, with per-call clones/functions and retained session/input/message behavior. No additional finding. | -| 6 | Renderer fixtures/benchmarks supply captured parses; syntax, isolation, cancellation, artifact-text, and requirements comparisons remain covered. No additional finding. | -| 7 | Smoke syntax and schema checks use separate prepared plans; malformed syntax leaves schema matching unevaluated, including scalar/nonmatching controls. No additional finding. | -| 8 | Direct validator interface/methods and on-demand callbacks are deleted; shared instance validation and eager source-specific preparation remain. No additional finding. | -| 9 | Tests use prepared plans and retain source containment, portability, references, precision, freezing, and synchronous cancellation coverage. No additional finding. | -| 10 | Trusted schema detachment removes provider-copy budgets. Deep schema isolation and a real schema with over 100,000 annotation values are covered. No additional finding. | -| 11 | Registry/profile lookup uses trusted detachment after final acceptance, while configuration validation/overlay and runtime acceptance checks remain. No additional finding. | -| 12 | Private target resolution transfers owned maps, preserving empty-map inheritance and nil projections. Outward/injected projections still detach values. No additional finding. | -| 13 | One redacted metadata snapshot is shared, and cleanup no longer writes it; owner/claim/discard state and outward copying remain. Built-in custom encoding can still mutate the snapshot: R1. | -| 14 | Smoke projection is infallible and outward values are detached; credentials/endpoints remain sanitized. Built-in encoding can mutate effective settings before final report projection: R1. | -| 15 | Added tests cover injected initial/repair mutation, returned details/results, named/numeric runtime values, and concurrent lifecycle reads. The built-in custom-encoder path is missing: R1. | -| 16 | Canonical format, integration, model-client, runner, source, and component guidance reflects the selected changes without new resource/provider promises. The immutability description is not fully realized until R1 is fixed. | -| 17 | Fresh full race tests and vet pass, but those checks do not cover R1. Documentation/history links and delivered scope were reviewed; completion still requires the ownership correction. | - -### Review Validation And Limits - -At the reviewed pre-fix baseline, using Go 1.27.3 with `GOTOOLCHAIN=local`, -`GOWORK=off`, and `GOPROXY=off`: - -- `go test -race ./...` passed, including the maintained offline examples. -- `go vet ./...` passed. -- The public ownership probe reproduced both metadata mutations, with nil Go - errors and unchanged caller values; the two-file historical overlay prevented - both mutations. - -The review changed no production code or repository tests. Its source analysis -and targeted probe established R1 despite the passing existing suite; the -completion marker above records the subsequent fix and validation. The review -does not claim universal callback safety, provider compatibility, or defect-free code. - -## Earlier Audit Remediation Review: Scope And Status - -Reviewed all 33 stages of the [implementation plan](implementation.md) against -the [selected roadmap](audit-remediation.md), its accepted catalog decisions, -and the implementation through `6673284312de23bde86c20eb8de4f3865b157471`. -The implementation changes begin after planning commit `6a1a32a`. - -An integration-focused follow-up reviewed the same implementation at `356563a`, -the documentation-only commit recording the original review. It adds finding 5; -the earlier findings were still open at that audit baseline and are not duplicated. - -The broad architecture and selected simplifications are in place. All five -findings are now addressed, with completion markers below: three correctness/ -completeness issues, one error-projection efficiency issue, and one documentation -status gap. Finding descriptions and audit evidence retain the historical -pre-fix behavior; the completion markers describe the delivered corrections. - -The accepted artifact ownership, supported convenience-source types, and -self-contained schema changes are intentional compatibility changes. No -inadvertently strengthened contract was identified beyond those deliberately -selected changes. Historical measurements are observations, not performance -contracts. - -## Earlier Audit Findings - -### 1. Preserve Caller Cancellation When Authentication Also Overflows [P2] - -**Status:** Complete. **Stages:** 7, 21–22. - -Completion: interrupted body reads preserve caller cancellation before overflow -classification. Public below-limit and overflow-boundary regressions retain HTTP -facts, unknown acceptance, incomplete cancellation reports, and body closure. - -**Locations:** [authentication adapter](../../internal/llm/openrouter_authentication.go), -`ProbeOpenRouterKey`, lines 61–67; -[outcome mapping](../../internal/usecase/smoke_authentication.go), -`authenticationProbe.run`; -[completion](../../internal/usecase/smoke_tester.go), `finishHTTP`. - -The adapter checks the number of returned bytes before the read error. A legal -`io.Reader` can return the allowance-plus-one byte and `context.Canceled` in the -same read. The adapter sets `Overflow` and returns nil, discarding cancellation. -The use case then selects `ExecutionFailureOther`; `finishHTTP` does not apply -caller cancellation because it only does so for context-classified failures. - -**Verified reproduction:** An offline public `SmokeTest` uses an HTTP 200 body -that cancels the caller and returns `context.Canceled` with its final bytes. -With 1,024 bytes it returns the cancellation and an incomplete report. With -1,048,577 bytes it instead returns: - -- nil Go error despite `ctx.Err() == context.Canceled`; -- `Completed == true`; -- failure `other` and reason `authentication_response_limit_exceeded`. - -The body is closed and the read allowance is honored in both cases; those parts -of the remedy work. The completed overflow result violates Stage 7's explicit -exception for caller cancellation. - -**Required correction:** Preserve context interruption across the adapter and -outcome mapping when overflow and interruption coincide. Retain independently -observed HTTP facts and unknown credential acceptance. Ordinary overflow must -still be a completed diagnostic with nil Go error. Do not change the established -treatment of a known, concluded authentication result merely because cancellation -arrives later. - -**Regression:** Exercise bytes-plus-cancellation at the overflow boundary through -the public diagnostic path, alongside ordinary overflow and interrupted -below-limit controls. Keep caller deadline handling consistent with the same -existing cancellation policy. - -### 2. Check Schema Self-Containment Independently Of Its Source Filename [P2] - -**Status:** Complete. **Stages:** 11–13. - -Completion: shared compilation checks both the source base and an unrelated -submission base through the same compiler rules, including unused declarations. -Host and filesystem regressions preserve fragment, explicit-ID, and embedded-ID -references while rejecting unsent source-name dependencies. - -**Locations:** [schema compilation](../../internal/validate/standard_validator.go), -`compileSchemaDocument`, lines 426–445, and its host/`fs.FS` callers; -[public wire regression coverage](../../schema_file_contract_test.go). - -The compiler registers the document at its physical file or source-relative -retrieval URI. Its denying loader rejects unknown resources, but references -back to that registered retrieval URI do not invoke the loader. Consequently, -successful local compilation does not establish that the outbound root is -self-contained without PromptKit's source context. - -**Verified reproduction:** Load a file named `schema.json` containing: - -```json -{"$defs":{"value":{"type":"integer"}},"$ref":"schema.json#/$defs/value"} -``` - -Public `PrepareExecution` succeeds. An offline HTTP client captures the actual -native structured-output payload, returns `3`, and `RunPrepared` passes local -validation. However, compiling the captured schema at a neutral submission URI, -`https://provider.invalid/submitted-schema`, with external loading disabled -attempts to load `https://provider.invalid/schema.json` and fails. The source -retrieval URI was never sent to the provider. Changing the reference to -`#/$defs/value` succeeds both locally and at the neutral URI. - -This is a packaging gap in the selected self-contained-schema remedy, not a -claim about any particular provider's schema support or enforcement. - -**Required correction:** Establish self-containment without relying on the -source retrieval URI being available upstream. Detect references that depend -on that unsent filename/path and provide an actionable error. Preserve valid -fragment references, explicit/embedded IDs, anchors, recursion, dialect handling, -and checks of otherwise unused schema declarations. Keep source location for -root loading and diagnostics; do not add external loading, a lexical ban on all -non-fragment references, a bundler, or a provider-acceptance guarantee. - -**Regression:** Cover host-file and `fs.FS` source-name dependencies, retain -explicit-ID and fragment controls, and evaluate the actual outbound schema -without the original source base. Preparation must fail before dispatch when -the selected self-contained contract is not satisfied. - -### 3. Avoid Duplicating An Already-Retained Error Cause [P3] - -**Status:** Complete. **Stages:** 3–4. - -Completion: ordinary operation projection retains its mapped cause once, without -joining the original subtree a second time. Multi-engine regression coverage -checks exact cause/owner matching and one occurrence of an independent leaf; -specialized projection and redaction tests remain intact. - -**Location:** [public error projection](../../errors.go), `mapPublicError`, -particularly lines 26–38. - -The ordinary operation branch joins the owner, the mapped cause, and the -original cause. For ordinary errors, the mapped cause already retains the -original through wrapping or is that same error. This introduces two paths to -the same cause. When an injected client returns an error from another PromptKit -engine, each additional enclosing engine repeats the previous cause tree in -both branches. Formatting duplicates diagnostics; unsuccessful error-tree -searches can also revisit the repeated branches. - -**Verified reproduction:** Begin with a public missing-prompt error, then run a -valid parent whose injected client returns that error; repeat with each parent's -result. Owner and exact-cause matching remain correct, but formatted byte counts -at depths zero through eight are: - -```text -158, 361, 767, 1579, 3203, 6451, 12947, 25939, 51923 -``` - -The input is a linear chain of consumer operations; the projection creates -exponential repeated text/traversal. Even one ordinary error repeats its cause -unnecessarily. This is introduced by the new join, not required to preserve the -owning category. - -**Algorithmic analysis:** Let `d` be the number of enclosing operations, with a -fixed-size leaf diagnostic and fixed-size per-operation context. The ordinary -mapped cause already wraps the previous error, yet the join also stores that -previous error as another child. Its formatted length therefore follows -`L(d) = 2*L(d-1) + O(1)`, giving `Theta(2^d)` output bytes. The cause graph needs -only `O(d)` distinct error objects, but standard `errors.Is`/`errors.As` traverse -both paths without remembering previously visited objects. An unsuccessful -search consequently follows `W(d) = 2*W(d-1) + O(1)` as well. The mapper itself -performs such unsuccessful category/type searches, so this is not limited to -callers that log the final diagnostic. - -Retaining one already-mapped cause path removes the avoidable branching: -formatted length becomes `L(d) = L(d-1) + O(1)`, and an individual formatting or -unsuccessful search over that corrected ordinary chain can be linear in `d`. -This is a structural alternative, not a request for error-graph memoization, -new nesting limits, or exact timing/allocation guarantees. It does not assert -that a sequence of `d` separate public operations, each processing its own -growing error, has linear total cost. Arbitrary consumer error implementations -also retain their own costs. - -**Required correction:** Retain the owning category and each necessary cause -without adding an original cause that the projection already retains. Preserve -`errors.Is`/`errors.As`, specialized capacity/provider/retry projections, and -redacted transport formatting. Do not require category exclusivity, discard -consumer causes, or impose a new nesting limit. - -**Regression:** Extend composed-error coverage across several enclosing -operations. Verify owner and exact-cause matching and that an independent leaf -diagnostic is not duplicated at every boundary. Protect this relationship rather -than exact complete error wording or a timing threshold. - -### 4. Reconcile The Feature Roadmap's Implementation Status [P3] - -**Status:** Complete. **Stage:** 33. - -Completion: the feature roadmap now records the delivered sprint, links stage -evidence and this follow-up record, and distinguishes historical baseline -measurements from current status without prematurely closing finding 5. - -**Location:** [feature roadmap](audit-remediation.md), Purpose And Status and -workstream completion reporting. - -The feature roadmap still says neither document describes completed remediation -and all workstreams are pending. The implementation plan now records all 33 -completed stages and post-change -measurements. Stage 33 specifically required actual completion/evidence to be -recorded in the roadmap. Its current status contradicts the delivered work and -makes it difficult to distinguish original findings from remaining review work. - -**Required correction:** Record that the implementation landed, point to the -stage evidence and outstanding findings here, and distinguish historical -assessment-only statements from current overall status. Preserve the original -baseline measurements and decision provenance as historical evidence. Do not -claim all issues are resolved until the findings are addressed; normal roadmap -retirement can follow when appropriate. - -### 5. Integrate Operation Ownership With Terminal Observation Classification [P2] - -**Status:** Complete. **Related stages:** 3–4, 25–26. - -Completion: a private public-boundary wrapper retains operation ownership and a -single mapped cause. Terminal observations classify that owner while retaining -context precedence and generation-specific transport/provider details. Observed -run, prepared-run, repair, and artifact regressions retain child cause identity. - -**Locations:** [terminal classifier](../../observations.go), `observedFinish`, -lines 160–192; [public error projection](../../errors.go), `mapPublicError`; -[run boundaries](../../engine.go), `Run` and `RunPrepared`. - -The new operation errors and public projection preserve the enclosing operation's -category while retaining child causes. The existing terminal observer classifier -still uses a whole-chain sentinel-priority switch. A retained child -`ErrPromptNotFound` wins over the enclosing generation or artifact failure. -The generation coordinator, which classifies the actual injected call, reports -`generation` instead. Thus the owning-operation remedy is incomplete at the -existing observation boundary even though the returned error now retains the -correct outer category. - -**Verified reproduction:** Obtain a child engine's missing-prompt error, then -return it from a valid parent engine's injected client or artifact reader. With -an observer, the offline race-enabled probe produces: - -| Parent path | Last attempt failure | Terminal failure | Parent error category retained | -| --- | --- | --- | --- | -| `Run`, initial generation | `generation` | `prompt_not_found` | `ErrLLMGenerate` | -| `RunPrepared`, initial generation | `generation` | `prompt_not_found` | `ErrLLMGenerate` | -| `Run`, repair generation | `generation` | `prompt_not_found` | `ErrLLMGenerate` | -| `Run`, artifact reading | No model attempt | `prompt_not_found` | `ErrArtifactLoad` | - -The exact child error remains matchable in all four cases. The parent's prompt -exists; the terminal event nevertheless describes a missing parent prompt. This -can misdirect downstream monitoring and recovery. Existing composed-cause tests -check returned errors, while existing attempt/terminal consistency tests use -simple injected errors and miss this combination. - -**Required correction:** Preserve and consume the owning-operation information -when classifying terminal events instead of selecting a nested operation by -global sentinel priority. Keep legitimate context/deadline precedence, built-in -provider/transport/malformed classifications, retry metadata, and public -`errors.Is`/`errors.As` access. This does not require a new exported API or -category exclusivity. Simply moving generation ahead of other sentinels would -create the same ambiguity for other owners. - -**Regression:** Cover observed ordinary/prepared generation, repair failure, -and artifact failures carrying another engine's error. Preserve direct missing -prompt/profile classification and existing cancellation/provider cases. Do not -require universal attempt/terminal equality: later validation or cancellation -can legitimately change an operation's final outcome. - -## Earlier Stage-By-Stage Review Record - -Each stage was checked against its requested outcome, changed implementation, -relevant tests, and documentation. The entries below identify the review focus; -“no additional finding” is a review result, not a guarantee of defect-free code. - -| Stage | Current evidence and result | -| --- | --- | -| 1 | Development guidance selects the latest available local toolchain independently of the module minimum; validation used Go 1.27.3 and `go.mod` remains at 1.27.0. No additional finding. | -| 2 | Both source helpers hand provider maps directly to ConfigKit; the converter is deleted. Source tests cover capture, unsupported values, cycles, ordering, and inheritance. No additional finding. | -| 3 | Internal operation errors carry explicit owner/cause with `Is`/`Unwrap`; lookup and execution boundaries use them. See finding 3 for facade composition. | -| 4 | Public nested-error and specialized-capacity tests retain owner/cause. Repeated cause retention remains inefficient: finding 3. | -| 5 | Response-reader errors are recorded below the limited reader and distinguished from parser/framing failures at both decode points; cause/redaction tests cover bytes with errors. No additional finding. | -| 6 | Built-in preparation has explicit typed error/evidence markers; inference diagnostics check cancellation first and report construction errors without dispatch. Injected clients retain their separate semantics. No additional finding. | -| 7 | Authentication reads at most allowance plus one, closes bodies, and reports explicit overflow. Simultaneous cancellation remains incorrect: finding 1. | -| 8 | Artifact GoDoc, defensive adapter copying, migration guidance, and concurrent tests follow the accepted storage transfer. No additional finding. | -| 9 | Prepared handles retain an immutable non-zero-sized identity token, not the runner; claim/discard locking remains. Normalized session IDs clone accepted storage. No additional finding. | -| 10 | Runner and engine declare `ValidationPreparer`; concrete validator constructors preserve direct validation and nil-plan behavior. No additional finding. | -| 11 | Host and `fs.FS` paths share compilation, retain root loading, and deny external loader requests. Source-base dependence remains: finding 2. | -| 12 | Declared schema traversal uses compiler-resolved dialects, escaped pointers, and schema-bearing positions, excluding annotations. It does not resolve finding 2. No separate traversal finding. | -| 13 | Public schema-file precedence/errors, freezing, and native submission are tested; the unused helper result is gone. Outbound relocation coverage misses finding 2. | -| 14 | Overlay clones the lower map once, merges owned descendants, and copies replacements. Isolation/null/array cases and comparable allocation results are recorded. No additional finding. | -| 15 | Trusted projection copying moved to `internal/jsonvalue`; fallible runtime input copying remains distinct. Type/nil/number/schema-key behavior is covered. No additional finding. | -| 16 | Owned evidence sets merge smaller into larger; callers consume values/flags rather than independent map snapshots. Work-count evidence and requirements tests support the change. No additional finding. | -| 17 | Shared branch orchestration preserves `with` body-dot versus else-dot behavior, inactive paths, and conservative inspection. Independent rendering comparisons remain. No additional finding. | -| 18 | Bounded normalization preserves invalid-byte-run replacement and rune boundaries; encoding delegates to standard JSON marshaling. No additional finding. | -| 19 | Complete/omission feedback paths use bounded prefixes and retain complete stored diagnostics/candidate behavior. Boundary/escaping tests and comparable allocation results are recorded. No additional finding. | -| 20 | Scope resolves before inventory selection; named ranges use binary search, All copies once, and unqualified selection scans without a full prior copy. Ordering, ownership, cancellation, and allocation evidence remain. No additional finding. | -| 21 | OpenRouter endpoint derivation and account-response interpretation live in the outbound adapter behind a narrow capability. Finding 1 remains in its read/error ordering. | -| 22 | Use cases retain selection/applicability/reporting; target-only detachment replaces unused broad cloners. Overflow/context outcome composition contributes to finding 1. | -| 23 | Extracted record validators retain final-handoff traversal, resource indexes, field paths, and existing checks. No raw-source validation phase was added. No additional finding. | -| 24 | Input metadata, message/template capture, output conversion, and effective targets are separated while preserving presence, source-relative reads, lazy origins, and final hash assembly. No additional finding. | -| 25 | Attempt results replace parallel returns without moving acquisition/release outside attempt scope. Built-in and injected dispatch remain distinct. No additional finding. | -| 26 | Shared accounting runs after permit release and preserves distinct HTTP/usage facts; observation and scheduling tests exercise the real coordinator. No additional finding. | -| 27 | Synthetic markers exercise value/pointer formatting and direct-key JSON omission; credential, identity, listing, and provider-map documentation is corrected. No additional finding. | -| 28 | One internal test owns the complete bundled asset oracle; smaller public option/wire/smoke tests retain distinct assembled behavior. No additional finding. | -| 29 | Artifact tests compare same-body inline/file opaque hashes and actual byte/size behavior; benchmark correctness setup remains outside timing. No additional finding. | -| 30 | Unconsumed backend presence/queue state, endpoint echo, helper result, and named orphan helpers are removed; effective policy/provenance and public features remain. No additional finding. | -| 31 | Runtime definitions use the output contract and captured message content without duplicate fields. Configuration `content_file`, override defaulting, hash relations, and migration guidance remain. No additional finding. | -| 32 | Runner scheduling uses `generation.NewInjected`; the production wrapper is deleted. Useful manager scenarios use test-only one-shot scaffolding. No additional finding. | -| 33 | Current GoDoc/consumer/format/integration/internal guidance largely matches the code; validation passes. Feature-roadmap completion reporting remains stale: finding 4. | - -## Earlier Validation And Limits - -At the reviewed commit, using Go 1.27.3 with `GOTOOLCHAIN=local`, `GOWORK=off`, -and `GOPROXY=off`: - -- `go test -race -count=1 ./...` passed, including the maintained examples. -- `go vet ./...` passed. -- Tracked Go files produced no `gofmt -l` output. -- `git diff --check 6a1a32a..HEAD` passed. -- Additional temporary Go test overlays outside the repository reproduced - findings 1–3. The cancellation and relocated-wire-schema assertions fail as - described; the nested-error probe measured the duplicated text while retaining - correct category/cause matching. - -The probes use controlled readers, HTTP transports, and the pinned schema -compiler. No credentials, provider calls, external schema retrieval, dependency -changes, or publication were needed. They establish local failure mechanisms, -not universal provider behavior. Optimization review checked source changes and -the recorded comparable measurements; it did not add benchmark thresholds or -new release gates. - -## Earlier Integration Follow-Up Coverage - -The follow-up focused on interactions between the changed components and the -existing consumer workflows, rather than repeating the stage-local review. -Production code is unchanged between `6673284` and the follow-up baseline -`356563a`. - -| Integration seam | Review and validation result | -| --- | --- | -| Convenience sources, ConfigKit inheritance, and optimized provider overlays | An assembled probe captured backend/profile source maps before caller mutation, inherited a profile, and verified recursive backend-to-profile overlay plus detached inspection results. Passed. | -| Namespace-filtered inventories, inspection, and execution selection | The assembled probe listed a scoped prompt, inspected requirements, validated its request, and prepared it through the same view. An unscoped owning engine then ran the handle with its captured prompt/profile/backend identities. Passed. | -| Trusted copying, frozen validation, runtime overrides, and repair | The assembled probe combined direct session and reasoning/numeric overrides, appended messages, a schema with internal definitions, and bounded repair. It mutated prepared details, client-owned nested provider/schema values, and the source schema after preparation. Repair and frozen validation remained correct, and inspection/preparation hashes agreed. Passed. | -| Generation accounting, retries/repair, and capacity | Existing assembled tests for shared/independent pools, backoff permit release, logical admission, repair, frozen credentials, usage, and observer events passed under the race detector. No additional scheduling/accounting finding. | -| Prepared-owner token, namespace views, and observation isolation | Existing lifecycle, foreign-owner, concurrent claim/discard, delegated-operation, and observer-isolation tests passed under the race detector. No additional lifecycle finding. | -| Operation-owned errors and existing terminal observers | New offline observed-call probes fail in all four combinations documented in finding 5. Existing plain injected-error tests still pass. | -| Shared schema compiler and provider-native output | Source-to-native-payload review retains finding 2; the new assembled frozen-schema/repair probe passed for a genuinely self-contained fragment-reference schema. No duplicate finding. | -| HTTP read classification and diagnostic adapter/reporting | Adapter/coordinator/use-case review and the focused internal suites retain the existing read-failure classifications. The overflow/cancellation conflict remains finding 1. No duplicate finding. | -| Requirement-evidence ownership and renderer/preflight behavior | Evaluation/branch ownership was reviewed alongside actual rendering and the assembled requirements/preflight workflow. Existing prompt requirements and use-case suites passed; no additional recognition scope or execution-readiness guarantee was introduced. | - -Validation used Go 1.27.3 with `GOTOOLCHAIN=local`, `GOWORK=off`, and -`GOPROXY=off`: - -- The focused race-enabled public integration selection passed, covering - retry/repair/accounting, backend capacity, frozen prepared execution, - delegation, namespace views, schema-file wiring, programmatic sources, and - existing injected-error observation classifications. -- Race-enabled configuration, prompt, validation, generation, use-case, and - model-client suites passed. -- The additional composed-snapshot/repair probe passed under the race detector. -- The new operation-ownership observation probe failed as documented in - finding 5, independently of the already recorded findings. - -Temporary probes were supplied using Go test overlays outside the repository; -no implementation or permanent test changes were made. All provider behavior -was controlled offline. The prior full-suite/vet evidence above remains -applicable to the unchanged implementation; this follow-up adds focused -integration evidence rather than a new release gate. - -## Earlier Algorithmic Follow-Up Coverage - -A further source-based review at `7203e5d` focused on algorithmic correctness, -efficiency, and scalability of the implemented stages. It confirms the -exponential behavior in finding 3 and adds the recurrence and refactoring -opportunity there, rather than creating a duplicate finding. No additional -distinct algorithmic or ordinary-correctness finding was identified in this -pass. All five findings were open at that audit baseline; their completion -markers above own the current remediation status. - -Remediation completion validation: Go 1.27.3 with `GOTOOLCHAIN=local`, -`GOWORK=off`, and `GOPROXY=off` passed the complete -`go test -race -count=1 ./...` suite and `go vet ./...`. New regressions cover -authentication read interruption at the overflow boundary, host/FS schema -portability, relocated native schema submission, nested-cause duplication, and -operation-owned terminal observations. Focused checks passed before each code -unit was committed. Documentation links, formatting, and whitespace were checked. - -The review used the plan and roadmap to exclude deliberate algorithm choices: -the small-to-large evidence merge, owned provider overlay, bounded feedback -encoding and prefix selection, namespace binary searches, declared-schema -location compilation, trusted-copy traversal policies, and selected ownership -copies were not reopened. In particular, the chosen unqualified-list scan and -schema-location strategy are not reported as new optimization requests. - -For the remaining implementation glue: - -- Source helpers map records once before ConfigKit's handoff, without the old - recursive compatibility expansion. ConfigKit's own processing is outside - this review's implementation scope. -- Extracted record validators use a backend-ID index rather than scanning every - backend for each profile. Prompt input duplicate detection uses a set; input - and message capture process their records once, with one final definition - hash and one lazily shared metadata snapshot. Existing sorted inventory - construction and dependency/template work are separate costs, not disguised - linear-time guarantees. -- The response-error recorder adds constant metadata work per read. The - authentication recognizer parses a fixed number of document levels/fields, - so its processing is linear in the consumed body size, not multiplied by - arbitrary input depth through a custom recursive recognizer. -- Generation accounting updates a fixed number of usage/event fields per - attempt and appends captured attempt records with amortized constant append - cost. Storing `A` requested attempt records requires `O(A)` space. Error - classification additionally pays for the supplied cause graph, including - finding 3; arbitrary collaborator/callback work is not constant-time. -- The repair request uses the original message prefix plus a bounded-count - suffix, not an accumulating conversation from every prior repair. Repeated - prefix processing follows the amount of message data actually resubmitted; - it is not an unintended growing-history algorithm. - -The complexity finding follows from current source and the standard library's -error-tree traversal. No new benchmarks, real-world performance collection, -provider calls, implementation changes, or public complexity guarantees were -introduced. Existing validation evidence above applies to the unchanged code. diff --git a/docs/roadmap/responsibility-simplification.md b/docs/roadmap/responsibility-simplification.md deleted file mode 100644 index abc2b41..0000000 --- a/docs/roadmap/responsibility-simplification.md +++ /dev/null @@ -1,228 +0,0 @@ -# Responsibility Simplification Roadmap - -## Purpose And Status - -Simplify PromptKit's internal machinery by removing unused accommodations, -redundant representations, and repeated acceptance work while preserving the -library's substantive feature set and selected ownership boundaries. - -This roadmap captures the target state and work arising from the accepted -[responsibility audit](audit/A1-responsibility.md). Findings 1–7 are selected -for remediation; finding 8 selects preservation of runtime input behavior. -The selected implementation and combined final validation are complete. - -The work began from the `f660b6e` decision baseline and production audit -baseline `1ff6d77`. The audit owns historical evidence and decision rationale; -this document owns the selected target and delivery scope. The -[implementation plan](implementation.md) records implementation and validation. -The older -[audit remediation](audit-remediation.md) describes an earlier completed sprint; -its previous plan is retained in Git history, not the current implementation plan. - -## Targeted End State - -PromptKit retains its public facade, ConfigKit composition boundary, captured -configuration, generation coordination, and existing consumer extension points. -Within those boundaries: - -- The built-in client consumes resolved targets without its own unused - endpoint/model fallback settings and builds one root provider payload. -- Prompt/profile lookup and rendering consume captured configuration directly, - without alternate uncaptured workflows maintained for private fixtures. -- Output validation uses the existing prepared-plan lifecycle throughout, - including both syntax and schema checks in smoke diagnostics. -- Already accepted values are detached when ownership requires it, rather than - repeatedly revalidated merely to make a copy. -- Executable preparation retains one private, redacted metadata snapshot, with - separate execution-only state and one outward details projection/copy. -- Schema acceptance is governed by the decoder/compiler and the selected - self-contained-schema rule, without additional copier-derived tree budgets. -- Runtime provider-value acceptance, limits, and current copying semantics - remain unchanged. - -The intended benefit is fewer execution paths, traversals, retained duplicate -graphs, and incidental error branches. This does not establish allocation, -throughput, complexity, or numerical-precision guarantees. - -## Selected Work - -The following sections record the accepted work and its rationale. The target -above is implemented; final checks are recorded in the implementation plan -rather than inferred from the audit's baseline tests. - -### Resolved Targets And Direct Wire Construction - -**Audit findings:** 1–2. - -At the audit baseline, the [LLM adapter](../../internal/llm/openai_compatible_client.go) -stored constructor-level endpoint/model defaults that engine assembly never -supplied, and built an intermediate root wire struct before encoding. - -Retire those constructor settings, stored fallback state, and fallback branches. -Backend/profile/request resolution remains the owner of target selection. Keep -checks needed to encode and dispatch the selected target, and preserve HTTP -client ownership, timeouts, authentication, and request preparation. - -Construct the root wire payload directly from the resolved generation request. -Typed nested message, content-block, cache-control, and schema-envelope values -can remain where useful. Preserve optional-control omission and explicit-zero -presence, session IDs, structured output, reserved-field checks, extension-field -flattening, and private dispatch evidence. This is removal of an intermediate -representation, not a new serializer or replacement settings-resolution layer. - -Fixtures that currently place defaults on an internal client should instead -provide resolved request targets. Retain meaningful wire and invalid-target -coverage without preserving the deleted fallback mechanism. - -### Captured Lookup And Rendering Only - -**Audit finding:** 3. - -Production [engine assembly](../../internal/engine/engine.go) already supplies -the captured configuration repository and compiled renderer. Align the private -lookup contracts with the captured operations actually consumed, retaining -narrow metadata queries rather than loading entire definitions for inspection. - -Retire optional-capability probes, fallback full-definition/profile loads, -invocation-time definition hashing, and the renderer's uncaptured parsing path. -Definition hashes and templates remain supplied by configuration capture. -Keep per-call template cloning and request-function binding, which protect -captured templates from request-specific mutation. - -Migrate fixtures to captured templates/metadata using the existing compiler or -real in-memory configuration. Preserve meaningful renderer and inspection -behavior rather than reconstructing a second production workflow inside test -helpers. Convenient no-repair runner construction and useful external-boundary -doubles need not disappear merely because they are test conveniences. - -### One Output-Validation Lifecycle - -**Audit finding:** 4. - -The [validation component](../../internal/validate/standard_validator.go) -remains responsible for output syntax and schema checks. Use its existing -prepared-validation plans for both checks in -[smoke inference](../../internal/usecase/smoke_inference_case.go): a JSON-only -plan for syntax and a schema plan for conformance. The JSON-only plan requires -no schema I/O or compilation. Schema evaluation follows successful syntax -validation; invalid syntax leaves schema conformance unevaluated. Preserve -existing cancellation handling and neutral diagnostic facts, including valid -JSON scalars that fail the diagnostic schema. - -Retire the standalone direct-validation interface, concrete direct methods, -and redundant on-demand schema-loading callbacks after their smoke caller is -migrated. Meaningful source, schema, and cancellation cases move to prepared -validation. Shared instance validation and source-specific root loading remain. -Do not preserve the removed interface through a test-only reimplementation. - -This deliberately supersedes the earlier selection to retain the private direct -workflow and its decode-before-schema-I/O behavior. It does not change public -`ValidateRequest`, request preflight, repair policy, or eager schema preparation. -No composite validator or diagnostic framework is needed. - -### Acceptance, Detachment, And Prepared Snapshot Ownership - -**Audit findings:** 5–6, with finding 8's preservation decision. - -Acceptance and detachment have distinct responsibilities across captured -backend/profile lookup, prepared details, execution results, injected-client -and repair requests, and smoke reports. The target removes redundant traversals -across those paths together, rather than moving a copy into another helper. -The implementation plan records the corresponding ownership inventory. - -Keep validation at actual input-acceptance boundaries. For previously accepted -values, use existing trusted detachment where a mutable copy is needed and -share or transfer private values where ownership permits. Remove redundant -validation/error plumbing only when the relevant acceptance and mutation paths -justify doing so. A blanket replacement of validating copies is not the target. -Infallible projection should not acquire fallible-copy behavior or hide a -copying failure behind an empty result. - -Executable preparation should retain one immutable, credential-redacted -metadata snapshot shared by details and the execution payload. Direct -credentials, observer callbacks, retry settings, and validation resources remain -separate execution-only state with their existing lifecycle. Remove the current -payload-cleanup write into metadata before sharing it. Public details still -receive an independent mutable projection and remain available after execution -or discard; eliminating the internal copy before that projection must not expose -borrowed internal values to callers. - -Preserve prepared-handle identity, single claim/discard synchronization, source -freezing, credential privacy, and injected-client ownership. Retain current -runtime provider-value acceptance and limits, supported number/container forms, -and type/nil behavior. Runtime values do not become ConfigKit inputs merely to -reuse source restrictions. The existing trusted and validating walkers may -remain distinct; a configurable reflection or ownership framework is not needed. - -### Schema Copying Without An Incidental Resource Policy - -**Audit finding:** 7. - -Remove the additional schema-document depth/node rejection inherited from -`jsonvalue.Copy` during structured-output capture and prepared-details copying. -Schema detachment remains necessary; runtime provider-value budgets remain -unchanged. The decoder/compiler and the existing self-contained submission -checks continue to own schema acceptance. - -Retain source-specific schema loading, containment behavior, supported reference -and dialect handling, completeness checks, and outward mutation isolation. -Update tests and internal descriptions that currently sustain the copier-derived -schema rejection. A schema otherwise accepted by those owners should not fail -solely because an ownership copy uses runtime provider-value limits. - -This intentionally broadens acceptance of schemas rejected only by that copier. -It does not promise unlimited resources, stack safety, extra precision, provider -acceptance, or provider enforcement, and does not introduce a new schema budget. - -## Preserved Boundaries And Scope - -Follow [ADR 0001](../adr/0001-configuration-boundary.md): ConfigKit owns -composition up to its final handoff; PromptKit interprets final domain values. -Configuration history remains an explicit operator-inspection capability, not -an execution validation phase. - -The audit's retained-machinery analysis remains applicable. In particular, -preserve the public/internal facade and needed ownership copies, artifact-reader -defensive copying, separate admission and active permits, prepared-handle -lifecycle, template request isolation, schema root-source handling, HTTP -evidence/deadline behavior, and operation-owner error classification. - -This work does not require dependency upgrades, upstream publication, new public -extension points, changes to supported template syntax, new retries or limits, -capacity-manager redesign, or release-tooling changes. Equivalent private names -and package-local organization remain implementation choices. Additional -contract changes should be surfaced rather than inferred from this cleanup. - -## Tests And Documentation - -Follow the [testing policy](../policy/testing.md). Keep meaningful behavior -coverage and migrate fixtures to the surviving production paths. Remove tests -that exist only to sustain deleted accommodations; retain or consolidate their -useful wire, source, cancellation, ownership, and error-boundary protection. - -Use focused offline checks during implementation, race-enabled validation for -changed shared-state/ownership paths, and representative public preparation, -execution, repair, and diagnostic coverage. Reuse existing tests where they -protect the relevant behavior. Neither exact allocation assertions nor a new -benchmark or recurring release gate is required to establish this refactor. - -Update affected GoDoc and the [internal component overview](../internal/overview.md), -[model-client](../internal/llm.md), [runner](../internal/runner.md), and -[source/validation](../internal/sources.md) descriptions when behavior lands. -Consumer and format guidance should record the removed incidental schema -restriction where relevant, without implying new resource or provider promises. -Other public API/wire behavior remains governed by its existing canonical owners. - -## Completion - -Completion means findings 1–7 have their selected changes, finding 8's runtime -behavior is preserved, and relevant behavioral checks and canonical documentation -match the result. Caller review should confirm that retired private workflows -and duplicate state are actually gone rather than renamed or relocated. - -The audit's test results describe the original baseline, not evidence of the -replacement implementation. The implementation plan records the focused and -combined checks performed on the delivered changes. -No significant product decisions remain open. Release selection/publication and -retirement of the older completed audit documents are separate work from this -roadmap.