diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index e67a0bf..d60dbd0 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,916 +1,402 @@ -# Audit Remediation Implementation Plan - -## Scope And Execution Model - -Implement the selected end state in [audit-remediation.md](audit-remediation.md) -against production baseline `e85ea2d`. Its workstreams, accepted catalog -decisions, compatibility choices, and historical assessment evidence remain the -background authority. This plan owns implementation order and stage outcomes. -All stages are initially pending; all 34 catalog entries are included. - -Follow the stages in order as one end-to-end sprint. Each stage is sized as a -focused Luna coding-agent prompt. Intermediate stages may leave callers/tests -unmigrated or the repository temporarily unable to compile. Do not introduce -transitional adapters, duplicate implementations, extra public APIs, or separate -publication/review gates merely to make each intermediate stage coherent. -Record deferred checks and run them when the affected path becomes runnable; -the final state must compile, satisfy the selected behavior, and pass validation. - -Read [development guidance](../development.md) and its applicable policy/contract -documents. In particular follow agents, architecture, testing, documentation, -and [ADR 0001](../adr/0001-configuration-boundary.md). Use the graph tools for -code discovery and inspect actual callers before changing signatures/deleting -helpers. Preserve unrelated work. Follow the local commit policy for completed -units, reporting temporary incompleteness accurately. This sprint does not -authorize pushing, tagging, publishing, or selecting a release version. - -### Shared Implementation Rules - -- ConfigKit owns composition, immediate Mapping capture, and raw-source errors. - PromptKit interprets/validates the final handoff. Operator provenance remains - available but is not analyzed during execution or used as another validator. -- Preserve public contracts except the accepted artifact ownership, - self-contained schema, and convenience-source value changes. Retain runtime - override support, template recognition scope, status-only retries, capacity - policy, and privacy. Definition hashes may change as documented opaque values. -- Private names below describe selected seams, not new public compatibility - promises. Keep the public facade thin and internal dependencies one-way. -- Tests protect meaningful behavior, not helper structure, exact allocation - counts, old hash encodings, incidental independent-error ordering, or complete - diagnostic text. Use existing real internal collaborators and offline external - doubles. Do not duplicate ConfigKit's raw-source validation suite. -- Include the owning GoDoc/documentation with its behavior change. The final - documentation stage reconciles omissions; it is not a reason to describe - incomplete behavior as implemented earlier. -- Stage checks below are focused development checks, not new recurring release - gates. Reuse successful checks on unchanged content. Do not run the complete - suite after every stage. No provider calls, credentials, external publication, - generalized benchmark framework, or human approvals are required between stages. - -## Stage 1: Select The Maintainer Toolchain - -**Catalog:** 1. **Scope:** environment selection and contributor guidance only. - -1. Inspect the current tree/toolchain selection. Put the newest already available - local Go binary directory first on PATH and set `GOTOOLCHAIN=local` for the - sprint. The assessment found Go 1.27.3 in the module toolchain cache; use it - unless a newer local installation is now available. Set `GOWORK=off` for - repository-isolated validation, without prohibiting consumer workspaces. -2. Record `go version`. Keep `go 1.27.0` unchanged for the assessed series/patch; - follow existing release policy if a genuinely newer series is selected. - Do not raise the consumer minimum merely for a validation patch. -3. Clarify ordinary development selection in `docs/development.md`, linking to - the existing release/toolchain policy. Reuse the roadmap's focused advisory - reassessment unless changed evidence makes another check necessary. - -**Validation/outcome:** The selected toolchain runs and its version is recorded. -No new installation, vulnerability gate, consumer advisory, or dependency update -is needed for the assessed environment. - -Completion: selected the locally available Go 1.27.3 toolchain (`go version -go1.27.3 linux/amd64`) with `GOTOOLCHAIN=local` and `GOWORK=off` for isolated -validation. The module minimum remains Go 1.27.0; contributor guidance now -describes local toolchain selection. - -## Stage 2: Remove Convenience Source Conversion - -**Catalog:** 3, 5. **Files:** `profiles.go`, `backends.go`, `provider_values.go`, -source-helper GoDoc and source/construction contract tests. - -1. Assign `config.ExtraParams` directly into each helper's ordinary fields map; - let the existing final `configkit.Mapping` call capture it immediately. - Preserve optional-field projection, ordering, and the `[]any` record wrapper. -2. Remove `providerMapForMapping`, its recursive converter, number placeholder, - and the resulting unused file/imports. Do not replace them with validation, - memoization, precision checking, or another clone. -3. Exercise both helpers through engine loading with supported nested built-ins, - mutation after source creation, and representative formerly converted types - and cyclic values that ConfigKit reports when the source is consumed. -4. Explain ConfigKit-supported forms and caller-selected numeric conversion in - GoDoc/consumer guidance. Runtime overrides retain their separate value rules. - -**Validation/outcome:** Focused source/construction tests establish immediate -capture and deferred upstream diagnostics, not acceptance of `json.Number`, -typed containers, named scalars, or nested pointers by the removed adapter. - -Completion: convenience sources pass provider maps directly to ConfigKit; -the converter was removed. Focused source and construction tests passed with -Go 1.27.3, including nested capture and deferred unsupported/cyclic errors. - -## Stage 3: Represent The Owning Error Operation - -**Catalog:** 7, first part. **Files:** use-case error/runner/preparation boundaries. - -1. Introduce an internal `OperationError` containing `Category error` and - `Cause error`. Implement ordinary contextual formatting, `Unwrap` returning - the cause, and `Is` matching the owning category; retain safe existing cause - formatting rather than adding raw request/provider data. -2. Use it at the artifact, render, generation, validation, and resource lookup - boundaries that currently add operation sentinels. Select genuine prompt/ - profile not-found categories at their owning lookup boundary, distinguishing - them from other repository load failures. Retain existing request and - credential classifications and nested cause identity. For generation request - construction, use the direct preparation wrapper introduced in Stage 6, - not a whole-chain match that can mistake a child client's error for this - operation's preparation failure. -3. Expose this type only internally to facade translation. Do not add a public - error taxonomy or infer the owner from the deepest matching sentinel. - -**Validation/outcome:** Focused internal cases establish the explicit outer -category and matchable cause. Finish facade behavior in Stage 4; do not add -temporary mapping paths just for this intermediate stage. - -Completion: operation errors retain their owning category and original cause, -including direct prompt/profile lookup categories. Focused use-case tests pass -on Go 1.27.3. Public projection and built-in preparation classification remain -for their respective later units. - -## Stage 4: Preserve Categories Through Public Error Projection - -**Catalog:** 7, completion. **Files:** `errors.go` and public error contracts/tests. - -1. Select the outer operation category before searching arbitrary nested causes - for public categories or capacity/provider/retry-exhaustion projections. - Keep the mapping of unclassified existing errors where still applicable. -2. Preserve specialized public error types and their fields, redaction, and - retry counts. Project them without losing the owning category or original - injected cause; use wrapping/joining as needed, not category exclusivity. - A child's existing public error remains matchable without becoming the - parent's sole classification. -3. Add public nested-engine regressions: a valid parent client and artifact - reader each return a child missing-prompt error. Check the parent category, - exact child cause, and legitimate direct not-found behavior. Include - representative specialized error composition so the earlier early-return - branches cannot bypass the owner. - -**Validation/outcome:** Root error tests and relevant use-case/generation tests -preserve `errors.Is`/`errors.As`, existing capacity metadata, exhaustion and -provider projections, and cancellation identity. - -Completion: public projection preserves the owning operation alongside nested -causes and specialized projections. Root and generation tests pass on Go 1.27.3; -two use-case preparation-classification tests await the direct preparation -marker specified below. - -## Stage 5: Retain Successful-Response Read Failures - -**Catalog:** 8. **Files:** `internal/llm/openai_compatible_client.go` and focused -HTTP/public smoke tests. - -1. Record actual non-EOF errors from the response reader beneath the existing - limited reader. Keep this a small private wrapper, not a new reader framework. -2. At initial decoding and final EOF verification, retain observed context - precedence and the existing size-overflow classification; otherwise wrap an - actual reader failure using redacted `requestFailedError` so its cause remains - inspectable. Distinguish reader-returned errors from parser-created errors. -3. Preserve malformed JSON/type/framing errors, EOF-truncated JSON, extra values, - and size overflow as malformed response. Do not add transport retries. -4. Cover interrupted initial decoding and complete JSON followed by a failing - read, including a reader returning bytes and an error together. Check - transport cause/redaction and diagnostic failure classification. - -**Validation/outcome:** Relevant llm tests plus focused public execution/smoke -cases establish both failure locations and unchanged genuine malformed behavior. - -Completion: successful-response body read failures retain their redacted -transport cause, including errors returned with bytes. The llm suite and focused -public execution and smoke classification tests pass on Go 1.27.3. - -## Stage 6: Mark Built-In Diagnostic Preparation Failure - -**Catalog:** 9. **Files:** generation evidence/coordinator and smoke orchestration. - -1. Add internal `GenerationEvidence.PreparationFailed bool`. Set it only when - built-in `PrepareGeneration` fails before attempts; do not expose it publicly. - Wrap that same return in an internal generation `PreparationError` with its - cause so ordinary `Generate` also carries direct preparation provenance. - `wrapGenerationError` recognizes that direct wrapper, not a nested child's - sentinel/type, to select invalid-request rather than generation failure. -2. After caller-cancellation handling, let inference diagnostics use that marker - to return an error matching use-case/public `ErrInvalidRequest`, preserving - the preparation cause instead of publishing a completed failure report. -3. Preserve injected-client errors and intentional completed no-dispatch reports - for missing required credentials/capacity, unsupported and inapplicable work. - Neither sentinel matching alone nor an empty attempt slice identifies stage. - -**Validation/outcome:** A reserved `Execution.ExtraParams["model"]` override -fails with the public invalid-request category and zero dispatch. Existing -provider rejection, missing-key, capacity, injected, and cancellation tests -retain their respective outcomes. - -Completion: built-in preparation failures carry direct error and evidence -markers; diagnostics return invalid-request errors before dispatch. Generation, -use-case, and root tests pass on Go 1.27.3, including the previously deferred -injected-client classification cases. - -## Stage 7: Bound Authentication Recognition And Report Overflow - -**Catalog:** 2. **Files:** current authentication probe, public/domain smoke reasons. - -1. Define a named internal 1 MiB allowance and read no more than allowance plus - one byte for HTTP 200 account recognition. Enforce actual bytes irrespective - of declared length; detect overflow before JSON parsing and close without - draining. Preserve the body-independent handling of 401 and other statuses. -2. Add matching public/domain - `SmokeTestReasonAuthenticationResponseLimitExceeded` constants with value - `authentication_response_limit_exceeded`. Report overflow with existing - `ExecutionFailureOther`, unknown acceptance, available HTTP/timing evidence, - and a completed report with nil Go error unless caller cancellation applies. -3. Preserve privacy, anonymous probing, and ordinary observer isolation. No new - report Boolean, failure enum, public budget knob, or inference limit is added. - -**Validation/outcome:** Focused authentication/public tests cover exact limit, -limit plus one, missing/inaccurate lengths, malformed bounded bodies, closure, -401 without reading, cancellation, and redacted explicit overflow. Keep this -fix independent of the later protocol relocation. - -Completion: authentication recognition reads at most 1 MiB plus one byte and -reports a distinct overflow reason without asserting credential rejection. -Focused authentication and public smoke tests pass on Go 1.27.3. - -## Stage 8: Correct Artifact Result Ownership - -**Catalog:** 4. **Files:** public ArtifactReader GoDoc, adapter and ownership tests. - -1. State that successful `Read` transfers the returned artifact and mutable body - storage to PromptKit. The reader stops mutating it through retained aliases; - private buffers may be reused only when the returned storage is independent. -2. Keep the signature and defensive adapter copy. Update any fixtures relying - on immediate reuse of returned mutable buffers to honor the selected contract. -3. Add focused concurrent preparation coverage using independently returned - storage; retain adapter mutation isolation and cause propagation. - -**Validation/outcome:** Race-enabled focused artifact/preparation tests follow -the accepted handoff, without promising safety for contract-violating readers. -Explain migration in consumer guidance; no borrowing/release protocol is added. - -Completion: ArtifactReader ownership and migration guidance require independent -transferred storage. Focused concurrent preparation, adapter isolation, and -cause-propagation tests pass with the race detector on Go 1.27.3. - -## Stage 9: Detach Prepared Ownership And Session Storage - -**Catalog:** 16, 18. **Files:** runner/prepared state and domain session normalization. - -1. Allocate one private non-zero-sized identity object per runner construction. - Store its immutable pointer in prepared handles instead of `*Runner` and - compare tokens for owner validation. Namespace views keep the same runner. -2. Keep the token after use/discard; preserve mutex-protected ready/claimed/ - discarded transitions, payload clearing, foreign-owner check order, and - independent details. Do not create mutable owner-field synchronization work. -3. Clone each accepted nonempty normalized session string with `strings.Clone` - after existing validation. Do not cap or otherwise reinterpret the raw input. - -**Validation/outcome:** Existing and focused race tests cover foreign engines, -same-engine views, one-shot/concurrent claim/discard and details after use. -Session Unicode/trim/content/wire/hash tests stay unchanged in meaning. Reference -analysis, not a GC-timing assertion, establishes that the token retains no runner. - -Completion: prepared handles retain only a private nonzero-sized runner identity -token, including after use or discard; accepted session strings own their -storage. Focused prepared/session race tests pass on Go 1.27.3. - -## Stage 10: Declare Validation Preparation As The Internal Dependency - -**Catalog:** 21, interface part. **Files:** validator constructors, engine Options, -runner constructors, and their test doubles. - -1. Store/accept `validate.ValidationPreparer` where engine/runner execution needs - preparation. Remove the runtime assertion that discovered this requirement. -2. Return concrete validator pointers from internal constructors so direct - validation callers still have `Validate`. Retain the separate direct - `Validator` interface where actually consumed; do not expose a new public API. -3. Migrate applicable doubles to the declared interface. Replace the former - validation-only runtime-rejection test with meaningful preparation/nil-plan - behavior. Preserve nil-validator no-op and nil-plan error handling. - -**Validation/outcome:** Inventory affected call sites and migrate them without -compatibility adapters. Run applicable validation/preparation tests when this -signature change is wired; temporary compilation gaps may continue into Stage 11. - -Completion: runner and engine validation dependencies now require preparation; -concrete constructors retain direct validation, and nil-plan behavior is tested. -Validation, use-case, and engine tests pass on Go 1.27.3. - -## Stage 11: Share Root Loading And Schema Compilation - -**Catalog:** 6, 21, compilation part. **Files:** `internal/validate` implementations. - -1. Keep host-file and `fs.FS` root resolution/loading adapters, including contained - paths, single-file sources, escaped resource URLs, context checks, exact JSON - numbers, one-document decoding, and existing root dialect validation. -2. Share `compileSchemaDocument(ctx, resourceURL, document)` for compiler creation, - Draft2020 selection, AddResource, contextual errors, and compilation. Install - a context-aware loader that returns an actionable external-resource error - without reading any other file or network resource. Keep bundled metaschemas - and compiler URI/ID/anchor resolution intact. -3. Route both prepared and both direct schema paths through it. Prepared plans - retain the root and executor; direct validation reaches compilation only - after decoding the output instance. Remove obsolete external loaders rather - than leaving an alternate external-loading path. - -**Validation/outcome:** Retain independent root-source, cancellation, precision, -framing, containment, and freezing cases. External-reference successes become -expected errors. Root compilation alone is intentionally not the completed -self-containment remedy; Stage 12 checks otherwise unused schema declarations. - -Completion: host and filesystem validation share root-document compilation with -a context-aware external-resource rejection loader. Focused validation tests -pass on Go 1.27.3; checking unused declarations remains in the following unit. - -## Stage 12: Check Declared Schema Locations Through The Compiler - -**Catalog:** 6, 21, resource-policy completion. **Scope:** schema-location traversal. - -1. Add one private traversal of schema-bearing locations in the decoded document. - Compile the root and each located schema using the same compiler/cache and - retrieval URL plus escaped JSON-pointer fragment. Preserve the compiled root - as the executor used for actual validation; do not rewrite/bundle the document. -2. Use each compiled schema's exported `DraftVersion` to select the applicable - pinned-dependency location table. This delegates embedded-resource dialect - resolution to the compiler rather than assuming the root dialect everywhere. - Visit object/Boolean schemas at the applicable locations, not property-name - dependency arrays or arbitrary example/annotation objects. -3. Match jsonschema 6.0.2 `draft.go` positions: Draft4 includes `definitions/*`, - `not`, `allOf/[]`, `anyOf/[]`, `oneOf/[]`, `properties/*`, - `additionalProperties`, `patternProperties/*`, `items`, `items/[]`, - `additionalItems`, and `dependencies/*`. Draft6 adds `propertyNames`/`contains`; - Draft7 adds `if`/`then`/`else`; Draft2019 adds `$defs/*`, - `dependentSchemas/*`, `unevaluatedProperties`, `unevaluatedItems`, - `contentSchema`; Draft2020 adds `prefixItems/[]`. Use the dependency's inherited - positions without broadening its dialect acceptance. Document this pinned - relationship locally; no new public dialect matrix is introduced. -4. Escape pointer segments (`~`, `/`) and URI fragments correctly. Check context - around traversal/compiler calls. Do not implement a parallel URI resolver, - lexical reference ban, generic schema engine, or new schema-size restriction. - -**Validation/outcome:** An unused `$defs` external reference fails, while literal -`$ref` keys in examples/default data do not become schemas. Cover fragment and -anchor references, explicit/embedded IDs, recursive/dynamic references, escaped -keys, and embedded-resource dialect distinctions. Keep independent valid/invalid -instances; do not manufacture expectations with the traversal under test. - -Completion: declared schema locations are compiled using the pinned compiler's -dialect-resolved schema positions and a shared cache. Focused validation tests -cover unused external references, annotations, escaped pointers, anchors, -recursive/dynamic references, embedded dialects, and independent instances. - -## Stage 13: Verify Public Schema Wiring And Native Submission - -**Catalog:** 6, 15, 21; schema-file half of 32. - -1. Add focused public `WithSchemaFile` construction and Prepare/Validate cases - with real temporary files, last-valid-option/source replacement, and relevant - missing-file/directory errors. Keep source-specific internal tests independent. -2. Add offline wire coverage that a self-contained schema, including internal - definitions, reaches native structured output without external file loading. - External references fail before dispatch; prepared validation remains frozen - after source mutation/removal. Do not assert provider-side enforcement. -3. Reduce private `fileSource` to `(fs.FS, string, error)` and its caller to the - used results. Retain the actual directory calculation, checks, and DirFS. -4. Align schema/consumer/internal documentation with self-contained resources, - the remaining root-source options, and direct/prepared behavior. - -**Validation/outcome:** Focused public preparation/validation/wire and internal -validate/engine tests finish the schema transition. No local-only mode, bundler, -new dialect, or release/publication prerequisite is introduced. - -Completion: schema-file option, frozen preparation, native submission, and -external-reference rejection are covered through public APIs. Root, validation, -and engine tests pass on Go 1.27.3; current schema guidance is aligned. - -## Stage 14: Merge Provider Maps Into One Owned Lower Copy - -**Catalog:** 19. **Files:** configuration targets and focused overlay tests. - -1. Copy the lower map once with the existing compatible ownership boundary. - Recurse into that owned result for overlapping built-in maps rather than - calling the full cloning entry point for each descendant. -2. Copy upper replacement values where needed; retain null/array replacement, - empty-map behavior, accepted types, errors, and isolation. Do not merge runtime - override maps or introduce a new combined-map size restriction. -3. Keep the merge policy specific to backend defaults plus profile values, - separate from ConfigKit composition/inheritance. - -**Validation/outcome:** Focused configuration cases cover recursive overlap, -disjoint keys, nulls, arrays, and input mutation. Repeat the roadmap's chain/ -width/disjoint baseline after implementation and record actual allocations/work; -do not assert exact benchmark thresholds as tests. - -Completion: overlapping maps now merge into one owned lower copy; focused -configuration tests pass. On Go 1.27.3, the documented 50-iteration, three-run -workloads measured: depth/width 2/2 overlap 5,144 B and 131 allocations; 12/2 -overlap 19,952 B and 464; 40/2 overlap 72,962 B and 1,392; 4/16 overlap -29,608 B and 765; 12/2 disjoint 32,496 B and 686. These are measurements, -not performance guarantees. - -## Stage 15: Centralize Trusted JSON Clone Ownership - -**Catalog:** 24. **Files:** `convert.go`, `internal/jsonvalue`, projection tests. - -1. Provide internal `CloneTrusted(any) any` for detached copies of already accepted - values. Route facade nested-value projection through this owner and remove - the facade's reflection traversal responsibility. -2. Preserve its existing concrete scalar/container/pointer/nil and fast-path - behavior, exact numbers, schema empty keys, and caller ownership. Audit callers - before changing helper signatures. Do not normalize these values as fresh input. -3. Keep fallible bounded `Copy`/`CopyMap` acceptance policies distinct. Share only - matching primitives; two small walkers are preferable to a configurable - reflection framework. No panic, rejection, silent zero value, or swallowed - copy error is added to an infallible projection. - -**Validation/outcome:** Relevant JSON-value and facade client/prepared/inspection -mutation tests use independently specified expectations. Include distinguishing -nil/type/schema-key cases; source conversion from Stage 2 must not reappear here. - -Completion: trusted nested-value cloning is owned by internal/jsonvalue while -bounded input copying remains separate. Focused type, nil, schema-key, client, -prepared-state, and inspection tests pass on Go 1.27.3. - -## Stage 16: Stop Recopying Growing Condition Evidence - -**Catalog:** 10. **Files:** prompt requirements evaluation and focused tests. - -1. Treat intermediate missing-variable/content-input sets as owned by their - evaluation result. Transfer nil sets and insert the smaller set's members - into the larger owned set instead of copying both into a new map. -2. Preserve unknown/unsupported/input-name flags and later operand value/truth - checks; verify no caller retains an independent set snapshot that is mutated. -3. Add representative left-heavy conditions and set/flag outcomes, keeping sorted - deduplicated results, partial discovery, and cancellation intact. - -**Validation/outcome:** Requirements tests protect classifications and independent -rendering comparisons. Source work counts or focused measurements demonstrate -removal of the identified growing-copy pattern; no grammar or expression limit -changes, universal complexity promise, or allocation-count assertion is added. - -Completion: owned condition sets merge smaller into larger without retaining -operand snapshots. For the documented distinct left-heavy cases, source work is -3, 15, and 127 insertions at N=4, 16, and 128, respectively, excluding leaf -creation (previously 9, 135, and 8,255). Prompt requirements tests pass on Go -1.27.3; these counts are not a general complexity guarantee. - -## Stage 17: Share Requirement Branch Orchestration - -**Catalog:** 23. **Files:** internal prompt requirement traversal/evaluation. - -1. Factor the common `parse.BranchNode` sequence for If/With: inactive traversal, - condition evaluation, then/else reachability, dependency-delta comparison, - and limitation recording. Handle With's changed body dot and unchanged else - dot explicitly; do not make the scopes interchangeable. -2. Keep recognition/evaluation and evidence/result responsibilities separately - readable, sharing recognized command-shape helpers only where they genuinely - match. Preserve scope entry/exit around named templates and skipped paths. -3. Do not combine this partial inspector with the real template executor or - introduce a generic AST visitor/interpreter framework. - -**Validation/outcome:** Existing node-specific tests and independent Go-renderer -comparisons retain known/skipped/unknown outcomes, inactive computed names, -optional index behavior, root access, and meaningful incompleteness. - -Completion: `if` and `with` share branch traversal while retaining distinct body -and else dot scopes. Focused internal and public requirements tests pass on Go -1.27.3, including existing independent rendering and skipped-path cases. - -## Stage 18: Add Bounded Diagnostic Prefix Encoding - -**Catalog:** 17, encoding primitive. **Files:** repair formatter private helpers/tests. - -1. Provide bounded normalization of the potentially useful prefix of one - diagnostic. Preserve `strings.ToValidUTF8` replacement behavior, including - runs of invalid bytes, without first normalizing the entire original string. - Report whether more normalized content exists so fitting is not guessed. -2. Use standard `json.Marshal` on bounded normalized strings and rune-safe - prefix selection with byte-budget accounting. Keep HTML/control escaping and - valid UTF-8 JSON. Do not build a general custom JSON escaper. -3. Retain only bounded candidate buffers; scanning a long invalid-byte run may - still be needed to preserve normalization. Do not invent a raw input limit - or promise constant CPU regardless of input. - -**Validation/outcome:** Focused helper behavior covers plain/escaped Unicode, -invalid-byte runs, empty/small budgets, and truncation boundaries. Wiring into -the public repair workflow follows in Stage 19; no interim alternative formatter -is exposed or new contract created for a helper's exact allocation count. - -Completion: bounded UTF-8 normalization and JSON-prefix encoding retain only -the useful candidate prefix and report complete versus truncated results. -Focused encoding and existing formatter tests pass on Go 1.27.3; formatter -wiring remains in the next unit. - -## Stage 19: Construct Repair Feedback Without Whole-Input Serialization - -**Catalog:** 17, completion. **Files:** `formatRepairDiagnostics` and repair tests. - -1. Attempt construction of the complete-list result within the existing 64 KiB - budget, processing only useful bounded prefixes. Preserve the complete-list - form when it fits, including existing empty/nil-list behavior. -2. If it cannot fit, build the omission form using the existing marker, ordering, - separator/reservation rules, and bounded prefix helpers from Stage 18. Do not - marshal the entire error list or an unbounded entry before deciding to truncate. -3. Leave full validation diagnostics and previous candidate stored as before; - only the feedback representation is bounded. Preserve original repair-message - ordering, attempts, provider-native structured output, and generation policy. - -**Validation/outcome:** Cover fitting/exact-boundary, oversized single entry, -many entries, heavily escaped and invalid strings, valid JSON, and omission -semantics. Repeat the roadmap's three formatter workloads and record real -allocation results; aggregate allocation is not a peak-heap guarantee. - -Completion: repair feedback uses bounded prefixes for both complete and -omission forms. Focused tests pass on Go 1.27.3. The documented 20-iteration, -three-run workloads measured 398 B/op for 64 ASCII bytes (68 output bytes), -767,634 B/op for 1 MiB ASCII (65,536 output bytes), and about 12,125,700 B/op -for 1 MiB escaped content (65,531 output bytes). These aggregate allocations -do not establish a peak-heap guarantee. - -## Stage 20: Select Namespace Ranges Before Copying Inventories - -**Catalog:** 20. **Files:** configuration/engine inventory interfaces and listing tests. - -1. Resolve filters/view scope in the engine first, then pass private - `configuration.IdentityFilter{All bool, Namespace string}` to the inventory. - All true selects all IDs; false plus empty namespace selects unqualified IDs; - false plus a name selects that exact namespace. -2. In captured sorted IDs, find a named prefix's start/end by binary search and - copy only that contiguous range. For All, copy once; for unqualified IDs, - scan immutable state directly and append matches without a prior whole copy. -3. Apply the same policy to profiles, backends, and prompt identities; migrate - private fakes. Preserve caller-owned results, lexical order, context checks, - empty success, namespace defaults, and explicit-filter precedence. - -**Validation/outcome:** Test prefix-sharing namespaces, interleaved unqualified -IDs, All/default/views, missing matches, mutation, and cancellation. Repeat the -baseline through the new select-before-copy path. No persistent buckets/cache, -new public signature, or unqualified-query complexity promise is required. - -Completion: namespace selection now precedes copying for all three inventories; -focused listing and ownership tests pass on Go 1.27.3. The documented 100-iteration, -three-run engine workload measured the following bytes/op, in All / sparse / -dense / absent order (one allocation except absent, which used none): - -| Total IDs | Bytes/op | -| --- | --- | -| 32 | 512 / 16 / 256 / 0 | -| 1,024 | 18,432 / 16 / 9,472 / 0 | -| 4,096 | 65,536 / 16 / 32,768 / 0 | - -The baseline included full-inventory copying and subsequent filtering; these -measurements are observations, not performance guarantees. - -## Stage 21: Put Authentication Protocol Mechanics In The Outbound Adapter - -**Catalog:** 26. **Files:** llm authentication adapter, selection, assembly and tests. - -1. Add an internal llm OpenRouter-key operation taking context, resolved endpoint, - timeout/settings, and returning HTTP evidence, optional acceptance, - overflow/unrecognized indicators, and operational error. It owns authenticated - GET, bounded body consumption/closure, and account-response recognition. -2. Move default `/key` endpoint derivation to an llm helper used by selection; - keep configured overrides and userinfo sanitization behavior. It is a focused - provider adapter, not a plugin registry or dependency on use-case types. -3. Supply this operation from normal engine assembly through a narrow - authentication capability alongside reachability HTTP capability. Update - relevant test doubles without expanding public injection APIs. - -**Validation/outcome:** Adapter tests use controlled HTTP for recognized/unknown -200, 401, other statuses, overflow, interruption, anonymous credentials, default/ -override endpoints, and privacy. Stage 7's bound remains in the actual path; -Stage 22 finishes use-case cleanup and report projection. - -Completion: the outbound adapter owns OpenRouter endpoint derivation, bounded -authentication requests, and account recognition. With maintainer approval, -orchestration was routed through it here to avoid duplicate implementations; -the report-copy cleanup remains in the next unit. Focused llm, use-case, engine, -and public smoke tests pass on Go 1.27.3. - -## Stage 22: Narrow Diagnostic Orchestration And Copies - -**Catalog:** 26, 27, completion. - -1. Route authentication orchestration through Stage 21's capability and remove - the use-case parser/body-consumption/default-protocol code. Keep support, - applicability, selection, completion, context and reason/failure mapping in - use cases. Preserve independently known HTTP/acceptance facts. -2. Replace `smokeReportTarget`'s empty-full-report clone with direct target - detachment and existing JSON-value provider-map ownership/error handling. - Retire unused request/full-report cloners and clone-specific helpers/tests - only after checking their callers. -3. Keep facade request/report projection and real backend metadata cloning. Do - not add an internal clone followed by another whole-report projection copy. - -**Validation/outcome:** Focused smoke/adapter/facade ownership tests establish -unchanged tri-state facts, neutral outcomes, no ordinary observer events, target -redaction, and independent provider maps. Update owning internal documentation. - -Completion: diagnostic targets detach provider values directly, and unused -request/report cloning helpers and their dedicated tests have been removed. -Authentication routing was completed in the preceding unit. Focused domain, -adapter, smoke, and facade tests pass on Go 1.27.3; internal adapter ownership -documentation is updated. - -## Stage 23: Extract Final-Configuration Record Validators - -**Catalog:** 22, validation part. **Files:** configuration validation and tests. - -1. Keep `validateEffective`'s resource/index traversal visible; extract coherent - backend/profile/prompt record checks with explicit record, path, and required - index arguments. Keep capacity arithmetic, reference validation, and domain - validation in their appropriate record operations. -2. Preserve final-handoff-only enforcement, recorded field presence/nulls, - normalization, source context, and meaningful error identities. Do not inspect - overwritten sources/history or create another resolution/validation phase. -3. Do not pin incidental ordering of independent validation failures or introduce - reflection-driven field descriptors merely to reduce function size. - -**Validation/outcome:** Existing configuration validation/inspection cases and -focused changed-seam cases retain final values, paths and relevant error meaning. -This stage changes organization, not configuration acceptance policy. - -Completion: final-handoff traversal retains its resource index and delegates -backend, profile, and prompt records to focused validators. Configuration tests -and focused public configuration, validation, inspection, and source tests pass -on Go 1.27.3; the extracted checks preserve their existing paths and rules. - -## Stage 24: Separate Prompt Capture And Effective Target Construction - -**Catalog:** 22, completion. **Files:** configuration prompts/targets and tests. - -1. Extract prompt input/metadata projection, message content/template capture, - and output-contract conversion into focused operations returning the values - needed by materialization. Retain lazy origin lookup and source-relative - `content_file` access, then one final definition/inspection/hash assembly. -2. Extract coherent backend/profile target construction as useful, keeping - presence interpretation together and explicit. Preserve fieldAbsent/null/ - value distinctions and Stage 14's selected overlay ownership. -3. Preserve fail-fast resource errors where meaningful; do not force every - materialization check into the previous validation stage or aggregate errors - that were previously operational/fail-fast. - -**Validation/outcome:** Configuration capture, inheritance, origins, templates, -input metadata, inspection, and execution-selection tests keep their behavior. -Update focused source/internal descriptions without duplicating format policy. - -Completion: prompt inputs, messages, and output contracts are captured separately -before final definition assembly, and backend/profile construction is isolated -with presence handling intact. Configuration, prompt, engine, and focused public -tests pass on Go 1.27.3; the internal source description is updated. - -## Stage 25: Replace Parallel Attempt Returns With A Result Value - -**Catalog:** 25, attempt boundary. **Files:** generation coordinator and tests. - -1. Define private `attemptResult` containing `Outcome llm.AttemptOutcome`, - `Dispatched bool`, `CapacityWait`, `RequestDuration`, and `StartedAt`. - Return it from built-in attempt execution instead of five parallel values. -2. Migrate consumers without changing acquire/dispatch/context transitions, - response retention before cancellation, start timestamps, duration meanings, - or evidence capture. Keep permit release deferred inside the attempt scope. -3. Leave retry eligibility/budgets/waits and injected dispatch strategy explicit. - Stage 6's preparation failure remains outside attempt accounting. - -**Validation/outcome:** Focused generation tests retain actual permit lifetime, -response/error behavior, timing facts and cancellation; no new event or HTTP fact -is fabricated for an injected client. - -Completion: built-in attempts return a private result containing outcome, -dispatch, capacity, duration, and start-time facts. Generation, use-case, and -engine tests pass on Go 1.27.3; permit lifetime and injected dispatch are -unchanged. - -## Stage 26: Share Generation Accounting Projections - -**Catalog:** 25, completion. **Files:** generation accounting/evidence/observer tests. - -1. Extract common usage accumulation and evidence/event projection operations - around the result boundary. Keep distinct built-in and injected paths where - dispatch, HTTP evidence, or usage-presence compatibility actually differ. -2. Keep the retry loop about eligibility, budget, delay and terminal outcomes; - preserve known/unknown/complete usage, private evidence and generation numbers. -3. Ensure permits release before observers and backoff, including panic unwinding. - Do not add another permit layer, retry framework, transport retry, or helper - that invokes observers while the attempt still holds capacity. - -**Validation/outcome:** Existing observation, retry, diagnostic evidence, -injected usage compatibility and panic/cancellation cases exercise the real -coordinator. Expectations remain independent of the extracted projection helpers. - -Completion: built-in and injected attempts share evidence, usage, and observer -projection after their permit scopes end, while retaining their distinct HTTP -and usage facts. Race-enabled generation, use-case, and observation tests and -focused public retry, smoke, capacity, usage, and observation tests pass on -Go 1.27.3; an injected-observer test verifies permit release before callbacks. - -## Stage 27: Cover Request Privacy And Correct Public Documentation - -**Catalog:** 11, 12, 13, 14. **Scope:** focused tests/comments, not new behavior. - -1. Add marker-based RunRequest privacy formatting tests for values/pointers, - `%v`, `%+v`, `%#v`, String/GoString and direct-key JSON omission. Put markers - in omitted variables/bodies/settings, not IDs intentionally in summaries. -2. Correct credential-inspection GoDoc/example, recursive configured-map versus - runtime-replacement wire prose, and stale trimmed-ID/prompt-Version comments. - Link canonical format policy; do not change current behavior to fit old prose. - -**Validation/outcome:** Focused formatting/JSON tests protect the actual privacy -boundary, and changed documentation links and semantics agree with canonical -GoDoc/formats. Do not snapshot whole diagnostic strings or unrelated JSON fields. - -Completion: marker-based request formatting and JSON checks cover value and -pointer forms. Profile credential inspection, exact IDs, prompt listing, and -provider-map precedence prose now match the implemented contracts. Focused -privacy, profile, inheritance, and provider tests pass on Go 1.27.3. - -## Stage 28: Give Bundled Catalog Expectations One Test Owner - -**Catalog:** 28. **Scope:** catalog tests, not bundled asset changes. - -1. Keep the complete bundled inventory/model/plugin/default oracle in - `internal/configuration/bundled_test.go`, with structural plugin assertions. - Public option tests own toggles/order/isolation/overrides. Retain small - controlled model/plugin wire and configured-default smoke tests instead of - duplicated complete catalog matrices. Do not derive expectations from loading. -2. Remove duplicated full inventories and model/default matrices from other - layers while preserving their distinct public option, routing, serialization, - and diagnostic-selection assertions. Keep independent expected asset values - in the owning test rather than deriving them from the loader under test. - -**Validation/outcome:** Focused internal bundle and public option/wire/smoke tests -retain assembled behavior with a single complete maintained-asset oracle. - -Completion: the configuration bundle test owns the complete inventory, model, -default, and structural plugin expectations. Public tests retain representative -option, composition, wire, and configured-default smoke behavior without a -second catalog matrix. Focused bundle tests pass on Go 1.27.3. - -## Stage 29: Test Artifact Hashes As Opaque Values - -**Catalog:** 29. **Scope:** artifact tests and benchmark correctness setup. - -1. Replace literal SHA artifact assertions with same-body inline/file equality - and actual bytes/size checks. Prepare an opaque expected hash outside benchmark - timing; retain meaningful parity/read-boundary coverage. -2. Remove obsolete algorithm-specific test imports/calculations. Do not change - the production artifact hash or delete the cases that catch hashing only a - misreported-size prefix instead of the complete body. - -**Validation/outcome:** Artifact parity, size-hint and read-boundary cases remain -meaningful without choosing an opaque value's algorithm. Keep benchmark setup -outside timed work and do not add a new encoding guarantee. - -Completion: size-hint tests and benchmark correctness setup compare file hashes -to the same content read inline, without selecting a hash algorithm. Artifact -tests and one-iteration checks of both benchmark paths pass on Go 1.27.3. - -## Stage 30: Remove Unconsumed State And Orphan Helpers - -**Catalog:** 30, remaining half of 32, 33. - -1. Remove backend-only presence capture/map/field list and runtime - `QueueCapacitySet` bookkeeping. Keep profile/input field-state consumers, - effective queue values/defaults/explicit zero and ConfigKit provenance. -2. Remove the private reachability outcome endpoint echo; keep the real public - attempt endpoint and sanitization evidence. The discarded `fileSource` - directory result was removed in Stage 13; verify no callers were missed. -3. Remove internal `filecatalog.RelativePath`, `resourceNamespaceFromID`, their - now-irrelevant self-tests, and unused runner-test `stringPtr`. Confirm caller - searches against the current tree; do not broaden this into a deletion sweep. - -**Validation/outcome:** Focused queue/provenance, reachability/privacy, file -discovery and identity/listing behavior remain covered. Optional public helpers, -capability fallbacks, and embedded/release assets are not deletion candidates. - -Completion: unused backend presence and queue bookkeeping, the reachability -endpoint echo, and the named orphan helpers were removed after caller checks. -Queue, provenance, reachability, identity, discovery, and related public tests -pass on Go 1.27.3. The remaining file-source caller uses its three results. - -## Stage 31: Remove Redundant Captured Prompt Fields - -**Catalog:** 31. **Files:** domain prompt values, configuration capture, runner/fixtures. - -1. Remove duplicate runtime `PromptDefinition.OutputFormat` and the runner - fallback to it; validated `Validation.Format` is the captured source of truth. -2. Remove runtime `PromptMessageTemplate.ContentFile` only. Preserve the actual - configuration `content_file` field, source-relative loading, origins, and - already-captured message content/templates. Adjust fixtures accordingly. -3. Keep request replacement-contract empty Format defaulting to text. A fixture - constructing a malformed internal definition does not justify a production - compatibility fallback outside supported construction. -4. Account for changed marshaled definition hashes: preserve opaque equality - relationships between inspection/preparation and note cache/comparison impact - in migration guidance, without freezing old encodings or changing artifact hashes. - -**Validation/outcome:** Prompt capture/rendering, public inspection/preparation, -output overrides and hash-relation tests pass with independently specified -behavior. No assertion of bit-identical previous hash values is added. - -Completion: captured definitions use the validation contract's format and no -longer retain a redundant content-file path. Source-relative loading and -request override defaulting remain intact. Configuration, prompt, use-case, and -focused public tests pass on Go 1.27.3; migration guidance notes hash/cache impact. - -## Stage 32: Retire The Legacy Production Capacity Wrapper - -**Catalog:** 34. **Files:** runner scheduling test, capacity wrapper/tests/docs. - -1. Use `generation.NewInjected` with the same manager in the initial/repair - scheduling test so it exercises actual engine composition and shared limits. -2. Migrate useful capacity wrapper scenarios to direct manager tests or small - test-only one-shot scaffolding. Preserve FIFO, peak concurrency, canceled - waiter/grant races, independent/unlimited pools and panic release. -3. Delete production `internal/capacity/client.go` after callers migrate and - remove its retention prose. Keep the manager and coordinator's distinct owners; - never wrap the coordinator in a second acquisition layer. - -**Validation/outcome:** Race-enabled focused capacity/generation/use-case tests -exercise the real scheduling path. No production scheduling policy or public -API changes; no-repair runner construction and other deliberate fallbacks remain. - -Completion: initial and repair scheduling use the real injected coordinator; -the production capacity wrapper was removed. Capacity tests retain one-shot -test-only scaffolding for manager scenarios. Race-enabled capacity, generation, -and use-case tests pass on Go 1.27.3, and internal capacity guidance is updated. - -## Stage 33: Reconcile Documentation And Validate The End State - -**Catalog:** all entries; complete the sprint, not a release publication. - -1. Reconcile owning GoDoc, consumer guidance, framework formats, wire integration, - contributor guidance and focused internal documents with the final code. - Describe the artifact handoff, supported source forms, self-contained schemas, - explicit overflow reason and deliberate hash changes where consumers need them. - Remove obsolete external-loader/wrapper/current-behavior descriptions. -2. Record the migration summary and actual completion/evidence in the roadmap - for subsequent minor-release preparation. Do not select/tag/push a release, - change historical notes, or impose publication/network checks in this sprint. -3. Check the coverage table below against actual edits and test results. Do not - mark an optimization complete without its comparable post-change evidence, or - a behavior fixed solely because another refactor landed nearby. Name remaining - work accurately if new evidence prevents completion; do not silently defer it. -4. Format changed Go files and run the complete offline final checks with the - selected toolchain: `go test -race ./...`, `go vet ./...`, and - `git diff --check`. The default Go suite includes maintained examples; do not - run a duplicate standalone build/example/full non-race suite as another gate. - Check changed local Markdown targets/anchors and repository hygiene. Reuse - focused evidence for unchanged code; rerun affected checks after corrections. - -**Final outcome:** The repository compiles coherently, the selected remediations -and compatibility guidance are implemented, meaningful tests remain independent, -post-change measurements are recorded without new performance guarantees, and -no unapproved feature/policy/dependency changes or temporary harnesses are left -in the repository. Follow normal commit policy; release preparation remains separate. - -Completion: all catalog entries are covered by the implementation and focused -evidence recorded above, including comparable measurements for the selected -optimizations. With Go 1.27.3, `GOTOOLCHAIN=local`, and `GOWORK=off`, the full -offline `go test -race ./...` and `go vet ./...` passed. Tracked Go files are -formatted; the diff has no whitespace errors; local Markdown targets and headings -in changed documentation were checked. No benchmark overlays were retained in -the repository. Release preparation, version selection, tagging, and publication -remain separate work. - -Migration summary: successful artifact readers transfer returned mutable storage; -convenience sources use ConfigKit's supported value forms; JSON Schemas must be -self-contained; authentication overflow has an explicit diagnostic reason; and -opaque prompt definition hashes may change, requiring cross-version caches or -comparisons to be refreshed. See the [consumer guide](../consumers/pkg-promptkit.md), -[migration guidance](../consumers/configuration-migration.md), and -[format reference](../formats.md) for the applicable contracts. - -## Catalog Coverage - -| Catalog finding | Owning stages | -| --- | --- | -| 1 | 1 | -| 2 | 7 | -| 3, 5 | 2 | -| 4 | 8 | -| 6 | 11–13 | -| 7 | 3–4 | -| 8 | 5 | -| 9 | 6 | -| 10 | 16 | -| 11–14 | 27 | -| 15 | 13 | -| 16, 18 | 9 | -| 17 | 18–19 | -| 19 | 14 | -| 20 | 20 | -| 21 | 10–13 | -| 22 | 23–24 | -| 23 | 17 | -| 24 | 15 | -| 25 | 25–26 | -| 26 | 21–22 | -| 27 | 22 | -| 28 | 28 | -| 29 | 29 | -| 30 | 30 | -| 31 | 31 | -| 32 | 13, 30 | -| 33 | 30 | -| 34 | 32 | - -Stage 33 verifies the combined end state. No significant product questions -remain open; equivalent private implementation details remain flexible within -the selected designs and preservation requirements. +# Responsibility Simplification Implementation Plan + +## Scope And Execution + +Implement [responsibility-simplification.md](responsibility-simplification.md) +from baseline `c69703b`, using the accepted decisions and evidence in +[A1-responsibility.md](audit/A1-responsibility.md). Production code is unchanged +from the audit baseline `1ff6d77`. Findings 1–7 require changes; finding 8 requires +preserving current runtime behavior. All stages below are initially pending. + +This replaces the completed 33-stage audit-remediation plan. Historical links +from the older audit/remediation documents refer to that earlier plan, available +with `git show c69703b:docs/roadmap/implementation.md`, not to the stages below. +Do not repeat the earlier sprint or treat its completion evidence as validation +of these changes. + +Execute stages in order as one end-to-end sprint. Each stage is a focused Luna +prompt. Intermediate stages may leave callers/tests unmigrated or fail to +compile; record deferred checks and run them once their affected paths are +coherent. Do not add compatibility wrappers or duplicate implementations just +to keep an intermediate commit green. No human-review or external-publication +gate is needed between stages. Completion concerns the combined end state. + +Read [development guidance](../development.md) and its applicable policies, +especially agents, architecture, testing, and documentation, plus +[ADR 0001](../adr/0001-configuration-boundary.md). Use graph tools for discovery +and verify actual callers before deletion. Preserve unrelated work. Follow the +local commit policy and record completion/evidence for finished stages without +claiming temporarily deferred checks passed. + +### Shared Rules + +- Keep ConfigKit composition/history outside runtime validation. Validate final + configuration and genuinely new runtime inputs at their existing boundaries. +- Preserve the public facade and substantive API/wire behavior, except removal + of incidental schema-copy budgets selected in finding 7. The removed direct + output-validator workflow is private, not public `ValidateRequest`. +- Retain runtime supported values, concrete type/nil behavior, limits, number + handling, and errors. Do not narrow runtime values to ConfigKit mapping forms. +- Keep needed ownership copies, artifact defensive copying, prepared-handle + identity/lifecycle, admission versus active permits, HTTP evidence/deadlines, + and operation-owner errors. Do not expand this into another subsystem audit. +- Use existing trusted cloning for accepted values; keep validating copying for + actual acceptance. Do not add a reflection framework, schema budget, numeric + converter, raw-source validator, or public internal-type alias. +- Private names are implementation seams, not new public promises. Test selected + behavior, not allocation counts, helper choreography, or incidental wording. +- Include relevant GoDoc/internal documentation with the owning change. Stage 16 + reconciles omissions, rather than authorizing stale documentation until then. +- Select the newest locally available Go installation as documented, with + `GOTOOLCHAIN=local` and `GOWORK=off` for isolated checks. The audit used Go + 1.27.3. This sprint does not select a release, change dependency versions or + the consumer Go minimum, or authorize pushing/tagging/publication. + +## Ownership Inventory For Stages 10–15 + +Use these decisions, checking the affected callers as they are migrated. Record +any discovered mutation edge and account for it in the same boundary; do not +introduce a new ownership subsystem. + +| Value/path | Acceptance and mutation | Selected ownership treatment | +| --- | --- | --- | +| Configured provider values | Final configuration validation and effective-target construction own acceptance; retained configuration/index state must not be mutated by runtime work. | Preserve acceptance/overlay semantics. Registry/profile lookup may continue returning detached values, but their copies use trusted cloning, not repeated acceptance checks. | +| Runtime provider override | `toDomainExecutionTargetOverride`, also used by smoke request conversion, validates/copies caller values. | Keep that boundary and its errors. Subsequent internal selection may transfer accepted private values without another deep validation pass. | +| Resolved target within one operation | Captured lookup returns detached values; runtime override conversion supplies owned values. Resolution changes scalar fields and selects whole provider maps, not nested content. | Transfer selected private maps through preparation/inspection; avoid repeated shallow/deep copying merely between private functions. Preserve current nil/empty projection behavior. | +| Loaded schema | The validator decodes/compiles a self-contained JSON document and retains immutable plan state. | Trusted-detach the provider-facing schema without provider-value depth/node validation. Keep the frozen compiled plan and existing acceptance checks. | +| Prepared metadata | Messages/input hashes are created during preparation; selected maps and schema belong to that operation. | Retain one immutable redacted snapshot for execution and details. Clear execution-only fields without modifying metadata. | +| Public prepared/details/result/inspection values | Root conversion crosses into caller-owned mutable state. | Keep one outward deep projection at that boundary. Never expose a borrowed internal map, schema, message pointer, or slice. | +| Injected generation and repair | The internal runner/repairer reads private settings; repair constructs a new message sequence. The root client adapter supplies a mutable public request. | Keep public request detachment for every dispatch, including repairs, and existing response copying. Do not add copies to compensate for removed internal copies. | +| Smoke reports | Selection and internal orchestration use accepted private targets; public report conversion creates caller-owned facts/settings. | Keep target redaction/endpoint sanitization and final projection. Remove validation-only copies and redundant private detachment, retaining any copy needed by an actual mutator. | + +## Stage 1: Remove Client-Level Routing Fallbacks + +**Finding:** 1. **Files:** `internal/llm/openai_compatible_client.go`, client +fixtures/tests, `internal/engine/engine.go` caller verification, `docs/internal/llm.md`. + +1. Delete `OpenAICompatibleConfig.BaseURL`/`Model`, stored `baseURL`/`defaultModel`, + constructor endpoint normalization, and request fallback branches. Use the + selected target endpoint/model directly; retain final validity checks. +2. Remove the default-model parameter from the request builder and its caller. + Preserve HTTP-client cloning/timeouts and request authentication behavior. +3. Make the internal constructor infallible once its only failure path is gone; + update engine assembly and internal callers rather than retaining an always-nil + error return. Update constructor fixtures to put routing settings on request + targets. Delete tests solely asserting constructor-default precedence; + retain endpoint composition/rejection and supplied HTTP-client ownership cases. +4. Retire internal configured-base fallback prose. Do not add a new default + source or change backend/profile/request precedence. + +**Check:** Relevant llm, generation, and loaded-target tests; defer compilation +where fixture migration remains. No production construction supplies the removed fields. + +## Stage 2: Encode One Root Wire Representation + +**Finding:** 2. **Files:** `internal/llm/openai_compatible_client.go` and request-mapping tests. + +1. Replace the root `domain request -> openAIChatRequest -> map` sequence with + one direct payload builder. Remove the intermediate root struct and obsolete + projection helper; retain useful nested typed wire structures. +2. Preserve explicit numeric zero versus omission, session normalization, + native schema envelopes, reserved/empty extension-key errors, and flattened + provider extensions. Derive structured-output evidence from the same selected + constraint, not a second independent mapping. +3. Keep successful payload encoding once and existing encoding-failure error + classification/privacy. Do not introduce JSON round trips or a custom serializer. + +**Check:** Run `go test ./internal/llm ./internal/generation`; reuse request-mapping, +presence, prepared-payload freezing, session, and structured-dispatch assertions. + +## Stage 3: Require Captured Repository Operations + +**Finding:** 3. **Files:** `internal/promptdef/repository.go`, `internal/profile/repository.go`, +`internal/usecase/{runner,prompt_inspection,profile_inspection,requirements_inspection}.go`, +`internal/configuration/prompts.go` and `targets.go`. + +1. Make the runner's prompt repository explicitly provide `GetPromptSnapshot`, + `GetPromptInspection`, and `GetRequirementSnapshot` with their existing + signatures. Require `GetProfileIdentity` alongside `GetProfile` on its profile + repository. Reuse the existing domain/template values; no public interfaces change. +2. Call these captured operations directly. Remove capability assertions and + alternate full-definition/profile lookup/hash paths. Do not add new rejection + rules for hashes or introduce runtime hashing in another helper. +3. Preserve direct not-found versus load-error ownership, context checks, exact + IDs, and metadata-only inspection. Keep captured definition detachment. +4. Remove unused legacy repository methods and the use-case hash wrapper once + callers are migrated; retain the hash implementation used by configuration. + +**Check:** Inspect the real configuration snapshot against the new interfaces. +Use-case tests may remain uncompilable until Stage 4; no transitional fallback is needed. + +## Stage 4: Migrate Repository Fixtures And Inspection Tests + +**Finding:** 3. **Scope:** private repository fixtures in use-case tests and affected configuration tests. + +1. Update small repository fixtures to supply captured hashes, metadata, and + requirement snapshots through Stage 3's operations. Prefer real in-memory + configuration where useful, rather than rebuilding configuration resolution. +2. Replace the missing-metadata-capability fallback case with captured-hash and + narrow-inspection behavior. Preserve useful tests for missing resources, + lookup errors, cancellation, independent returned metadata, and namespaces. +3. Remove fixture methods/helpers that solely support deleted alternatives. + Keep convenient no-repair runner construction and controlled external doubles. + +**Check:** Focused configuration/use-case prompt, profile, and requirement +inspection tests. Renderer-related failures may be deferred through Stage 6. + +## Stage 5: Keep Only Captured Rendering + +**Finding:** 3. **Files:** `internal/prompt/go_renderer.go`, template helpers, engine assembly. + +1. Remove `NewGoRenderer` and the nil-template lookup mode. Keep + `NewCompiledRenderer` consuming the existing `TemplateLookup`. +2. Remove per-call parsing branches and the uncompiled session-rendering helper. + Keep construction-time compilation, missing captured-template errors, + per-call template cloning/function binding, and direct-session suppression. +3. Preserve input requirements, roles/cache-control behavior, message order, + artifact text resolution, cancellation, and rendered/session hash inputs. + Do not expand recognized template syntax or add a template cache. + +**Check:** Engine assembly already provides captured templates. Test callers +are deliberately migrated in Stage 6 rather than accommodated in production. + +## Stage 6: Move Renderer Tests Onto Captured Templates + +**Finding:** 3. **Scope:** prompt rendering tests/benchmarks and their use-case/configuration callers. + +1. Use `CompileTemplate` and a small `TemplateLookup` fixture, or a loaded + configuration snapshot, to provide message/session parses. Keep parsing outside + benchmark timing where measuring execution rather than capture. +2. Preserve syntax-error protection at compilation/capture, runtime missing + variable/input cases, direct/rendered session behavior, artifact-text lifecycle, + cancellation, and independent requirement-versus-rendering comparisons. +3. Do not recreate the deleted parser mode inside a renderer fixture. Share only + simple fixture setup; avoid a general test configuration framework. + +**Check:** Run `go test ./internal/prompt ./internal/configuration ./internal/usecase`. +Revisit deferred Stage 3–5 checks; carry any still-blocked checks forward to +final validation without adding transitional code or an intermediate gate. + +## Stage 7: Use Prepared Plans For Both Smoke Output Checks + +**Finding:** 4. **Files:** `internal/usecase/smoke_inference_case.go` and diagnostic tests. + +1. Replace the smoke case's `jsonValidator validate.Validator` with a + `validate.PreparedValidation` JSON-only plan, prepared through the existing + validator. Keep the distinct compiled schema plan and provider schema. +2. Invoke JSON-only validation first and schema validation only after syntax + passes. Preserve nil/false/true facts, cancellation precedence, scalar JSON + syntax, construction errors, privacy, and current diagnostic completion rules. +3. Do not move `json.Valid` into orchestration or add a composite validation API. + JSON-only plan creation performs no schema loading/compilation. + +**Check:** Focused smoke tests for malformed JSON, valid nonmatching schema, +matching schema, cancellation, and native schema dispatch. Reuse existing coverage. + +## Stage 8: Remove Standalone Direct Output Validation + +**Finding:** 4. **Files:** `internal/validate/validator.go`, `standard_validator.go`. + +1. Remove the standalone `Validator` interface and concrete host/FS + `Validate(ctx, artifact, contract)` methods after Stage 7's migration. +2. Delete the direct on-demand schema callbacks. Keep `ValidationPreparer`, + `PreparedValidation`, shared instance-mode validation, and prepared schema + execution. Simplify callback plumbing tied only to the removed workflow + without writing a new validation mechanism. +3. Preserve host and `fs.FS` root-source handling, eager plan compilation, + synchronous context boundaries, self-containment/portability checks, numeric + decoding, and frozen plans. Keep nil-validator/no-op behavior in the runner. + +**Check:** Production no longer calls a direct validator. Old test calls may +temporarily fail to compile; migrate them in Stage 9, not through a wrapper. + +## Stage 9: Migrate Validator Tests To Prepared Execution + +**Finding:** 4. **Scope:** `internal/validate` tests, particularly standard-validator and cancellation cases. + +1. Express meaningful direct-validation cases as plan preparation followed by + plan validation. Keep malformed instance outcomes separate from preparation + failures, rather than retaining decode-before-schema-I/O as an expectation. +2. Retain independent host/FS path and containment coverage, source mutation/removal + freezing, references/unused declarations/portability, JSON precision, basic/JSON + modes, and cancellation around reading/compilation/execution where already selected. +3. Delete redundant direct/prepared duplicates and mocks of the retired interface. + In-progress synchronous dependency work is still not detached on cancellation. + +**Check:** Run `go test ./internal/validate ./internal/usecase` and relevant root +schema/smoke tests. Resolve Stage 8's deferred checks. + +## Stage 10: Remove Copier-Derived Schema Budgets + +**Finding:** 7. **Files:** `internal/usecase/runner.go`, `prepared_execution.go`, schema/preparation tests. + +1. Replace schema-document validation/copying in + `structuredOutputFromValidationPlan` and prepared metadata cloning with trusted + detachment of the decoded schema. Preserve nil-plan/missing-document failures + and the existing frozen schema boundary. +2. Remove `cloneJSONValue` if no longer needed; do not substitute another limited + walker, JSON-number conversion, or new schema-resource check. +3. Replace the fake-plan excessive-depth rejection expectation with selected + acceptance/isolation behavior. Include a representative real loaded schema + exceeding the old copier depth or node budget in annotation data, which the + compiler accepts; verify preparation and outward detachment. Generate large + fixtures in test setup, not as bulky checked-in files. +4. Preserve invalid/external-schema rejection and runtime provider-value budgets. + Do not turn the regression into a promise of arbitrary depth or provider support. + +**Check:** Focused prepared/schema-file tests plus `internal/validate` and +`internal/jsonvalue` tests establish that only schema-copy acceptance changed. + +## Stage 11: Detach Captured Backend And Profile Values Without Revalidation + +**Finding:** 5; preserve 8. **Files:** `internal/domain/smoke_diagnostics.go`, +`internal/backend/registry.go`, `internal/configuration/targets.go`, associated tests. + +1. Apply the ownership inventory: final validation/effective construction remains + responsible for configured acceptance; later lookup does not repeat it. +2. Make `CloneBackend` an infallible trusted deep clone, retaining smoke metadata + detachment. Remove copy-only errors from effective registry construction and + update its callers; `GetBackend` still reports genuine not-found errors. +3. Keep `GetProfile` returning a detached target/map using trusted cloning, + retaining context/not-found failures and presence semantics. Keep required + construction ownership separation from the retained ConfigKit result. +4. Leave configured overlay/acceptance semantics intact in this stage; assembling + a new effective map is not the same as copying a captured lookup result. + Do not remove final `validateExtraParams` or runtime override validation. + +**Check:** Backend/configuration tests cover mutation isolation, composed targets, +null/empty values, and construction errors without pinning deleted copy-error branches. + +## Stage 12: Transfer Accepted Provider Maps Through Internal Resolution + +**Finding:** 5; preserve 8. **Files:** `internal/usecase/runner.go`, +`profile_inspection.go`, `smoke_selection.go`, related ownership tests. + +1. With detached lookup values from Stage 11, pass selected maps through private + target resolution rather than copying them again. Runtime replacement maps + already belong to the converted request; scalar overrides need not mutate them. +2. Remove the extra validating map copy in profile inspection. Retain credential + redaction and the root facade's independent public projection. +3. Remove redundant `copyExtraParams` uses in target/override/smoke selection and + the helper if unused. Preserve existing nil/empty outcomes, including inheritance + for an empty runtime map; do not let sharing alter public JSON shapes. +4. Keep the facade's `toDomainExecutionTargetOverride` validation/deep copy and + actual client/result detachment. Do not expose internal request sharing as a + new mutable consumer interface. + +**Check:** Focused target precedence, profile inspection, runtime override, smoke +selection, and mutation-isolation tests. Retain typed/numeric runtime cases. + +## Stage 13: Share One Immutable Prepared Metadata Snapshot + +**Findings:** 5–6. **Files:** `internal/usecase/prepared_execution.go`, +`prepared_execution.go`, `convert.go`, lifecycle/ownership tests. + +1. Remove payload cleanup's write into prepared metadata. Keep clearing private + credential, observer, retry, validation, and execution payload references under + the existing lifecycle. Owner identity and claim/discard synchronization stay. +2. Retain the prepared operation's one redacted metadata snapshot for both + internal execution and details. Remove the extra provider-map clone and + separate deep details snapshot created by `PrepareExecution`. +3. Make the internal details accessor return that immutable private snapshot; + its root caller performs the sole outward `fromDomainPreparedRun` copy. + Update private comments/tests to distinguish borrowed internal metadata from + mutable public details. Do not expose an internal type in a public signature. +4. Delete `clonePreparedRun` and hidden copy-error-to-nil fallback once unused. + Preserve nil/zero public handle behavior, details after use/discard, source + freezing, and private direct-key lifetime. Execution and repair must not mutate + shared metadata, including nested maps, cache controls, or schema values. + +**Check:** Race-enabled prepared lifecycle, concurrent claim/discard, engine/view +binding, independent details, mutation, credential, and repair tests. Public Run +results also remain detached from metadata retained by the handle. + +## Stage 14: Simplify Smoke Target And Public Report Detachment + +**Finding:** 5. **Files:** `internal/usecase/smoke_tester.go`, +`smoke_diagnostics.go`, smoke conversion/ownership tests. + +1. Keep accepted targets private and read-only through diagnostic orchestration; + remove their validation-only provider-map copy in `smokeReportTarget` where + no mutable ownership handoff occurs. Keep endpoint sanitization, credential + redaction, and genuine endpoint errors. +2. Use the existing trusted public execution-target projection in + `fromDomainSmokeTestReport`, rather than another `CopyMap`. Make report + conversion infallible and update facade callers, removing only the impossible + report-copy error path. Other operational/configuration errors remain. +3. Retain detached report facts, attempts, pointer fields, and nested provider + values. Delete the internal invalid-provider-map conversion-error test that + sustains revalidation of fabricated unaccepted reports; retain its meaningful + public acceptance/ownership protection at the real boundary. + +**Check:** Smoke conversion isolation, privacy, partial/canceled reports, and +configured-default/runtime-override diagnostics through the public facade. + +## Stage 15: Verify Cross-Path Ownership And Runtime Compatibility + +**Findings:** 5–8. **Scope:** focused existing public and package contract tests. + +1. Reconcile the ownership inventory with the delivered call paths. Verify that + prepared details/results, profile inspection, smoke reports, and injected + initial/repair requests cannot mutate retained engine/handle state. Extend a + representative existing test only where these changes leave a meaningful gap. +2. Keep valid typed/named runtime containers, numbers, nil/empty forms, map + replacement, and current invalid/cyclic/over-budget rejection at acceptance. + Distinguish those provider budgets from removed schema budgets. +3. Exercise a mutating injected client across initial and repair calls and + concurrent details/claim/discard where existing coverage supplies the seam. + Keep the built-in HTTP payload freeze and existing artifact defensive copy. +4. Review remaining copy/validation callers for purpose. Remove overlooked + repeated acceptance work on the selected paths, not all copies indiscriminately. + Do not preserve private copy-failure tests as new public requirements. + +**Check:** Run targeted race-enabled root/use-case/configuration/jsonvalue tests. +Record results and any reconciled ownership edge; no benchmark framework or exact +allocation threshold is needed. + +## Stage 16: Reconcile Canonical Documentation And Retire Obsolete Descriptions + +**Findings:** 1–8. **Scope:** GoDoc, `docs/internal/{overview,llm,runner,sources}.md`, +relevant consumer/format guidance, feature roadmap completion status. + +1. Remove descriptions of client routing defaults, uncaptured renderer/repository + fallbacks, standalone direct output validation, duplicate prepared metadata, + and prepared-schema copier limits. Describe the delivered ownership paths. +2. Record broadened schema acceptance only where consumer/format guidance needs + it. Keep the self-contained-schema rule, runtime provider budgets, and existing + number handling; add no resource/provider/performance promises. +3. Keep exact public contracts in GoDoc, formats/wire behavior with their owning + references, and private mechanics in internal docs. Update affected comments + throughout, not just the package inventory. +4. Update the feature roadmap with delivered outcomes and pending final checks; + do not mislabel the audit's original tests as replacement validation. Leave + unrelated completed-roadmap retirement and release notes to their own work. + +**Check:** Documentation links and whitespace, plus searches for the retired +names/claims to separate historical audit evidence from stale current guidance. + +## Stage 17: Validate And Close The Sprint + +**Scope:** combined final state, all findings and deferred checks. + +1. Resolve remaining deferred compilation/tests; format changed Go files. Run + `go test -race ./...` and `go vet ./...` using the selected local toolchain. + The suite includes maintained examples; separate ordinary/full build/example + runs are not additional required checks for the same unchanged candidate. +2. Check documentation links, `git diff --check`, full intended/staged diff, + and repository status. Exclude secrets, generated artifacts, local workspace + files/replacements, and unrelated changes. No release publication check is needed. +3. Confirm removal of client defaults, duplicate root wire representation, + optional lookup fallbacks, uncaptured rendering, direct-validator lifecycle, + redundant copy-validation paths, duplicate prepared metadata, and schema-copy + budgets. Verify that remaining copies implement real ownership boundaries. +4. Record final evidence and completion for findings 1–7 and preservation of 8 + in the feature roadmap/plan. A deferred check or unimplemented selected change + is not completion. Preserve actual scope if a new blocker appears and surface + it rather than substituting a compatibility layer or narrowing the feature. +5. Commit the completed work under repository policy and report the result and + any remaining unrelated workspace changes. Do not push, tag, or publish. + +**End state:** The selected roadmap is implemented with fewer private workflows, +representations, and redundant traversals; public behavior and ownership remain +intact except for the explicitly removed incidental schema restriction. diff --git a/docs/roadmap/responsibility-simplification.md b/docs/roadmap/responsibility-simplification.md index 5a2d6bd..505dab9 100644 --- a/docs/roadmap/responsibility-simplification.md +++ b/docs/roadmap/responsibility-simplification.md @@ -14,9 +14,10 @@ All significant decisions are settled. Implementation is pending. The current baseline is `f660b6e`. The production code remains at the audited `1ff6d77` baseline; subsequent commits record the audit and accepted decisions. The audit owns evidence and decision rationale; this document owns the selected -target and delivery scope. The older [audit remediation](audit-remediation.md) -and its completed implementation plan describe an earlier sprint, not this work. -A separate implementation plan will define prompt-sized stages for this scope. +target and delivery scope. The [implementation plan](implementation.md) owns the +ordered stages and validation for this work. The older +[audit remediation](audit-remediation.md) describes an earlier completed sprint; +its previous plan is retained in Git history, not the current implementation plan. ## Targeted End State @@ -120,12 +121,11 @@ No composite validator or diagnostic framework is needed. **Audit findings:** 5–6, with finding 8's preservation decision. -Plan these changes together. For affected paths, establish where values are -accepted, whether they can subsequently be mutated, and where they cross into -caller/client-owned mutable state. A small ownership inventory in the -implementation plan is sufficient. Include captured backend/profile lookup, -prepared details, execution results, injected-client and repair requests, and -smoke reports; do not optimize one path by moving its extra copy into another. +Acceptance and detachment have distinct responsibilities across captured +backend/profile lookup, prepared details, execution results, injected-client +and repair requests, and smoke reports. The target removes redundant traversals +across those paths together, rather than moving a copy into another helper. +The implementation plan records the corresponding ownership inventory. Keep validation at actual input-acceptance boundaries. For previously accepted values, use existing trusted detachment where a mutable copy is needed and @@ -219,6 +219,6 @@ and duplicate state are actually gone rather than renamed or relocated. The audit's test results describe the original baseline, not evidence of the replacement implementation. Record actual implementation outcomes separately. -No additional product decision or evidence-gathering phase is required before -preparing the staged plan. Release selection/publication and retirement of the -older completed audit documents are separate work from this roadmap. +No significant product decisions remain open. Release selection/publication and +retirement of the older completed audit documents are separate work from this +roadmap.