From d7fe70520ac4ebcf05bc598ae53abfe3302f4dcd Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Fri, 9 Oct 2026 04:23:27 +0000 Subject: [PATCH] Plan profile and prompt listing in focused stages --- docs/roadmap/implementation.md | 1498 +++++--------------------------- docs/roadmap/listing.md | 4 +- 2 files changed, 234 insertions(+), 1268 deletions(-) diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 3ceb03c..70dc774 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1,1288 +1,254 @@ -# Computational Efficiency Implementation Plan +# Profile And Prompt Listing Implementation Plan -Implement the remaining local work in the [performance roadmap](performance.md) -in order. Each sequential stage is scoped for one Luna coding-agent prompt. -Update status and evidence as each work unit lands. -Stages 1–14 are complete. The completed -inspection plan is preserved at -`481e2d4:docs/roadmap/implementation.md` in Git history. +Implement the [listing roadmap](listing.md) in the numbered order below. +Each stage is scoped for one Luna coding-agent prompt and should leave a +compiling, tested unit. All four stages are pending. -## Baseline And Scope +This replaces the completed performance plan, preserved in +[Git history](https://gitea.maximumdirect.net/eric/promptkit/src/commit/7325a34c070c77c4784ab76c916fc8a4d79b5c46/docs/roadmap/implementation.md). +For older inspection-roadmap references to this pathname, the completed +[inspection plan](https://gitea.maximumdirect.net/eric/promptkit/src/commit/481e2d494c5a4e8b7f3a2a6f1cceaeb3f7c71d05/docs/roadmap/implementation.md) +is also preserved. These historical plans are not work for this sprint. -- Planning baseline: `481e2d494c5a4e8b7f3a2a6f1cceaeb3f7c71d05`. -- Roadmap item 1 is complete. ConfigKit received the performance request in its - `docs/roadmap/promptkit-performance-request.md`, commit `d53021e`. Do not - resubmit it, implement upstream changes, wait for a release, or upgrade the - dependency as part of this plan. A later ordinary dependency update will - pick up available upstream improvements. -- Roadmap items 2–5 are localized runtime improvements. Item 7 concerns - identity/metadata inspection; item 6 is conditional on ownership evidence - and measured benefit. The latter condition is part of the accepted scope, - not permission to leave the review unfinished. -- No public declarations, configuration semantics, dependencies, release - tooling, or version tags need changing. This is an internal optimization - effort, not a new consumer performance contract. +## Baseline And Working Rules -## Working Rules +Planning baseline: `7325a34c070c77c4784ab76c916fc8a4d79b5c46`. +There are no listing methods yet. Configuration already captures profiles in +`loaded.indexedProfiles` and prompts in `loaded.prompts`, after the final +ConfigKit handoff. `internal/engine.Engine` currently retains the runner and +the operator inspection view. -Start with [development](../development.md) and select applicable policies. -Read the [agent](../policy/agents.md), -[architecture](../policy/architecture.md), +Start with [development](../development.md), then read the applicable +[agent](../policy/agents.md), [architecture](../policy/architecture.md), [testing](../policy/testing.md), and -[documentation](../policy/documentation.md) policies; reuse unchanged policies -already read. Consult the [component overview](../internal/overview.md) and the -focused documents named below. +[documentation](../policy/documentation.md) policies. Reuse unchanged documents +already read. Consult [internal sources](../internal/sources.md), the +[component overview](../internal/overview.md), the +[consumer guide](../consumers/pkg-promptkit.md), existing inspection GoDoc, and +the [configuration ADR](../adr/0001-configuration-boundary.md). -Preserve the [configuration handoff ADR](../adr/0001-configuration-boundary.md). -Use captured final configuration; operator provenance stays out of ordinary -execution. Keep request/result ownership, error identities, exact selection, -prepared-handle lifecycle, cancellation, and generation/repair behavior intact. -Retain indexed lookup, captured parses, per-render text reuse, operation-local -schema reuse, frozen transport retries, and constant-time queue removal. +For each stage: inspect the current worktree, preserve unrelated edits, implement +only its scope, format changed Go files, run focused verification, review the +diff, update this plan's stage status with brief evidence, and commit the +nonempty completed unit with a plain-English message. Report any blocked +validation honestly. Publication and release preparation are separate tasks. -Use standard-library mechanisms. Add no global cache, writer pool, borrowed -public result, new execution/input limit, or performance release gate. These -are scope boundaries, not new permanent product prohibitions. Internal helper -names and allocation choices below are implementation decisions, not public -invariants. Tests must not freeze a particular capacity, clone count, internal -call sequence, or exact allocation/timing number. +Use the existing module versions. No dependency upgrade, configuration-format +change, execution refactor, new public package, or release-tool change is needed. +The storage and wiring choices below are implementation decisions, not permanent +consumer invariants. Tests protect outcomes, not private helper names, clone +counts, allocation counts, or timing thresholds. -For each stage, inspect the current tree, preserve unrelated work, implement -only that stage, format changed Go files, run its focused checks, review the -diff, and commit the completed unit with a plain-English message. Update owning -internal documentation when its mechanics change. Do not widen scope to -unrelated cleanup or retire other roadmaps. Publication is a separate task. +## Selected Public Contract -## Measurement And Evidence Method +Use these root-package declarations: -Add maintained benchmarks before changing the behavior they measure. Record -the benchmark-only baseline commit so the same fixture can be rerun later. -Keep setup, source acquisition, template compilation, fixture creation, and -engine construction outside timing unless the benchmark explicitly measures -that operation. Use `ReportAllocs`, retain a result in a sink where necessary, -and check representative output outside timing. Run benchmark comparisons -without the race detector. +```go +type PromptIdentity struct { + ID string + Version string +} -The default comparison command is: - -```sh -GOTOOLCHAIN=local GOWORK=off GOPROXY=off \ - go test -mod=readonly -run '^$' -bench 'BENCHMARK_PATTERN' \ - -benchmem -benchtime=500ms -count=5 PACKAGE +func (e *Engine) ListProfiles(ctx context.Context) ([]string, error) +func (e *Engine) ListPrompts(ctx context.Context) ([]PromptIdentity, error) ``` -Replace the pattern and package with each stage's values. Dependencies and the -selected Go toolchain must already be installed for this offline invocation. -Use the same environment and unchanged fixture for before/after comparisons. -Record revisions, Go/module versions, processor, parallelism, command, repeated -results, and a short interpretation in this plan's execution evidence. Compare -ranges and medians; no benchmark-analysis dependency is required. Keep raw -profiles and generated binaries outside the repository. - -For changes affecting buffer/string/handle lifetimes, also compare retained -heap with a documented, fixed synthetic workload: hold a small batch of -results or handles, force GC, compare live heap against an equivalent setup-only -baseline, and use `runtime.KeepAlive` for the retained objects. An opt-in -benchmark is sufficient; do not add heap/timing assertions to ordinary tests. -Distinguish immutable shared strings from mutable deep copies. Report noisy -retained-heap evidence honestly rather than deriving it from `B/op`. - -Probes from the original review are preliminary evidence. Maintained fixtures -and recorded comparisons own the implementation evidence. Retain improvements -when measurements support the intended reduction without a material correctness, -retention, or maintenance trade-off; the original percentages are not targets. - -## Stage 1 — Establish Validation And Captured-Rendering Benchmarks - -**Status: Complete (2026-10-09).** Maintained basic-validation, compiled -rendering, and retained-output benchmarks are committed. The benchmark-only -baseline is `0eb009ad5a3248e2a1e12cad9b3a05906d6b0850`; no production behavior -changed. - -**Outcome:** Reproducible baselines for roadmap items 2 and 3; no production -change in this stage. - -**Read:** `internal/validate/standard_validator.go`, its tests, -`internal/prompt/go_renderer.go`, `template.go`, renderer tests, and -[internal sources](../internal/sources.md). - -**Work:** - -1. Add `BenchmarkBasicValidation` with short text, 1 MiB nonempty text, - whitespace-only text, and Unicode whitespace. Construct bodies/contracts - outside timing and exercise the real basic validator. -2. Add `BenchmarkCompiledRendering` using `CompileTemplate` and a small - `TemplateLookup` fixture built before timing. Exercise - `NewCompiledRenderer`, not only `NewGoRenderer`'s reparsing fallback. -3. Cover a small message, an artifact-backed message around 256 KiB, and - repeated references across session/messages. Keep session output within the - existing session contract, e.g. render an input length rather than its body. - Retain the existing fallback benchmark where useful; label the paths clearly. -4. Add opt-in `BenchmarkRetainedRenderedOutput` using the evidence method - above. Capture baselines and record the benchmark commit for later comparison. - -**Verify:** `go test ./internal/validate ./internal/prompt`; compare the two new -benchmark families with the standard command. Benchmarks have no provider -calls, allocation assertions, or source parsing hidden in the compiled path. - -## Stage 2 — Check Basic Output Emptiness Without A String Copy - -**Status: Complete (2026-10-09).** Basic validation now checks the response -bytes directly. Behavioral coverage includes empty bytes, ASCII and Unicode -whitespace, ordinary text, and arbitrary non-UTF-8 bytes. The comparison with -the Stage 1 benchmark fixture is recorded below. - -**Outcome:** Basic validation avoids a response-sized temporary string. - -**Work:** - -1. Replace `strings.TrimSpace(string(artifact.Body)) == ""` in the basic branch - with `len(bytes.TrimSpace(artifact.Body)) == 0`. Remove imports only if unused. -2. Preserve statuses, `IsValid`, repair input, error handling, and the existing - before/after cancellation checks. Do not change the other validation modes. -3. Reuse existing behavioral tests. Add only missing meaningful cases for empty - bytes, ASCII/Unicode whitespace, and nonempty bytes, preserving behavior for - arbitrary bytes rather than adding a UTF-8 requirement. -4. Compare `BenchmarkBasicValidation` against Stage 1 and record the result. - -**Verify:** `go test ./internal/validate ./internal/usecase`; run the benchmark -family. Existing empty-output repair behavior remains unchanged. - -## Stage 3 — Render Messages And Sessions With Local String Builders - -**Status: Complete (2026-10-09).** Message and session template execution now -uses per-call `strings.Builder` values. Exact rendering, cancellation, and -concurrent compiled-template behavior pass; benchmark evidence shows lower -allocation without a measurable retained-output increase. - -**Outcome:** Template output becomes a string without a final buffer-to-string -copy. - -**Work:** - -1. Replace the local message `bytes.Buffer` in `Render` and the session buffer - in `executeSessionTemplate` with local `strings.Builder` values. Execute - templates into their addresses and obtain `String()` afterward. -2. Keep template cloning/binding, error checks, cache metadata, ordering, and - cancellation placement. Leave artifact-text conversion/cache unchanged. - Do not introduce eager growth based on all artifacts or shared writers. -3. Run existing exact-content, session, cancellation, and compiled-template - concurrency coverage. Add a behavioral case only if the changed path lacks - protection; do not test builder internals. -4. Compare Stage 1's compiled-rendering and retained-output measurements. - Check both allocated bytes and live output storage; explain any retention - trade-off before retaining the change. Update internal sources if needed. - -**Verify:** `go test -race ./internal/prompt`; run compiled-rendering benchmarks -without `-race`. Different request inputs must remain independent. - -## Stage 4 — Reserve A Useful Initial File-Read Capacity - -**Status: Complete (2026-10-09).** File reads now use the opened regular -file's size only as a capped initial-capacity hint, continue reading to EOF, -and compact a completed body when a stale hint leaves disproportionate unused -capacity. The exact-size and overreported-size benchmark comparisons are -recorded below. - -**Outcome:** Ordinary file reads avoid early geometric buffer growth without -trusting metadata as the amount of content to return. - -**Read:** `internal/artifact/reader.go`, `reader_test.go`, and internal sources. - -**Work:** - -1. Add `BenchmarkArtifactFileRead` using real temporary regular files of small, - 1 MiB, and larger representative sizes. Create files outside timing, warm - content-type lookup, then measure the real reader. Record a baseline before - production edits. -2. Reserve initial capacity from the opened file's size. Use zero for a - nonpositive hint and `min(size, 1 MiB)` for a positive hint before conversion - to `int`. The 1 MiB reservation ceiling is an internal allocation heuristic, - not a file/content limit; data continues to grow normally beyond it. This - avoids an enormous eager allocation from stale or unusually large metadata - and keeps integer conversion safe on supported architectures. -3. Keep the read-until-EOF loop and existing regular-file, error, hash, metadata, - closure, and cancellation behavior. Do not size the result from `Stat`, use - `ReadFull` to enforce its size, impose an input limit, or add background I/O. -4. Extend the existing opened-file seam to test an overreported and an - underreported size, including a very large reported hint with small actual - content. Assert returned bytes/size/hash and failures, not backing capacity. - Add opt-in `BenchmarkRetainedArtifactFileRead` for representative live-buffer - retention, including overreported size. Compare both allocation and retention - and update internal sources. - -**Verify:** `go test ./internal/artifact`; run `BenchmarkArtifactFileRead`. -Existing progress-cancellation and nonregular-file tests remain authoritative. - -## Stage 5 — Establish Encoding Benchmarks And Error-Boundary Coverage - -**Status: Complete (2026-10-09).** `PrepareGeneration` now has an offline -benchmark baseline for no extras, small extras, a 1 MiB string extra, nested -JSON extras, and structured output. Tests protect nested invalid-extra context -and request-error classification when the schema cannot be encoded. No -production behavior changed. - -**Outcome:** A repeatable request-preparation baseline and focused protection -for the classifications that Stage 6 must preserve. No production change. - -**Read:** `internal/llm/openai_compatible_client.go`, client/prepared-generation -tests, [integration contract](../integrations/openai-compatible-chat.md), and -[internal model client](../internal/llm.md). - -**Work:** - -1. Add `BenchmarkPrepareGeneration`, measuring actual `PrepareGeneration` - without HTTP dispatch: no extras, small extras, a 1 MiB string extra, and - nested JSON-compatible extras. Build inputs and the client outside timing; - use a direct synthetic credential or no credential so environment changes - do not affect results. Include a representative structured-output request. -2. Extend existing invalid-extra tests only as needed for a nested unsupported - value or cycle, checking `ErrInvalidRequest`, useful key context, and zero - transport calls. Do not freeze full diagnostic wording or all invalid-key - combinations. -3. Cover a serializable-extra request whose schema cannot be encoded: the - client currently classifies full-request encoding failure as - `ErrRequestFailed`. Also cover invalid extras plus an unencodable schema: - invalid extras retain `ErrInvalidRequest`. This protects classification - when validation moves to a failure-only path. -4. Capture the baseline. Existing body omission, explicit-zero, cache-control, - structured-output, and frozen-attempt tests own their established behavior. - -**Verify:** `go test ./internal/llm`; run `BenchmarkPrepareGeneration`. - -## Stage 6 — Encode Valid Extra Parameters Only In The Complete Request - -**Status: Complete (2026-10-09).** Successful request preparation now encodes -the assembled payload once. Extra-parameter classification remains covered on -the failure path, including precedence when the schema also cannot be encoded. -The Stage 5 benchmark comparison is recorded below. - -**Outcome:** Successful request preparation does not serialize extras twice. - -**Work:** - -1. Keep `openAIChatRequestPayload` responsible for body assembly and empty/ - reserved-key checks. Remove its successful-path per-value `json.Marshal` - calls; place accepted values directly into the payload map. -2. Marshal the complete assembled map once in `PrepareGeneration`. On success, - continue authentication resolution and construction of the frozen payload - exactly as before. Do not change body shapes or omission/presence semantics. -3. On complete-request marshal failure only, examine the extras with the - existing per-value marshal check to identify an invalid extra. Return the - existing `ErrInvalidRequest` classification and `extra_params.` context - for that failure. If extras are serializable, retain `ErrRequestFailed` for - the original full-request encoding failure. Cold-path repeated work is - acceptable; successful requests get the optimization. No new reflective - validator or stricter accepted-value rules are needed. -4. Adjust any helper-only tests whose incidental assembly boundary changed, - retaining protection at `PrepareGeneration`/`Generate`. Preserve frozen - request bytes across attempts and record Stage 5 comparisons. Update internal - model-client mechanics; change integration prose only enough to describe - validation before dispatch without prescribing two encodings. - -**Verify:** `go test ./internal/llm ./internal/generation`; run encoding -benchmarks. Invalid requests still dispatch nothing, and retry tests still -exercise one frozen body per generation. - -## Stage 7 — Establish Inspection Benchmarks - -**Status: Complete (2026-10-09).** Captured configuration lookup and public -requirement-inspection benchmarks are committed. Repeated benchmark-only -baseline measurements are recorded below; no production behavior changed. - -**Outcome:** Owned lookup and assembled requirement-inspection baselines, with -no production change. This separates fixture work from the selection refactor. - -**Work:** - -1. Add `BenchmarkInspectionSelection` in configuration tests using a captured - snapshot constructed outside timing. Measure existing prompt/profile lookups - with small and representative mutable nested-extra payloads. -2. Add root `BenchmarkRequirementInspection` using a real engine with a simple - prompt and the same small/large-extra distinction. Build the engine and - request outside timing, supply needed caller variables, and exercise only - the public inspection call. No artifacts, schemas, or providers are accessed. -3. Check selected identities and inventories outside timing. Record the - benchmark-only baseline commit and repeated measurements before routing - changes. Do not treat long immutable strings as deep-copy workload. - -**Verify:** `go test ./internal/configuration .`; run the two new benchmark -families in their owning packages. - -## Stage 8 — Add Shared Lightweight Configuration Lookups - -**Status: Complete (2026-10-09).** Captured prompt and profile selection now -share their lookup rules, with owned metadata-only results available for -inspection. The Stage 7 benchmark comparison is recorded below; root -requirement-inspection routing remains unchanged. - -**Outcome:** Captured configuration can provide identity/inspection metadata -without copying execution payloads. Existing runtime behavior stays unchanged -until Stages 9 and 10. - -**Read:** `internal/configuration/prompts.go`, `targets.go`, their tests, -`internal/usecase/prompt_inspection.go`, `profile_inspection.go`, and public -inspection GoDoc. - -**Work:** - -1. Extract existing prompt ID/version selection into a private - `lookupPromptEntry` helper. Preserve literal keys, blank checks, omitted - version ambiguity, not-found mapping, and construction-time hashes. Use it - from existing owned-definition/inspection construction; do not duplicate - selection rules. -2. Add internal `GetPromptInspection(context.Context, string, string) - (*domain.PromptInspection, error)` returning owned cached inspection data - with copied inputs, - without copying template structures. Keep `GetPromptSnapshot` independently - owned, but stop copying an unused inspection-input slice just to obtain its - cached hash. Retain the private `prompt` helper where callers need both - values, composing it from shared lookup/copy helpers. -3. Extract a shared private indexed-profile lookup used by `GetProfile` and - new `GetProfileIdentity(ctx, id) (string, error)`. The new method returns the - selected lookup ID after successful lookup, with no `ExtraParams` copy or - readiness/credential validation. `GetProfile` retains its owned deep copy. - Use existing context/not-found handling and the captured target invariant. -4. Test selection parity, owned returned prompt inputs, missing IDs, and partial - profiles through configuration APIs. Extend Stage 7's configuration benchmark - with lightweight lookup cases so old/new paths can be compared with the same - payload. Root requirement-inspection routing remains unchanged in this stage. - -**Verify:** `go test ./internal/configuration ./internal/usecase .`; run the two -benchmark families in their owning packages. No new public methods or ConfigKit -queries occur during lookup. - -## Stage 9 — Use Profile Identity Lookup During Requirement Inspection - -**Status: Complete (2026-10-09).** Requirement inspection now uses the -identity-only profile capability when available and keeps the full-profile -fallback for other internal repositories. Exact identity, error classification, -and partial-profile behavior are covered; benchmark comparison with the Stage 7 -baseline is recorded below. - -**Outcome:** Requirement inspection no longer deep-copies a target it does not -use. - -**Work:** - -1. Add a narrow optional repository capability in `internal/usecase` matching - `GetProfileIdentity`. Add `resolveProfileIdentity` beside the existing - profile selection helper, sharing basic ID/repository checks where useful. -2. Use the capability for the assembled configuration adapter. Wrap repository - failures with existing `ErrProfileLoad` and keep exact requested identities. - For internal repositories lacking the capability, fall back to - `resolveProfileSelection`, preserving its nil-profile/nil-target checks. - Do not change existing repository interfaces or public extension contracts. -3. Route only the identity requirement in `InspectRequirements` through this - helper. Keep default selection, partial-profile support, cancellation/error - order, direct-session handling, and analyzer behavior. `InspectProfile` and - preparation still need complete owned targets and keep their existing path. -4. Reuse existing selection tests, add focused capability/fallback parity - coverage, and compare `BenchmarkRequirementInspection` with Stage 7, - especially the mutable nested-extra case. Update internal runner mechanics. - -**Verify:** `go test ./internal/configuration ./internal/usecase .`; run the -requirement-inspection benchmark. No model, credential, admission, schema, or -provenance work is introduced. - -## Stage 10 — Select Prompt Inspection Metadata Without Execution-Definition Copies - -**Status: Complete (2026-10-09).** Prompt and requirement inspections now use -captured metadata when available, preserving the definition-based fallback. -Exact snapshot selection, hash parity, session bypass, and caller-owned results -are covered; race verification passed. The repeated inspection benchmark -comparison is recorded below. - -**Outcome:** Prompt and requirement inspection avoid copying template execution -structures they do not consume. - -**Work:** - -1. Add an optional `GetPromptInspection` repository capability in the runner. - Factor a shared inspection-selection helper from `InspectPrompt`: use the - captured metadata capability when available, otherwise derive the same - result from `resolvePromptDefinition` as today. Preserve guards, not-found/ - ambiguity categories, and cached-hash versus fallback-hash behavior. -2. Let `InspectPrompt` use that helper. Let `InspectRequirements` select the - same prompt metadata, use its default profile when needed, and request the - exact requirement snapshot by selected ID/version. -3. The analyzer currently reads only `definition.ID`/`Version`; pass a local - identity-only `domain.PromptDefinition` constructed from that metadata rather - than loading a full execution definition. Do not change analysis semantics - or enlarge its scope. Keep snapshot metadata/template ownership as today. -4. Protect exact selectors, default selection, session bypass, caller-owned - results, and hash equality with preparation through existing tests. Cover - both optional-capability and fallback paths without pinning helper call - counts. Compare inspection benchmarks and update runner/source documentation. - -**Verify:** `go test -race ./internal/configuration ./internal/usecase .`; -run inspection benchmarks without `-race`. Requirement inventories and public -prompt metadata remain identical for the same captured context. - -## Stage 11 — Establish Assembled Workflow Benchmarks - -**Status: Complete (2026-10-09).** The benchmark-only baseline is committed at -`743e78d`; it covers public preparation, prepared-execution creation, `Run`, -and repeated `Details()` with small, large-message, and mutable-payload -fixtures. Fixture checks and the repeated measurements are recorded below. - -**Outcome:** Reproducible end-to-end local benchmarks, including the baseline -for conditional snapshot optimization. No production change. - -**Read:** `internal/usecase/runner.go`, `prepared_execution.go`, root -`prepared_execution.go`, `convert.go`, client adapters, and -`prepared_execution_contract_test.go`. Locate the actual adapter file rather -than assuming an adapter filename. - -**Work:** - -1. Add root `BenchmarkEnginePreparation`, `BenchmarkEngineRun`, and - `BenchmarkPreparedExecutionDetails` using a real assembled engine and an - injected deterministic client. Use no live HTTP, environment credentials, - or source mutation in timing. Keep engine setup outside timing; discard - unused prepared handles so the benchmark does not retain every iteration. -2. Cover a small case and separate large-message and mutable-payload cases. - Use a few hundred optional schema properties and nested extra-map entries - within existing JSON-value limits, not merely long immutable strings. - Use the real schema validator for the assembled structured-output case. - Separate creation cost from repeated `Details()` cost. -3. Capture the benchmark-only baseline commit and repeated results. Verify - that the large-message case measures rendering/output storage while the - mutable-payload case exercises deep copying. Record the specific schemas, - input sizes, retained sinks, and handle lifetimes used by the fixtures. - -**Verify:** `go test .`; run the three new families. Check representative -content/validation outside timing. Keep preparation and `Run` measurements -separate from engine construction and provider latency. - -## Stage 12 — Establish Snapshot Ownership And Live-Heap Evidence - -**Status: Complete (2026-10-09).** Internal snapshot and fixed-batch retained- -heap benchmarks are committed at `11ecb92`. The ownership inventory confirms -that message/cache metadata, hashes, and schema are preparation-owned, while -nested `ExtraParams` still need a deep copy before any transfer. Repeated -measurements and lifecycle assumptions are recorded below; no production -behavior changed. - -**Outcome:** A measured ownership inventory for Stage 13's conditional change. -No production change; keep this analysis separate from creating the assembled -workflow benchmarks. - -**Read:** The preparation, handle, conversion, and adapter files listed in -Stage 11, plus `internal/domain/message.go` and `internal/jsonvalue/jsonvalue.go`. - -**Work:** - -1. Add a focused internal `BenchmarkPreparedSnapshot` using existing fast - runner collaborators to expose snapshot cost without schema-compilation - noise. Add opt-in `BenchmarkRetainedPreparedHandles` with ready, completed, - and discarded subcases, holding a fixed batch under the common live-heap - evidence method. These measurements are not lifecycle assertions. -2. Record the ownership inventory and baseline in execution evidence: message - slices/cache pointers are copied by concatenation; hashes are newly allocated; - schema is copied from the validation plan; nested extras can still alias an - internal repository/request after shallow target merging; details must remain - separate from execution and survive completion/discard; public conversion - and injected-client adapters create caller-owned values. Confirm these facts - against current code, including fake/internal collaborators, before Stage 13. -3. Record readiness/run/discard lifetime assumptions and repeated observations. - Check whether the candidate below can save mutable schema/message metadata - copies without transferring an externally accessible map. Do not remove a - production copy during this analysis stage. - -**Verify:** `go test ./internal/usecase .`; run the new families. Benchmarks must -check representative content/validation outside timing, avoid counting immutable -message length as deep-copied bytes, and document lifetime assumptions. - -## Stage 13 — Conditionally Consolidate Prepared Creation's First Snapshot Copy - -**Status: Complete (2026-10-09).** The selected transfer was retained. Prepared -execution now reuses preparation-owned messages, hashes, and schema, while -deep-copying nested provider extras and keeping an independent details -snapshot. Source-mutation and invalid-copy coverage pass; benchmark evidence -shows meaningful reductions for mutable payloads without a material increase -in retained heap. - -**Outcome:** The ownership inventory supports transferring preparation-owned -values. The simplified creation path is retained with its nested-extra copy, -independent details snapshot, and existing cleanup and caller-ownership -boundaries. - -**Selected candidate:** Transfer preparation's already-owned messages, hashes, -and schema into its private execution payload instead of deep-copying them -again in creation. Still detach nested extras and still make a separate, -complete details snapshot. - -**Work:** - -1. Use Stage 12's inventory to confirm that `operation.run`'s message/cache - structures, input-hash map, and copied schema have no mutable external owner. - Immutable string sharing is safe; no read-only promise should be imposed on - public or injected values to justify removing a copy. -2. If the inventory supports the candidate, replace the first general - `clonePreparedRun(operation.run)` with transfer of `operation.run`, after - deeply copying its `EffectiveModelParams.ExtraParams` using existing - `jsonvalue.CopyMap` and preserving the invalid-request error boundary. - The schema already passed through `jsonvalue.Copy` during preparation; - keep that validation. Leave the subsequent full clone for `details` intact. -3. Keep `Details()`'s defensive clone and facade conversion, claim/discard - synchronization, payload cleanup, observers, and generated-client copies. - Further conversion or borrowed-snapshot redesigns are not this stage's - candidate. Preserve public availability of details after execution and - discard, and the separation of mutable details from execution state. -4. Verify independence from retained internal source-profile, renderer, and - validation-plan data using existing tests plus missing meaningful nested- - mutation coverage. Compare Stages 11–12 creation and live-heap evidence. Retain - the candidate only if ownership remains correct and savings justify it - without a material retention/complexity trade-off. Otherwise restore the - candidate edit and record why the current design is retained; keep useful - benchmarks. Do not substitute a more elaborate refactor merely to claim a - reduction. Update runner mechanics if the change is retained. - -**Verify:** `go test -race ./internal/usecase .`; run snapshot/assembled/retained -benchmarks without `-race`. Existing single-claim, discard races, frozen-source, -redaction, repair, and details-after-completion behavior must continue to pass. - -## Stage 14 — Consolidate Evidence And Verify The Complete Local Scope - -**Status: Complete (2026-10-09).** The complete local roadmap has been -implemented, ownership decisions and before/after evidence are recorded, and -the final suite, vet, example, formatting, and changed-document link checks -passed. No release or publication work was performed. - -**Outcome:** The local roadmap scope is complete with clear measurements and -unchanged consumer contracts. - -**Work:** - -1. Record a concise before/after summary and revision/command references for - each changed path in execution evidence. Keep any Stage 13 retain decision - explicit. Include cumulative-allocation versus retained-heap distinctions - and small-input trade-offs; do not imply provider-latency improvements. -2. Review canonical internal source/runner/model-client descriptions against - the implemented flow. Consumer GoDoc and format semantics should remain - unchanged; remove incidental duplicated encoding mechanics where Stage 6 - required it. Update the inventory only if responsibilities actually changed. -3. Verify roadmap coverage: item 1 remains complete without dependency work; - items 2–5 have implemented local improvements; item 7 uses lightweight - inspection; item 6 has the selected conditional outcome and evidence; - production compiled rendering and assembled offline workflows have maintained - benchmarks. Do not leave a requested item silently unexamined. -4. Run assembled verification below once on final code, review the complete - diff and tree for artifacts, and mark stages/roadmap progress accurately. - Keep this plan as the implementation record until ordinary roadmap - retirement; do not add release gates or publish a release. - -**Verify:** +- List final effective captured identities, including maintained and consumer + profiles. Inherited profiles keep their own keys. Return each profile ID and + each prompt ID/version pair once; omit historical-only contributions. +- Preserve literal spelling. Sort profiles lexically by ID; sort prompts + lexically by ID, then version. Do not interpret versions as SemVer or select + a preferred version. Existing omitted-version inspection remains unchanged. +- Results are caller-owned. Concurrent listing and other supported engine + operations cannot mutate the captured inventory through these results. +- Empty inventories succeed. Nil and zero-value public engines return an error + matching `ErrInvalidConfig`, consistent with existing facade checks. +- For a valid engine, context cancellation/deadline errors retain their standard + identity. Return no partial inventory on error. Follow existing inspection + precedence: validate the public engine before delegating context handling. + Check context before and after the synchronous copy/conversion work; do not + promise immediate interruption or deterministic precedence in a cancel/return + race. No new nil-context behavior is selected. +- `PromptIdentity` follows other inspection values: no stable JSON + representation is promised. Do not add custom serialization or pin nil versus + nonnil empty slices in GoDoc or tests. +- Membership is not execution readiness. Listing performs no per-item full + inspection, target validation, credential resolution, resource reads, + rendering, admission, generation, or ConfigKit provenance traversal. + +The roadmap owns scope and intent; implemented GoDoc will own the exact API. +Do not add backend listing, filters, pagination, reloads, runtime mutation, +provider discovery, new caller limits, or stronger catalog availability promises. + +## Stage 1 — Capture And Read Identity Inventories + +**Status: Pending.** + +**Scope:** `internal/domain`, `internal/configuration`, and focused adapter tests. + +1. Add an internal `domain.PromptIdentity` with only `ID` and `Version` strings. + Keep it separate from the eventual public declaration; no public type alias. +2. Add private profile-ID and prompt-identity slices to `loaded`. Populate them + once at the end of `load`, after both `materializePrompts` and + `buildExecutionProfiles` have succeeded. Enumerate the finished + `indexedProfiles` keys and the finished `prompts` ID/version keys, not + ConfigKit metadata or raw contributions. +3. Build the ordered slices with existing `sortedKeys` or standard-library + sorting. Flatten actual prompt versions; an ID with no versions contributes + no prompt identity. Store only strings, not targets, definitions, or histories. +4. Add `ListProfiles(context.Context) ([]string, error)` and + `ListPrompts(context.Context) ([]domain.PromptIdentity, error)` on `loaded`, + in a focused `internal/configuration/listing.go`. Check context, copy the + captured slice, check context again, and return the independent result. + Ordinary slice copying suffices because elements contain only strings. + Reads are linear in returned inventory size and require no repeated sorting. +5. Add adapter tests using the real `Load` and existing mapping helpers. Cover + a small unsorted profile/prompt fixture, multiple versions such as `"10"` + and `"2"`, literal whitespace/slash keys, a partial profile, and an empty + load. Mutate returned entries and verify subsequent lists are unchanged. + Cover an already-canceled context for each method without timing assertions. + +Do not change existing lookup methods or add inventory mutation. Keep +construction complexity proportionate; there is no benchmark gate. + +**Verify:** format changed files; run +`go test ./internal/configuration ./internal/domain` and +`go vet ./internal/configuration ./internal/domain`. + +**Done when:** successful loads retain sorted identity-only inventories and +return owned lists without rereading ConfigKit or requiring runnable targets. + +## Stage 2 — Wire The Public Facade And Its Basic Contract + +**Status: Pending.** Depends on Stage 1. + +**Scope:** `internal/engine`, root API/conversion, and focused facade tests. + +1. Add a private, consumer-owned inventory interface in `internal/engine` + containing just the two listing methods from Stage 1. Store the loaded + snapshot through this interface during `New`, alongside the existing + operator inspection view. Delegate new internal engine methods to it. + There is no need to extend execution repository interfaces or the use-case + runner, add optional capabilities, or provide a full-inspection fallback. +2. Add `listing.go` at the module root, containing public `PromptIdentity` and + the two methods. Apply the same `e == nil || e.internal == nil` check as + existing inspection methods, returning `ErrInvalidConfig` with useful + context. Map delegated errors through `mapPublicError`. +3. Return the already-owned profile slice. Convert internal prompt identities + into a separate public slice by copying their two strings. Check context + after conversion so a detected cancellation yields no partial result. + Do not export an internal type in any public signature. +4. Write concise GoDoc covering effective membership, exact lexical ordering, + all prompt versions, captured-state behavior, caller ownership, concurrent + use, empty success, nil-engine and context errors, and absence of readiness + checks or external work. Identifiers may be sensitive application data; + do not describe these lists as automatically safe for disclosure. +5. Add external-package basic tests in `listing_contract_test.go`: a usable + public engine lists a configured profile and two exact prompt versions; + returned identities can be inspected; nil/zero engines match + `ErrInvalidConfig`; canceled and expired contexts preserve context identity + with no partial inventory. Use real constructors and synthetic configuration. + +**Verify:** format changed files; run +`go test ./internal/engine .` and `go vet ./internal/engine .`. +Existing root boundary tests must continue to pass; do not weaken them. + +**Done when:** consumers can call both methods through the root facade and +receive independent public values with the selected error behavior. + +## Stage 3 — Verify Effective Membership, Ownership, And Concurrent Use + +**Status: Pending.** Depends on Stage 2. + +**Scope:** focused additions to the external-package listing contract tests. +Production edits are limited to defects those tests reveal. + +1. Build one representative layered engine with a consumer base profile, an + inherited child, an overlay of the same profile ID, and multiple prompt + versions. Verify effective identities, no duplicate overlay entry, and + child/base identities remaining distinct. Inspect one listed identity to + confirm it resolves to the effective captured definition. +2. Verify maintained profiles are included without asserting a complete catalog + inventory or a specific upstream model. The two PromptKit-owned standard + profile names may serve as stable membership examples under the existing + release policy; other expectations should use fixture-controlled identities. +3. Exercise empty effective inventories through a public engine: a final main + mapping with `profiles: null` and `prompts: null` clears the sections above + the catalog underlays. Assert successful zero-length lists, not a particular + nil/empty representation. This also checks that lower-layer historical + profile paths are not returned. Use existing ConfigKit semantics; add no + source-validation pass or special listing-only removal rule. +4. Mutate elements in both public results and relist to verify independent + storage across calls. Add one bounded concurrent test that lists and mutates + independent results while other goroutines list/inspect the same captured + engine. Coordinate deterministically; avoid sleeps or shared test-owned + mutable result slices. +5. Cover snapshot/no-execution behavior with a compact fixture: construct from + an application filesystem, then remove/change its content after construction + and verify listing still reports the captured identities. Inject existing + model-client and artifact-reader stubs that fail if invoked. Include a + partial profile or a named but unavailable API-key environment variable; + listing succeeds without provider readiness. Do not add new extension points + or exact private-call-count assertions solely for these tests. + +Place effective-loading/ordering mechanics in Stage 1's adapter tests and +public assembled outcomes here. Avoid duplicating every loader, inheritance, +or existing inspection case. No execution/network call is required to prove +identity discovery. + +**Verify:** `go test -race ./internal/configuration ./internal/engine .`. + +**Done when:** public tests demonstrate effective—not historical—membership, +independent results, concurrent safety, and listing without execution work. + +## Stage 4 — Document And Verify The Completed Feature + +**Status: Pending.** Depends on Stage 3. + +**Scope:** owning documentation, final review, and complete offline verification. + +1. Update `doc.go` to include listing in its operation and concurrent-use + summaries, and `PromptIdentity` among values without stable JSON contracts. + Keep exact behavior in the method/type GoDoc from Stage 2. +2. Add a short consumer-guide discovery example listing identities and then + inspecting a selected entry. Distinguish listing from configuration history + and from execution readiness; link to the exact GoDoc. A new runnable example + project or CLI is unnecessary. +3. Update `docs/internal/sources.md` and `docs/internal/overview.md` for captured + inventories and direct internal-engine delegation. The execution runner has + not acquired a listing responsibility. Add a brief format-reference link to + the listing guidance where it currently introduces inspection, without + changing configuration semantics. +4. Repair old `implementation.md` links in the completed `performance.md`, + `inspection.md`, and historical sections of `remediation.md` to their + respective history links at the top of this plan. Preserve finding statuses + and recorded evidence; do not retire unrelated roadmaps or imply their + work remains pending. This is navigation repair, not another feature. +5. Review the entire feature diff against the listing roadmap and selected + contract. Confirm only effective indexes are used; no runtime imports of + ConfigKit, provenance traversal, readiness checks, resource reads, new + limits, or stable serialization promises have been introduced. +6. Run the complete offline race suite and full vet, plus maintained examples + if changed or affected. Validate changed documentation links and + `git diff --check`. Reuse successful focused checks for unchanged content. +7. Mark the listing roadmap and this plan complete with concise verification + evidence. Commit the completed unit. Leave release notes, roadmap retirement, + tags, and publication for the later release-preparation task. + +Use the existing local Go installation and prepared module cache. Suggested +final commands (not new release gates): ```sh -go test -race -vet=off -mod=readonly ./... -go vet -mod=readonly ./... -python3 scripts/check-examples.py +GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \ + go test -race -vet=off -mod=readonly -count=1 ./... +GOTOOLCHAIN=local GOWORK=off GOPROXY=off go vet -mod=readonly ./... git diff --check ``` -Check local Markdown links and changed-file formatting using existing tooling -or an appropriate focused check. Reuse successful checks for unchanged content -rather than rerunning merely to create the final commit. - -## Execution Evidence - -### Inspection Baseline - -The benchmark-only fixture is committed at `04d5d695e0201a97c1070e1ff49ef4f0c9a7f5ba` -(`Add requirement inspection benchmarks`). The configuration benchmarks build -one captured snapshot per case before timing and measure the existing prompt -snapshot and owned-profile lookups. The public benchmark builds one real Engine -and both requests before timing, then calls only `InspectRequirements` with -the caller variable supplied. None of the paths reads artifacts or schemas or -contacts a provider. The small fixture has two provider-option entries; the -larger fixture has 64 nested provider maps, each with mutable label/route slices -and nested limit/route maps. Prompt lookup allocation is independent of profile -extra size because it does not copy the selected profile. The profile lookup -and assembled requirement inspection expose the mutable-map copy cost being -measured. Environment: Linux/amd64, Intel Core i5-7260U @ 2.20 GHz, benchmark -parallelism 4, Go 1.26.8, ConfigKit `v0.5.0`, and JSON Schema `v6.0.2`. - -Command (five 500 ms samples per case, without the race detector): - -```sh -GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \ - go test -mod=readonly -run '^$' \ - -bench 'BenchmarkInspectionSelection|BenchmarkRequirementInspection' \ - -benchmem -benchtime=500ms -count=5 ./internal/configuration . -``` - -Times list all five `ns/op` samples in command order, followed by median and -range. Allocation figures are `B/op; allocs/op`. - -| Configuration lookup | `ns/op` samples (median; range) | Allocations | -| --- | --- | --- | -| Prompt, small extras | 356.8, 354.2, 370.1, 352.8, 364.5 (356.8; 352.8–370.1) | 400; 4 | -| Prompt, nested extras | 423.2, 478.8, 456.2, 436.6, 460.4 (456.2; 423.2–478.8) | 400; 4 | -| Profile, small extras | 18785, 21077, 18952, 19701, 18682 (18952; 18682–21077) | 9538; 211 | -| Profile, nested extras | 624104, 635513, 600281, 600667, 620228 (620228; 600281–635513) | 261024–261028; 5979 | - -| Public requirement inspection | `ns/op` samples (median; range) | Allocations | -| --- | --- | --- | -| Small extras | 23022, 25193, 24687, 24974, 23157 (24687; 23022–25193) | 11922; 240 | -| Nested extras | 635726, 642297, 591992, 605001, 590312 (605001; 590312–642297) | 263370–263372; 6008 | - -The prompt snapshot lookup remained at 400 B and four allocations in both -fixtures, while owned profile lookup rose from about 9.5 KB/211 allocations to -about 261 KB/5979 allocations. Public requirement inspection rose from about -11.9 KB/240 allocations to about 263 KB/6008 allocations. These results -establish a baseline in which unrelated nested provider extras dominate the -large profile-selection and requirement-inspection cases; they are local -allocation measurements, not end-to-end provider-latency claims. Fixture -identity and prompt/requirement inventories were checked outside timing. - -### Lightweight Inspection Lookups Comparison - -The before measurements are the Stage 7 benchmark-only baseline at `04d5d69`; -the after measurements use the same fixtures at `b7ff371` (`Add lightweight -configuration lookups`). The command, environment, and five-sample procedure -are those in the Inspection Baseline subsection. Each after-case also checks -the selected identity or requirement/input inventory outside timing. The new -prompt-inspection cases return copied metadata without execution templates; -the profile-identity cases use the same two-entry and 64-entry nested-extra -snapshots as the full profile cases. - -| Configuration path | Before `ns/op` (median; range) | Before allocations | After `ns/op` (median; range) | After allocations | -| --- | --- | --- | --- | --- | -| Prompt snapshot, small extras | 356.8, 354.2, 370.1, 352.8, 364.5 (356.8; 352.8–370.1) | 400; 4 | 292.6, 272.5, 332.9, 277.4, 302.3 (292.6; 272.5–332.9) | 336; 3 | -| Prompt snapshot, nested extras | 423.2, 478.8, 456.2, 436.6, 460.4 (456.2; 423.2–478.8) | 400; 4 | 359.0, 357.1, 388.8, 356.4, 365.7 (359.0; 356.4–388.8) | 336; 3 | -| Prompt inspection, small extras | — | — | 279.0, 194.3, 199.2, 187.8, 182.3 (194.3; 182.3–279.0) | 208; 2 | -| Prompt inspection, nested extras | — | — | 277.0, 236.8, 253.4, 222.5, 230.3 (236.8; 222.5–277.0) | 208; 2 | -| Full profile, small extras | 18785, 21077, 18952, 19701, 18682 (18952; 18682–21077) | 9538; 211 | 18887, 18555, 18517, 18761, 20126 (18761; 18517–20126) | 9538; 211 | -| Full profile, nested extras | 624104, 635513, 600281, 600667, 620228 (620228; 600281–635513) | 261024–261028; 5979 | 616937, 612152, 666875, 639239, 641841 (639239; 612152–666875) | 261024–261029; 5979 | -| Profile identity, small extras | — | — | 18.62, 18.62, 17.89, 18.57, 20.24 (18.62; 17.89–20.24) | 0; 0 | -| Profile identity, nested extras | — | — | 20.69, 19.83, 18.86, 18.65, 19.22 (19.22; 18.65–20.69) | 0; 0 | - -The prompt snapshot now avoids the unused inspection-input copy, saving 64 B -and one allocation in these fixtures. The dedicated metadata lookup uses -208 B/two allocations for either payload size, and identity lookup uses no -allocation despite the nested provider data. Full owned-profile allocation -counts are unchanged and their before/after time ranges overlap. - -The root benchmark still uses the existing inspection route; these small -changes are from its prompt snapshot dependency, not profile-identity routing: - -| Public requirement inspection | Before `ns/op` (median; range) | Before allocations | After `ns/op` (median; range) | After allocations | -| --- | --- | --- | --- | --- | -| Small extras | 23022, 25193, 24687, 24974, 23157 (24687; 23022–25193) | 11922; 240 | 24320, 24715, 23919, 23911, 23885 (23919; 23885–24715) | 11858; 239 | -| Nested extras | 635726, 642297, 591992, 605001, 590312 (605001; 590312–642297) | 263370–263372; 6008 | 595187, 647166, 675430, 595950, 626551 (626551; 595187–675430) | 263304–263309; 6007 | - -Root requirement-inspection allocation decreased by 64–66 B and one allocation; -the timing ranges overlap. The large profile extras still dominate until the -next routing change. No public method or ConfigKit query was added. Focused -verification passed: `go test ./internal/configuration ./internal/usecase .`; -both benchmark families were rerun without the race detector. - -### Requirement-Inspection Identity Comparison - -The before values use the Stage 7 benchmark-only baseline at `04d5d69`; after -values use the same root fixture at `6c5dec7` (`Inspect requirements by -profile identity`). The command and environment are those in the Inspection -Baseline subsection. The small case has two nested provider-option entries and -the larger case has 64; neither benchmark case accesses a provider, artifact, -or schema. Times list all five `ns/op` samples in command order, followed by -median and range. - -| Requirement-inspection case | Stage 7 `ns/op` (median; range) | Stage 7 allocations | Identity lookup `ns/op` (median; range) | Identity lookup allocations | -| --- | --- | --- | --- | --- | -| Small extras | 23022, 25193, 24687, 24974, 23157 (24687; 23022–25193) | 11922; 240 | 2895, 2794, 2807, 3047, 2876 (2876; 2794–3047) | 2264; 26 | -| Nested extras | 635726, 642297, 591992, 605001, 590312 (605001; 590312–642297) | 263370–263372; 6008 | 2917, 2730, 2797, 2804, 2787 (2797; 2730–2917) | 2264; 26 | - -The 64-entry case reduced cumulative allocation by about 261 KB and 5982 -allocations per inspection; its median fell from about 605 microseconds to -2.8 microseconds. The small case fell from about 11.9 KB/240 allocations to -2.3 KB/26 allocations. These fixture-local results support using identity-only -selection for the information inspection consumes; they do not imply provider -latency improvements. The optional-capability and complete-profile fallback -paths preserve exact requested IDs, profile-load error wrapping, and -nil-profile/nil-target validation. Focused verification passed: -`go test ./internal/configuration ./internal/usecase .`. - -### Prompt-Metadata Inspection Comparison - -The before values use the Stage 9 implementation at `6c5dec7`; after values -use `c5f733a` (`Select prompt metadata for inspection`) with the same inspection -fixtures. The command and environment are those in the Inspection Baseline -subsection: Linux/amd64, Intel Core i5-7260U @ 2.20 GHz, benchmark parallelism -4, Go 1.26.8, ConfigKit `v0.5.0`, JSON Schema `v6.0.2`, and backend modules -`v3.20260930.0`. Five 500 ms samples were collected without the race detector. -Times list all five samples, followed by median and range; allocations are -`B/op; allocs/op`. - -| Public requirement inspection | Stage 9 `ns/op` (median; range) | Stage 9 allocations | Metadata inspection `ns/op` (median; range) | Metadata inspection allocations | -| --- | --- | --- | --- | --- | -| Small extras | 2895, 2794, 2807, 3047, 2876 (2876; 2794–3047) | 2264; 26 | 2681, 2837, 2852, 2978, 2847 (2847; 2681–2978) | 2112; 24 | -| Nested extras | 2917, 2730, 2797, 2804, 2787 (2797; 2730–2917) | 2264; 26 | 2898, 3004, 2955, 4766, 4410 (3004; 2898–4766) | 2112; 24 | - -The metadata route saves 152 cumulative bytes and two allocations per public -requirement inspection in both fixtures. Timing ranges overlap for the small -case; the larger after-run is noisy and does not show a reliable timing change. -The result supports the targeted copy reduction, not a provider-latency claim. -Configuration metadata inspection separately measured 208 B/two allocations -for both small and nested-extra fixtures, independent of their profile payload. -The benchmark command was: - -```sh -GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \ - go test -mod=readonly -run '^$' \ - -bench 'BenchmarkInspectionSelection|BenchmarkRequirementInspection' \ - -benchmem -benchtime=500ms -count=5 ./internal/configuration . -``` - -Verification passed on `c5f733a`: -`go test -race -mod=readonly ./internal/configuration ./internal/usecase .`. - -### Assembled Workflow Benchmark Baseline - -The benchmark-only fixture is committed at `743e78d` (`Add assembled workflow -benchmark baseline`). The same committed source produced the measurements -below. Environment: Linux/amd64, Intel Core i5-7260U @ 2.20 GHz, benchmark -parallelism 4, Go 1.26.8, ConfigKit `v0.5.0`, JSON Schema `v6.0.2`, and backend -modules `v3.20260930.0`. - -Command (five 500 ms samples per case, without the race detector): - -```sh -GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \ - go test -mod=readonly -run '^$' \ - -bench '^(BenchmarkEnginePreparation|BenchmarkEngineRun|BenchmarkPreparedExecutionDetails)$' \ - -benchmem -benchtime=500ms -count=5 . -``` - -All cases use one assembled engine constructed before timing and a deterministic -injected client; there is no live HTTP or credential lookup. The small prompt -renders `Hello Ada`. The large-message prompt has a 256 KiB literal prefix and -renders that prefix plus a short interpolated suffix. The mutable-payload -prompt uses 64 nested provider-option entries, each with labels, limits, and -route maps/slices, plus a local JSON Schema with 256 optional string -properties and `additionalProperties: false`. Preflight confirms that the -injected `{}` response passes the real schema validator; the `Run` benchmark -also measures that validation. The schema, configuration, and request fixtures -are created before each timer starts. - -`BenchmarkEnginePreparation` measures ordinary `Prepare` separately from -`PrepareExecution` creation. The prepared-execution case discards each unused -handle inside its timed iteration and retains only the most recent discarded -handle pointer. `BenchmarkPreparedExecutionDetails` creates and validates one -ready handle outside timing, measures repeated public `Details()` calls on it, -retains only the latest caller-owned result, then discards the handle. Ordinary -prepare and run sinks are overwritten per iteration and cleared after timing. -The large-message result shares immutable string content, so its allocation -figures are not treated as deep-copy bytes. No benchmark retains every -iteration or includes engine construction/provider latency. - -Times list the five `ns/op` samples in command order, followed by median and -range. Allocation figures are `B/op; allocs/op` (ranges where they varied). - -| Benchmark case | `ns/op` samples (median; range) | Allocations | -| --- | --- | --- | -| Prepare, small | 10935, 10121, 13018, 10288, 8485 (10288; 8485–13018) | 4513; 50 | -| Prepare, large message | 2064223, 1627980, 1588515, 1008441, 911123 (1588515; 911123–2064223) | 872964–872988; 50 | -| Prepare, mutable payload | 6430306, 6519806, 5970344, 5854621, 6456738 (6430306; 5854621–6519806) | 2914750–2914824; 41134–41135 | -| Prepared execution creation + discard, small | 8194, 8016, 8095, 8073, 7939 (8073; 7939–8194) | 5210; 56 | -| Prepared execution creation + discard, large message | 937857, 955672, 919810, 1058503, 1094114 (955672; 919810–1094114) | 873679–873681; 56 | -| Prepared execution creation + discard, mutable payload | 8114785, 8394320, 10387083, 8418604, 8528766 (8418604; 8114785–10387083) | 3711282–3711329; 60727 | -| Run, small | 12157, 11921, 12128, 13762, 12491 (12157; 11921–13762) | 6091; 66 | -| Run, large message | 1082064, 1058363, 1040185, 996377, 951818 (1040185; 951818–1082064) | 874562–874577; 66–67 | -| Run, mutable payload | 6072294, 6338179, 6386632, 7167500, 6805715 (6386632; 6072294–7167500) | 2993599–2993661; 41808–41809 | -| Details, small | 1104, 1039, 1010, 1045, 1058 (1045; 1010–1104) | 1056; 6 | -| Details, large message | 975.1, 2480, 4734, 3402, 1008 (2480; 975.1–4734) | 1056; 6 | -| Details, mutable payload | 2275096, 3281027, 1504724, 1365349, 1329800 (1504724; 1329800–3281027) | 670315–670339; 11548 | - -The 256 KiB message does not make `Details()` allocate 256 KiB per call; -current public conversion shares immutable message strings while copying the -message slice and other metadata. By contrast, `Details()` over the mutable -payload costs about 670 KB and 11.5 thousand allocations per call, providing a -specific snapshot-copy baseline for later ownership analysis. Large-message -timings, especially `Details()`, are noisy and are retained as ranges rather -than point claims. Focused verification passed: `go test -mod=readonly .`; -representative rendered messages, prepared details, generated output, and -structured validation were checked outside benchmark timing. - -### Prepared Snapshot Ownership And Live-Heap Baseline - -The benchmark-only baseline is committed at `11ecb92` (`Add prepared snapshot -ownership benchmarks`). `BenchmarkPreparedSnapshot` exercises the internal -runner with the existing fake prompt/profile repositories, renderer, artifact -reader, a stateless injected client, and a fast prepared validator. It measures -`PrepareExecution` and discard without compiling a schema. Its small fixture -uses one rendered message, no provider extras, and a one-property schema. The -mutable fixture uses eight rendered messages plus one appended message, cache -metadata on each, one input/hash, 64 nested provider-option entries, and a -256-property schema. Fixture checks run outside timing; each measured handle is -discarded and only the latest discarded handle is retained as a sink. - -`BenchmarkRetainedPreparedHandles` uses the mutable fixture and a fixed batch -of four handles. After fixture setup and checks, it preallocates an empty handle -slice, forces GC, and records a setup-only `HeapAlloc` baseline. It then retains -four ready, completed, or discarded handles, forces GC again, and reports the -heap delta. The retained benchmark command uses `-benchtime=1x`; the single -timed operation is followed by filling the remaining batch slots outside the -timer. Both samples call `runtime.KeepAlive` for the runner, request, and -retained batch. The metric is a live-heap difference, not a derivation from -`B/op`. - -Environment: Linux/amd64, Intel Core i5-7260U @ 2.20 GHz, benchmark parallelism -4, Go 1.26.8, ConfigKit `v0.5.0`, JSON Schema `v6.0.2`, and backend modules -`v3.20260930.0`. - -Snapshot command (five 500 ms samples, without the race detector): - -```sh -GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \ - go test -mod=readonly -run '^$' -bench '^BenchmarkPreparedSnapshot$' \ - -benchmem -benchtime=500ms -count=5 ./internal/usecase -``` - -Times list the five `ns/op` samples followed by median and range; allocation -figures are `B/op; allocs/op` (ranges where they varied). - -| Snapshot case | `ns/op` samples (median; range) | Allocations | -| --- | --- | --- | -| Small | 45776, 51524, 46933, 46834, 44663 (46834; 44663–51524) | 13014–13016; 216 | -| Mutable payload | 5343223, 5234718, 4213550, 4419520, 5046181 (5046181; 4213550–5343223) | 1211931–1212104; 25243–25244 | - -Retained-heap command (five fixed-batch samples): - -```sh -GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \ - go test -mod=readonly -run '^$' -bench '^BenchmarkRetainedPreparedHandles$' \ - -benchmem -benchtime=1x -count=5 ./internal/usecase -``` - -| Retained handle state | `retained-B` samples (median; range) | Timed-operation allocations | -| --- | --- | --- | -| Ready | 1458352, 1458256, 1459320, 1459048, 1459160 (1459048; 1458256–1459320) | 1211896–1213256 B; 25242–25251 | -| Completed | 731040, 730768, 729352, 729448, 729624 (729624; 729352–731040) | 1213024–1214368 B; 25255–25264 | -| Discarded | 728560, 730928, 730544, 730496, 730496 (730496; 728560–730928) | 1211896–1213256 B; 25242–25251 | - -The ready four-handle batch retains about 1.46 MB above setup; completed and -discarded batches retain about 0.73 MB. The roughly 0.73 MB difference is -consistent with the extra execution snapshot held only while a handle is -ready. After claim/run or discard, `payload.clear` removes that execution -payload while the independent details snapshot remains available. Heap deltas -include runtime/GC noise; the five observed ranges are reported rather than -treated as an exact object-size guarantee. - -The source ownership inventory is: - -- `completePreparationWithStructuredOutput` builds a fresh input-hash map and - obtains rendered messages through `domain.ConcatRenderedMessages`, which - copies message slices and each `CacheControl` pointer while sharing immutable - string contents. -- `structuredOutputFromValidationPlan` copies the validation plan's schema - document with `jsonvalue.Copy` before storing it in the prepared run. -- `resolveExecutionTarget` and `mergeExecutionTargetOverride` use the - shallow-copy helper `copyExtraParams`. The configuration repository's full - `GetProfile` and the public request conversion deep-copy extra values, but - internal profile fakes return only shallow struct copies and internal - `RunRequest` override maps may be source-owned. Nested extras can therefore - remain aliased through the runner's outer-map copy. Stage 13 must keep a - `jsonvalue.CopyMap` boundary before transferring the prepared run. -- `PrepareExecution` deep-clones `operation.run` for execution, then clones - that snapshot again for details. `Details()` deep-clones details per call; - the root facade conversion also creates caller-owned maps, schema, messages, - and cache metadata. Claim completion and `Discard` clear only the private - payload, so the details copy survives both lifecycles. -- The injected-client adapter converts generated requests to caller-owned - prompt messages/cache metadata, execution-target extra maps, and structured - schema values. Its client cannot mutate the private prepared snapshot through - a generation request. - -The assembled public `Details()` benchmark independently measured 670315–670339 -B and 11548 allocations for one mutable-payload details call. The internal -snapshot benchmark isolates creation with fakes; together these baselines show -that mutable schema/extras copies dominate the small message-slice cost. The -Stage 13 candidate can remove the execution clone of preparation-owned -messages, hashes, and schema, but must retain a deep copy of extras and the -separate details clone. Focused verification passed: `go test -mod=readonly -./internal/usecase .`; representative content, copied schema, fake validation -status, and post-completion details were checked outside timing. - -### Prepared Creation Snapshot Comparison - -The before measurements use the Stage 11 assembled benchmark fixture committed -at `743e78d` and the Stage 12 ownership fixture committed at `11ecb92`. The -after measurements use the Stage 13 candidate in this completion. Both runs -used the same fixtures and environment: Linux/amd64, Intel Core i5-7260U @ -2.20 GHz, benchmark parallelism 4, Go 1.26.8, ConfigKit `v0.5.0`, JSON Schema -`v6.0.2`, and backend modules `v3.20260930.0`. Benchmarks ran without the race -detector. The commands are those in the Assembled Workflow Benchmark Baseline -and Prepared Snapshot Ownership And Live-Heap Baseline subsections above. - -The assembled creation benchmark measures `PrepareExecution` followed by -discarding its unused handle. Times list five `ns/op` samples, then median and -range; allocations are `B/op; allocs/op`. - -| Case | Before `ns/op` (median; range) | Before allocations | After `ns/op` (median; range) | After allocations | -| --- | --- | --- | --- | --- | -| Small | 8194, 8016, 8095, 8073, 7939 (8073; 7939–8194) | 5210; 56 | 11028, 11059, 11756, 12279, 11089 (11089; 11028–12279) | 4665; 53 | -| Large message | 937857, 955672, 919810, 1058503, 1094114 (955672; 919810–1094114) | 873679–873681; 56 | 1751124, 1647596, 1761392, 1556071, 1584325 (1647596; 1556071–1761392) | 873135–873144; 53 | -| Mutable payload | 8114785, 8394320, 10387083, 8418604, 8528766 (8418604; 8114785–10387083) | 3711282–3711329; 60727 | 12157300, 12363416, 12185745, 11868982, 21225784 (12185745; 11868982–21225784) | 3483089–3483252; 56324–56326 | - -Small and large cases each save about 0.5 KB and three allocations, while the -mutable case saves about 228 KB and 4,401–4,403 allocations. The after timing -samples in this run are higher than the recorded before samples, while other -repetitions on the same machine varied substantially; the timing evidence -does not establish a reliable latency improvement or regression. The ordinary -`Prepare`, `Run`, and public `Details()` allocation counts are unchanged within -their recorded baseline ranges; this candidate only changes prepared-execution -creation. The `B/op` and `allocs/op` values are cumulative allocation per -operation; the separate `retained-B` table is a post-GC live-heap delta for four -handles. They measure different effects and are not inferred from one another. - -The focused internal snapshot benchmark isolates the same creation path using -the Stage 12 fake-collaborator fixture. Times again list five samples, median, -and range. - -| Snapshot case | Before `ns/op` (median; range) | Before allocations | After `ns/op` (median; range) | After allocations | -| --- | --- | --- | --- | --- | -| Small | 45776, 51524, 46933, 46834, 44663 (46834; 44663–51524) | 13014–13016; 216 | 50907, 79812, 62186, 65569, 62599 (62599; 50907–79812) | 9405–9406; 156 | -| Mutable payload | 5343223, 5234718, 4213550, 4419520, 5046181 (5046181; 4213550–5343223) | 1211931–1212104; 25243–25244 | 5991662, 5508812, 6244981, 5176459, 4365122 (5508812; 4365122–6244981) | 983033–983309; 20837–20839 | - -The internal fixture saves about 3.6 KB/60 allocations for the small case and -about 229 KB/4,404–4,407 allocations for the mutable case. Timing samples vary -between repeated runs and do not establish a consistent speedup or regression. - -The retained-heap comparison uses the documented fixed batch of four mutable- -payload handles. These are five independently repeated `retained-B` samples -after the candidate; before samples are recorded in the baseline subsection. - -| Handle state | Before samples (median; range) | After samples (median; range) | -| --- | --- | --- | -| Ready | 1458352, 1458256, 1459320, 1459048, 1459160 (1459048; 1458256–1459320) | 1458936, 1459048, 1460592, 1460464, 1460192 (1460192; 1458936–1460592) | -| Completed | 731040, 730768, 729352, 729448, 729624 (729624; 729352–731040) | 731312, 730496, 730768, 730544, 731040 (730768; 730496–731312) | -| Discarded | 728560, 730928, 730544, 730496, 730496 (730496; 728560–730928) | 731040, 731088, 730720, 730768, 730608 (730768; 730608–731088) | - -All before/after retained ranges overlap. The median shifts are about 1.1 KB -for ready and completed handles and 272 B for discarded handles, all small -relative to their live sets and within observed run-to-run variation. This is -not evidence of a material retention increase, nor an exact object-size -guarantee. Together with the measured allocation reduction and mutation tests, -it supports retaining the simple ownership transfer while keeping the nested -extra-map copy, details clone, lifecycle cleanup, and caller-owned conversions. -Focused validation passed: `go test -race -mod=readonly ./internal/usecase .`. - -### Validation And Compiled-Rendering Baseline - -The benchmark-only baseline is commit -`0eb009ad5a3248e2a1e12cad9b3a05906d6b0850` (`Add validation and rendering -benchmarks`), on top of planning baseline -`3fde1f6cbbc2cc5b3dc17779ba44e0af6cd8a836`. The measured source is unchanged -from that commit. No production files changed. - -Environment: Linux/amd64, Intel Core i5-7260U @ 2.20 GHz, benchmark parallelism -4; `go1.26.8`, module directive `go 1.26.0`, ConfigKit `v0.5.0`, JSON Schema -`v6.0.2`, and the backend modules `v3.20260930.0`. - -Command (five 500 ms samples per benchmark, without the race detector): - -```sh -GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \ - go test -mod=readonly -run '^$' \ - -bench 'Benchmark(BasicValidation|CompiledRendering|RetainedRenderedOutput|GoRendererArtifactReferences)$' \ - -benchmem -benchtime=500ms -count=5 \ - ./internal/validate ./internal/prompt -``` - -Times below list all five `ns/op` samples in command order, followed by median -and range. Allocation figures are `B/op; allocs/op` (ranges where they varied). - -| Benchmark case | `ns/op` samples (median; range) | Allocations | -| --- | --- | --- | -| Basic validation, short text | 46.59, 44.98, 43.31, 46.23, 43.19 (44.98; 43.19–46.59) | 0; 0 | -| Basic validation, 1 MiB nonempty | 193859, 329884, 418896, 386126, 378173 (378173; 193859–418896) | 1048576–1048577; 1 | -| Basic validation, ~1 MiB whitespace | 1112498, 1100439, 1175913, 1290678, 1108746 (1112498; 1100439–1290678) | 1048592–1048595; 2 | -| Basic validation, Unicode whitespace | 170.8, 164.2, 158.6, 159.2, 165.2 (164.2; 158.6–170.8) | 16; 1 | -| Compiled rendering, small message | 3931, 8488, 2041, 1902, 2181 (2181; 1902–8488) | 1632; 20 | -| Compiled rendering, 256 KiB artifact message | 285812, 380128, 655925, 664387, 537684 (537684; 285812–664387) | 1050656–1050713; 25 | -| Compiled rendering, repeated across session/messages | 1870972, 1362368, 1142268, 1073474, 1173588 (1173588; 1073474–1870972) | 3945983–3946034; 74 | -| `NewGoRenderer` fallback, one artifact reference | 407143, 629149, 522549, 562200, 470393 (522549; 407143–629149) | 1056409–1056477; 69–70 | -| `NewGoRenderer` fallback, repeated references | 1439883, 1535813, 1300176, 1541795, 1416267 (1439883; 1300176–1541795) | 2897171–2897255; 157 | -| Retained output, four 256 KiB rendered results | 398753, 392206, 384503, 380386, 406691 (392206; 380386–406691) | 1050702–1050724; 25–26 | - -For the retained-output benchmark, post-GC live-heap deltas above the same -setup-only fixture were 1048272, 1050176, 1046032, 1049856, and 1049016 bytes -(median 1049016; range 1046032–1050176). The batch contains four separately -rendered 256 KiB messages. This is retained-heap evidence, not an inference -from `B/op`; it includes result structure and allocator/runtime noise. The -initial exploratory run exposed cross-count sink retention in the measurement -baseline; the benchmark now clears prior sinks before measuring setup, and the -recorded values are from the corrected committed fixture. - -The short validator case allocates nothing, while the 1 MiB nonempty body has -about one response-sized allocation; the whitespace-only case also retains a -small failure-result allocation. Compiled rendering provides separate small, -artifact-sized, and repeated-reference baselines, with the existing parsing -fallback reported for context. Several render timings show wide ranges on this -machine, so the median/range—not any single run—are the comparison baseline. -The retained-output results consistently remain near 1 MiB for four outputs. - -Focused verification passed: `go test -mod=readonly ./internal/validate -./internal/prompt`. Record later before/after comparisons, ownership analysis, -conditional decisions, and final verification in this section as the remaining -stages complete. Keep the feature roadmap focused on target state and progress -rather than duplicating the staged procedure. - -### Basic-Validation Copy Comparison - -The before measurement uses the unchanged Stage 1 benchmark fixture at commit -`05fe13d` (`Record validation and rendering baselines`), immediately before the -production change. The after measurement uses the same fixture and benchmark -implementation at `ed7c273` (`Avoid copying output during basic validation`). -Environment and module versions match the baseline above: Linux/amd64, Intel -Core i5-7260U @ 2.20 GHz, parallelism 4, Go 1.26.8, ConfigKit `v0.5.0`, JSON -Schema `v6.0.2`, and backend modules `v3.20260930.0`. - -Command for both revisions (five 500 ms samples, without the race detector): - -```sh -GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \ - go test -mod=readonly -run '^$' -bench '^BenchmarkBasicValidation$' \ - -benchmem -benchtime=500ms -count=5 ./internal/validate -``` - -Times list all five `ns/op` samples in command order, followed by median and -range. Allocation figures are `B/op; allocs/op`. - -| Benchmark case | Before `ns/op` (median; range) | Before allocations | After `ns/op` (median; range) | After allocations | -| --- | --- | --- | --- | --- | -| Short text | 45.39, 45.10, 42.20, 44.19, 44.21 (44.21; 42.20–45.39) | 0; 0 | 41.80, 41.19, 39.90, 41.02, 39.18 (41.02; 39.18–41.80) | 0; 0 | -| 1 MiB nonempty text | 216006, 220902, 226243, 228846, 184748 (220902; 184748–228846) | 1048576–1048577; 1 | 39.76, 42.16, 41.61, 40.79, 40.92 (40.92; 39.76–42.16) | 0; 0 | -| ~1 MiB whitespace-only text | 766108, 709398, 731226, 915411, 787797 (766108; 709398–915411) | 1048592–1048594; 2 | 558919, 562202, 561004, 555487, 564593 (561004; 555487–564593) | 16; 1 | -| Unicode whitespace | 126.6, 113.2, 116.7, 111.6, 115.2 (115.2; 111.6–126.6) | 16; 1 | 115.1, 116.9, 106.1, 111.2, 110.9 (111.2; 106.1–116.9) | 16; 1 | - -The 1 MiB nonempty case's median fell from 220902 ns/op to 40.92 ns/op while -response-sized allocation fell from 1048576–1048577 bytes to zero. The -whitespace case still scans the body and returns a small validation failure; -its response-sized temporary string is gone, reducing allocation from about -1 MiB and two allocations to 16 bytes and one allocation. Small-input timings -and the Unicode case remain in the same order of magnitude. The benchmark no -longer reports bytes per second because the nonempty fast path can classify a -body from its boundary bytes without scanning the full body; latency and -allocation comparisons remain meaningful. - -Focused verification passed on the implementation commit: -`go test -mod=readonly ./internal/validate ./internal/usecase`. The benchmark -family above was rerun on that commit with the recorded command. - -### Compiled-Rendering And Retained-Output Comparison - -The maintained Stage 1 benchmark fixture was measured before the rendering -change at `dd89be3` (after the basic-validation change, which does not affect -rendering), then again after replacing the message and session buffers in the -working tree based on that revision. Setup and fixture contents are unchanged. -Environment: Linux/amd64, Intel Core i5-7260U @ 2.20 GHz, benchmark parallelism -4, Go 1.26.8, ConfigKit `v0.5.0`, JSON Schema `v6.0.2`, and backend modules -`v3.20260930.0`. - -Command for both runs (five 500 ms samples per benchmark, without the race -detector): - -```sh -GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \ - go test -mod=readonly -run '^$' \ - -bench '^(BenchmarkCompiledRendering|BenchmarkRetainedRenderedOutput)$' \ - -benchmem -benchtime=500ms -count=5 ./internal/prompt -``` - -Times list all five `ns/op` samples in command order, followed by median and -range. Allocation figures are `B/op; allocs/op`. - -| Compiled-render case | Before `ns/op` (median; range) | Before allocations | After `ns/op` (median; range) | After allocations | -| --- | --- | --- | --- | --- | -| Small message | 2066, 1906, 2003, 2027, 2010 (2010; 1906–2066) | 1632; 20 | 1912, 2030, 1877, 2045, 1926 (1926; 1877–2045) | 1568; 20 | -| Artifact-backed 256 KiB message | 279116, 341635, 285472, 348345, 341764 (341635; 279116–348345) | 1050713–1050718; 25 | 165860, 239221, 255435, 223641, 214475 (223641; 165860–255435) | 788485–788489; 24 | -| Repeated across session and messages | 966012, 1073914, 1098700, 1154416, 1064160 (1073914; 966012–1154416) | 3946015–3946038; 74 | 778584, 777091, 888988, 739473, 802395 (778584; 739473–888988) | 2446632–2446651; 71–72 | - -The retained-output workload holds four separately rendered 256 KiB results -and reports post-GC live-heap change over the same setup-only fixture. Its -allocation and retained-heap samples were: - -| Measurement | Before | After | -| --- | --- | --- | -| `ns/op` (median; range) | 331625, 333337, 343766, 302864, 329355 (331625; 302864–343766) | 244284, 239994, 238773, 239356, 264424 (239994; 238773–264424) | -| `B/op; allocs/op` | 1050696–1050708; 25 | 788486–788495; 24 | -| Retained heap (`retained-B`, median; range) | 1049744, 1046160, 1052768, 1044928, 1047264 (1047264; 1044928–1052768) | 1045824, 1054000, 1048288, 1047504, 1047168 (1047504; 1045824–1054000) | - -The artifact-backed case removes about 256 KiB of allocation per render, and -the repeated case removes about 1.5 MiB; both also lose one allocation. Small -rendering changes by 64 bytes and 0 allocations. The retained-heap ranges -overlap, with medians differing by only 240 bytes for four 256 KiB outputs, -so the returned strings keep approximately the same live output storage rather -than retaining an additional builder buffer. This is expected: the builder's -backing storage is the returned string instead of being copied into a second -allocation. Measurements support retaining the simpler local-builder change -without a material retention trade-off; the existing per-render artifact-text -conversion and cache remain unchanged. - -Focused verification passed: `go test -race -mod=readonly ./internal/prompt`, -including exact message/session output, cancellation, artifact-text lifecycle, -and a new concurrent-render case that verifies independent request variables, -artifact text, and session IDs against shared compiled templates. - -### Artifact File-Read Capacity Comparison - -The benchmark-only fixture was committed as `996295d` (`Add artifact file-read -benchmarks`) before the reader change. Before measurements use that revision; -after measurements use the same fixture after the bounded reservation and -post-read compaction were implemented. The real-file cases contain 4 KiB, -1 MiB, and 4 MiB of repeated `x` bytes. The retained case reads a real 256 KiB -file whose opened-file seam reports 16 MiB, retaining four results. Fixture -creation and MIME lookup are outside timing. Environment: Linux/amd64, Intel -Core i5-7260U @ 2.20 GHz, benchmark parallelism 4, Go 1.26.8, ConfigKit -`v0.5.0`, JSON Schema `v6.0.2`, and backend modules `v3.20260930.0`. - -Command for both runs (five 500 ms samples per benchmark, without the race -detector): - -```sh -GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \ - go test -mod=readonly -run '^$' \ - -bench '^(BenchmarkArtifactFileRead|BenchmarkRetainedArtifactFileRead)$' \ - -benchmem -benchtime=500ms -count=5 ./internal/artifact -``` - -Times list all five `ns/op` samples in command order, followed by median and -range. Allocation figures are `B/op; allocs/op`. - -| File-read case | Before `ns/op` (median; range) | Before allocations | After `ns/op` (median; range) | After allocations | -| --- | --- | --- | --- | --- | -| 4 KiB | 37904, 38759, 37073, 36437, 36930 (37073; 36437–38759) | 70513–70514; 12 | 37222, 37414, 36521, 37867, 39988 (37414; 36521–39988) | 70513; 12 | -| 1 MiB | 3576064, 4010278, 3919714, 3918643, 3844238 (3918643; 3576064–4010278) | 4752589–4752677; 23–24 | 3105775, 3055909, 3160450, 3268202, 3079892 (3105775; 3055909–3268202) | 1115066–1115090; 12 | -| 4 MiB | 16818098, 15977605, 16465923, 15753739, 15589001 (15977605; 15589001–16818098) | 20244106–20244130; 33 | 17060165, 16462547, 15995322, 17206316, 16180077 (16462547; 15995322–17206316) | 21177936–21178008; 24–25 | - -The 1 MiB case reduced cumulative allocation from about 4.75 MB to 1.12 MB -and from 23–24 to 12 allocations, with a lower median time. For 4 MiB, the -reservation reduced allocation count from 33 to 24–25 but increased cumulative -allocated bytes by about 0.93 MB; timing ranges overlap. The small-file case -was effectively unchanged. These results support the specified capped hint as -a way to avoid repeated growth for ordinary 1 MiB reads, while showing that -the 1 MiB initial reserve is not a cumulative-byte win for every larger file. - -| Overreported-size retained workload | Before | After | -| --- | --- | --- | -| `ns/op` (median; range) | 1144696, 1073206, 1060135, 1059339, 1073194 (1073194; 1059339–1144696) | 1188822, 1201670, 1199592, 1175553, 1173590 (1188822; 1173590–1201670) | -| `B/op; allocs/op` | 779427–779463; 18 | 1377557–1377574; 17 | -| Retained heap (`retained-B`, median; range) | 1082272, 1082496, 1082944, 1082384, 1082496 (1082496; 1082272–1082944) | 1049728, 1049392, 1049280, 1049504, 1049168 (1049392; 1049168–1049728) | - -For the stale 16 MiB hint and 256 KiB actual body, the bounded reserve adds -about 0.6 MB of cumulative allocation and roughly 0.1 ms to the read, but the -post-read compaction prevents it from increasing live retained storage; the -four-result retained-heap median is about 33 KiB lower than the baseline. This -is intentionally a stale-metadata path, not a normal exact-size read. The -ordinary reader still derives returned bytes, size, and hash from content read -through EOF. Focused verification passed: `go test -mod=readonly -./internal/artifact`; the existing cancellation and nonregular-file checks -remain in the passing suite. - -### Request-Preparation Baseline - -The benchmark and boundary-test fixture is committed at `3f15715` (`Add -request preparation baseline and boundary tests`). The benchmark measures -`PrepareGeneration` without dispatch, with client construction, request -fixtures, and the structured schema created before timing. It uses a synthetic -endpoint and no credential or provider calls. Environment: Linux/amd64, Intel -Core i5-7260U @ 2.20 GHz, benchmark parallelism 4, Go 1.26.8, ConfigKit -`v0.5.0`, JSON Schema `v6.0.2`, and backend modules `v3.20260930.0`. - -Command (five 500 ms samples per case, without the race detector): - -```sh -GOTOOLCHAIN=local GOWORK=off GOPROXY=off GONOPROXY=none GOSUMDB=off \ - go test -mod=readonly -run '^$' -bench '^BenchmarkPrepareGeneration$' \ - -benchmem -benchtime=500ms -count=5 ./internal/llm -``` - -Times list all five `ns/op` samples in command order, followed by median and -range. Allocation figures are `B/op; allocs/op`. - -| Preparation case | `ns/op` samples (median; range) | Allocations | -| --- | --- | --- | -| No extras | 2771, 2873, 2872, 2907, 2885 (2873; 2771–2907) | 1632; 23 | -| Small extras | 4216, 4255, 4440, 4090, 4245 (4245; 4090–4440) | 1928; 32 | -| 1 MiB string extra | 3108655, 3046197, 3165484, 3115831, 3074278 (3108655; 3046197–3165484) | 2147898–2213942; 26–27 | -| Nested JSON extras | 5933, 5803, 5675, 5490, 5611 (5675; 5490–5933) | 2304; 40 | -| Structured output | 6421, 6018, 6479, 6153, 6712 (6421; 6018–6712) | 2632–2633; 47 | - -The baseline covers payload construction and complete JSON encoding for both -ordinary and representative structured requests. The 1 MiB case's higher -allocation volume is expected to include the large string and encoded body; -these measurements are the comparison reference for the following request -encoding change, not a provider-latency estimate. Focused verification passed: -`go test -mod=readonly ./internal/llm`. - -### Request-Preparation Copy Comparison - -The before results above use the Stage 5 benchmark fixture committed at -`3f15715`; the after results below use the unchanged fixture after the client -change in the working tree based on `c63ca53`. Both measurements use the same -offline environment and command recorded in the baseline subsection. `ns/op` -values list all five samples, followed by median and range. Allocation figures -are `B/op; allocs/op`. - -| Preparation case | Before `ns/op` (median; range) | Before allocations | After `ns/op` (median; range) | After allocations | -| --- | --- | --- | --- | --- | -| No extras | 2771, 2873, 2872, 2907, 2885 (2873; 2771–2907) | 1632; 23 | 2849, 2841, 2872, 2746, 2902 (2849; 2746–2902) | 1632; 23 | -| Small extras | 4216, 4255, 4440, 4090, 4245 (4245; 4090–4440) | 1928; 32 | 3772, 3865, 3888, 3804, 3931 (3865; 3772–3931) | 1904; 29 | -| 1 MiB string extra | 3108655, 3046197, 3165484, 3115831, 3074278 (3108655; 3046197–3165484) | 2147898–2213942; 26–27 | 1524191, 1516315, 1537340, 1565157, 1538894 (1537340; 1516315–1565157) | 1140425–1197988; 25 | -| Nested JSON extras | 5933, 5803, 5675, 5490, 5611 (5675; 5490–5933) | 2304; 40 | 4440, 4446, 4389, 4302, 4481 (4440; 4302–4481) | 2016; 32 | -| Structured output | 6421, 6018, 6479, 6153, 6712 (6421; 6018–6712) | 2632–2633; 47 | 6253, 6137, 6267, 6315, 6232 (6253; 6137–6315) | 2632–2633; 47 | - -The 1 MiB extra reduced median preparation time by about half and cumulative -allocation by approximately 1 MB. Small and nested extras also used fewer -bytes and allocations; no-extras and structured-output allocations were -unchanged. Timing ranges are machine-local evidence, not a consumer guarantee. -Invalid nested extras retain `ErrInvalidRequest` and key context before -transport. With an unencodable schema, serializable extras retain -`ErrRequestFailed`; invalid extras retain `ErrInvalidRequest`. Prepared retry -attempts continue to use the frozen body. Verification passed: -`go test -mod=readonly ./internal/llm ./internal/generation`, and the benchmark -family above was rerun without the race detector. - -### Final Coverage And Verification - -Stage 13's ownership-transfer implementation, tests, runner documentation, and -decision evidence are committed at `80aecb6` (`Reuse prepared execution -snapshot ownership`). The completed local scope maps to the accepted roadmap as -follows: - -- Item 1's ConfigKit performance request remains recorded at `d53021e`; no - upstream implementation or dependency update was included. -- Items 2–5 are implemented in validation, rendering, request encoding, and - artifact reading. Their before/after fixtures, measurements, and focused - verification appear in the corresponding execution-evidence sections. -- Item 6's ownership review selected the simpler private-payload transfer in - Stage 13. It retains nested-extra and details copies; source isolation, - invalid-copy classification, allocation savings, and fixed-batch heap - observations are recorded above. -- Item 7's lightweight prompt/profile selection and requirement inspection are - implemented and measured in the inspection evidence sections. Production - compiled rendering and assembled offline preparation/execution benchmarks - are also maintained and recorded. - -The canonical runner document describes the new snapshot ownership mechanics. -The source/validation and model-client documents remain consistent with their -implemented flows; no package responsibility, public API, or format semantics -changed. This plan remains the implementation record, and the performance -roadmap's status now reflects completion rather than proposed work. - -Final verification passed on the Stage 13 code: `go test -race -vet=off --mod=readonly ./...`, `go vet -mod=readonly ./...`, and -`python3 scripts/check-examples.py` (prepare, run, observe). The changed Go -files were `gofmt`-clean, and changed-file Markdown local targets were checked -separately. These checks are local correctness and documentation validation; -they do not add a release gate or imply provider-latency improvement. +**Done when:** all four stages are verified, listing is documented by its +canonical owners, and the feature is ready for ordinary release preparation. ## Open Questions -No consumer-policy decisions remain open. File reservation and encoding error -handling have concrete approaches above. The prepared-copy candidate was -evaluated and retained under the documented ownership and evidence criteria. -No implementation-scope questions remain open. +None. The selected signatures, result shape, ordering, ownership, error +behavior, implementation boundary, and verification scope are specified above. diff --git a/docs/roadmap/listing.md b/docs/roadmap/listing.md index e19cc3f..b990317 100644 --- a/docs/roadmap/listing.md +++ b/docs/roadmap/listing.md @@ -10,8 +10,8 @@ history. This supports application selectors, CLI discovery, and configuration editors while keeping PromptKit's public API small. Listing supplies identities; the existing inspection methods supply details. -A separate implementation plan will define staged work. This roadmap describes -the intended feature, not currently available methods. +The [implementation plan](implementation.md) defines staged work. This roadmap +describes the intended feature, not currently available methods. ## Current Baseline